All In One Custom PostgreSQL For Database Developer Docker Image

$ docker load < bigdata.tgz
$ docker load < etl.tgz
$ docker load < docker_samza.tar.gz
$ docker load < postgresql.tgz
$ docker image lsREPOSITORY                    TAG       IMAGE ID       CREATED        SIZE
jentekllc/postgresql latest 1557485c708b 22 hours ago 1.43GB
jentekllc/etl latest c3b44db4c5d6 8 days ago 6.61GB
jentekllc/bigdata latest c7d3fb6d8221 8 days ago 5.53GB
jentekllc/samza latest e9086c0ddaab 10 days ago 12.3GB
$ tar -xzf additional_files_20211125.tar.gz
$ nohup docker-compose -p j up --scale spark-worker=3 &
$ docker psea6ccaad5924   nginx:latest                  "/docker-entrypoint.…"   9 minutes ago   Up 9 minutes   80/tcp, 0.0.0.0:5000->5000/tcp                                                                                                                            nginx-lb
0ae10942def2 jentekllc/bigdata:latest "/run_sshd_worker.sh" 9 minutes ago Up 9 minutes 22/tcp, 0.0.0.0:50000->38080/tcp j_spark-worker_3
9075364e60f3 jentekllc/bigdata:latest "/run_sshd_worker.sh" 9 minutes ago Up 9 minutes 22/tcp, 0.0.0.0:50001->38080/tcp j_spark-worker_2
c92b4e131c1d jentekllc/bigdata:latest "/run_sshd_worker.sh" 9 minutes ago Up 9 minutes 22/tcp, 0.0.0.0:50002->38080/tcp j_spark-worker_1
0ed5d7ed0d22 jentekllc/samza:latest "/home/hadoop/start_…" 9 minutes ago Up 9 minutes 0.0.0.0:50022->22/tcp, 0.0.0.0:58088->8088/tcp, 0.0.0.0:58888->8888/tcp, 0.0.0.0:58889->8889/tcp samza-server
0178dc3df8b1 jentekllc/bigdata:latest "/run_sshd_hive.sh" 9 minutes ago Up 9 minutes 0.0.0.0:30022->22/tcp, 0.0.0.0:38088->8088/tcp, 0.0.0.0:39000->9000/tcp, 0.0.0.0:39083->9083/tcp hadoop-hive
e11a0f1c4dd9 dpage/pgadmin4:latest "/entrypoint.sh" 9 minutes ago Up 9 minutes 443/tcp, 0.0.0.0:20080->80/tcp pgadmin4-container
4effecf897e6 jentekllc/etl:latest "/start_etl.sh" 9 minutes ago Up 9 minutes 0.0.0.0:40022->22/tcp, 0.0.0.0:48080->8080/tcp, 0.0.0.0:48888->8888/tcp, 0.0.0.0:48889->8889/tcp, 0.0.0.0:49000->9000/tcp, 0.0.0.0:40000->30000/tcp etl-server
7b25d8dcd063 jentekllc/postgresql:latest "/start_postgres_ssh…" 9 minutes ago Up 9 minutes 5050/tcp, 0.0.0.0:10022->22/tcp, 0.0.0.0:15432->5432/tcp, 0.0.0.0:18888->8888/tcp, 0.0.0.0:18889->8889/tcp postgresql-container
af0f519a43ce jentekllc/bigdata:latest "/run_sshd_master.sh" 9 minutes ago Up 9 minutes 0.0.0.0:4040->4040/tcp, 0.0.0.0:7077->7077/tcp, 0.0.0.0:8080->8080/tcp, 0.0.0.0:8088->8088/tcp, 0.0.0.0:8888-8889->8888-8889/tcp, 0.0.0.0:20022->22/tcp spark-master
$ docker-compose -p j down
http://localhost:8080
http://localhost:5000
http://localhost:8888
http://localhost:48888
http://localhost:48080
http://localhost:49000
http://localhost:58088
http://localhost:58888
#From host
$ ssh -p 20022 hadoop@localhost
#Inside container
$ cd $SPARK_HOME
$ bin/spark-submit /spark/examples/src/main/python/pi.py
$ spark-sql2021-11-06 23:26:42,842 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java class es where applicable
Setting default log level to "WARN".
Spark master: local[*], Application Id: local-1636241205417
spark-sql>
$ ssh -p 40022 hadoop@localhost
$ ssh -p 50022 hadoop@localhost
$ jps144 ResourceManager
1233 ClusterBasedJobCoordinatorRunner
1729 LocalContainerRunner
1848 LocalContainerRunner
1689 LocalContainerRunner
586 Kafka
1946 Jps
426 NodeManager
76 QuorumPeerMain
1324 ClusterBasedJobCoordinatorRunner
1421 ClusterBasedJobCoordinatorRunner
$ file * | grep directoryhello-samza:        directory
samza: directory
http://localhost:20080
http://localhost:18888
#On the host, password for user postgres is 123456$ ssh -p 10022 postgres@localhost#Inside the container$ psqlpsql (14.1 (Debian 14.1-1.pgdg110+1))
Type "help" for help.
postgres=# \dn List of schemas
Name | Owner
--------+----------
public | postgres
(1 row)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
George Jen

George Jen

I am founder of Jen Tek LLC, a startup company in East Bay California developing AI powered, cloud based documentation/publishing software as a service.