Running NuoDB in Docker – Part 4: Using Docker Compose

In the first NuoDB in Docker blog, we outlined how to deploy the NuoDB database in Docker containers using Docker Run. In this blog post, we will improve on that process by providing a single-click deployment of the NuoDB database.

To achieve this, we will use Docker Compose, a tool for defining and running multi-container applications in Docker. Compose significantly simplifies the deployment of NuoDB by automating the database process creation steps.

For this blog post, we will be using a very simple database consisting of

• 2 Admin processes
• 1 Storage Manager (SM)
• 2 Transaction Engines (TE)

As with the previous NuoDB Docker blogs, we will run NuoDB with a Limited Use license. If you do not have your NuoDB Limited Use license, please contact NuoDB Customer Support at NuoDB.Support@3ds.com for your license and installation instructions.

Step 1. Verify Docker Compose V1 is available

First, you should verify that your machine has the docker compose software installed. This example uses the compose file version “3.7” which is available with Docker Engine version 18.06.0+.

$ docker-compose -v
docker-compose version 1.27.2, build 18f557f9

NOTE: For this demonstration, if you are using Docker Desktop, also ensure that Docker Compose V2 is not enabled in the Docker Desktop “Preferences” section.

Step 2. Create the docker-compose.yml configuration file

To start, create a docker-compose.yml file with the following contents:

version: "3.7"
  
volumes:
  nuoadmin-raft-1:
  nuoadmin-raft-2:
  hockey-arch-vol-1:
 
services:
 nuoadmin1:
   image: nuodb/nuodb:latest
   environment:
     NUODB_DOMAIN_ENTRYPOINT: nuoadmin1
     NUOCMD_API_SERVER: http://nuoadmin1:8888
   hostname: nuoadmin1
   volumes:
     - nuoadmin-raft-1:/var/opt/nuodb
   command: ["nuoadmin", "--", "ssl=false"]
 nuoadmin2:
   image: nuodb/nuodb:latest
   environment:
     NUODB_DOMAIN_ENTRYPOINT: nuoadmin1
     NUOCMD_API_SERVER: http://nuoadmin2:8888
   hostname: nuoadmin2
   volumes:
     - nuoadmin-raft-2:/var/opt/nuodb
   command: ["nuoadmin", "--", "ssl=false"]
 sm:
   image: nuodb/nuodb:latest
   hostname: sm
   environment:
     NUOCMD_API_SERVER: http://nuoadmin1:8888
   volumes:
     - hockey-arch-vol-1:/var/opt/nuodb
   depends_on:
   - nuoadmin1
   command: ["nuodocker", "start", "sm", "--db-name", "hockey", "--server-id", "nuoadmin1", "--dba-user", "dba", "--dba-password", "goalie", "--servers-ready-timeout", "15000"]
 te:
   image: nuodb/nuodb:latest
   hostname: te
   environment:
     NUOCMD_API_SERVER: http://nuoadmin1:8888
   depends_on:
   - nuoadmin1
   - sm
   deploy:
     replicas: 2
   command: ["nuodocker", "start", "te", "--db-name", "hockey", "--server-id", "nuoadmin1", "--servers-ready-timeout", "15000"]

This docker-compose.yml file specifies:

• a domain consisting of 2 Admin processes (service:nuoadmin1 and service:nuoadmin2)
• 1 Storage Manager (service:sm)
• 2 Transaction Engine (service:te)

Step 3. Start the NuoDB domain

With your docker-compose.yml file present in your current directory, and docker-compose installed, run: 

$ docker-compose up -d
Creating network "nuodb_default" with the default driver
Creating volume "nuodb_nuoadmin-raft-1" with default driver
Creating volume "nuodb_nuoadmin-raft-2" with default driver
Creating volume "nuodb_test-arch-vol-1" with default driver
Creating nuodb_nuoadmin1_1 ... done
Creating nuodb_nuoadmin2_1 ... done
Creating nuodb_sm_1        ... done
Creating nuodb_te_1        ... done
Creating nuodb_te_2        ... done

Verify that Docker Compose started five containers as expected:

$ docker ps
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                       NAMES
503acb3b7472        nuodb/nuodb:latest   "docker-entrypoint.s…"   54 seconds ago      Up 53 seconds       8888/tcp, 48004-48006/tcp   nuodb_te_2
b6cf209144bb        nuodb/nuodb:latest   "docker-entrypoint.s…"   54 seconds ago      Up 53 seconds       8888/tcp, 48004-48006/tcp   nuodb_te_1
37a1db309b29        nuodb/nuodb:latest   "docker-entrypoint.s…"   54 seconds ago      Up 53 seconds       8888/tcp, 48004-48006/tcp   nuodb_sm_1
4fa0ab0255e7        nuodb/nuodb:latest   "docker-entrypoint.s…"   55 seconds ago      Up 54 seconds       8888/tcp, 48004-48006/tcp   nuodb_nuoadmin1_1
2621e1b79ed3        nuodb/nuodb:latest   "docker-entrypoint.s…"   55 seconds ago      Up 54 seconds       8888/tcp, 48004-48006/tcp   nuodb_nuoadmin2_1

You should see nuodb_te_1, nuodb_te_2, nuodb_sm_1, nuodb_nuoadmin_1, and nuodb_nuoadmin2. The container name depends on the service name (sm/te/nuoadmin1/nuoadmin2) and the name of the current directory. In our example, we are executing from within a directory called “nuodb”.

Docker Compose automatically creates a network for all containers, so it is not required to expose any ports, specify networks, or link containers.

$ docker network ls
NETWORK ID          NAME                      DRIVER              SCOPE
f2e75c2c3086        compose_default           bridge              local

You can list the persistent storage volumes that were created by Docker Compose by using the appropriate docker commands. In this example, we created 1 volume for each admin process and 1 volume for the SM process.

$ docker volume ls
DRIVER              VOLUME NAME
local               nuodb_nuoadmin-raft-1
local               nuodb_nuoadmin-raft-2
local               nuodb_hockey-arch-vol-1

Step 4. Verify that the domain is running

To verify that all containers are fully connected and have formed a healthy domain, run the nuocmd command:

$ docker-compose exec nuoadmin1 nuocmd show domain
server version: 4.1.1-3-2203dab8dd, server license: Limited
server time: 2020-10-09T18:14:51.848, client token: c6cbc5381db737078b4541740ad520570f7f0900
Servers:
  [nuoadmin1] nuoadmin1:48005 [last_ack = 0.85] [member = ADDED] [raft_state = ACTIVE] (LEADER, Leader=nuoadmin1, log=0/23/23) Connected *
  [nuoadmin2] nuoadmin2:48005 [last_ack = 0.85] [member = ADDED] [raft_state = ACTIVE] (FOLLOWER, Leader=nuoadmin1, log=0/23/23) Connected
Databases:
  hockey [state = RUNNING]
    [SM] sm/192.168.144.4:48006 [start_id = 0] [server_id = nuoadmin1] [pid = 38] [node_id = 1] [last_ack =  4.04] MONITORED:RUNNING
    [TE] te/192.168.144.6:48006 [start_id = 1] [server_id = nuoadmin1] [pid = 38] [node_id = 3] [last_ack =  0.19] MONITORED:RUNNING
    [TE] te/192.168.144.5:48006 [start_id = 2] [server_id = nuoadmin1] [pid = 38] [node_id = 2] [last_ack =  1.02] MONITORED:RUNNING

Running SQL

With a domain up and running, you can now follow the steps outlined in Docker Blog I and explore the SQL functionality of NuoDB. To connect to the admin container, simply execute

$ docker exec -it compose_nuoadmin1_1 bash
bash-4.2$ 

or

$ docker-compose exec nuoadmin1 bash
bash-4.2$ 

Scaling the Database

Using docker-compose, it is easy to scale the number of NuoDB TE processes. To add a third TE, simply execute docker-compose scale as follows:

$ docker-compose up -d --scale te=3
Starting nuodb_te_1 ... done
Starting nuodb_te_2 ... done
Creating nuodb_te_3 ... done

After a short wait, the new TE is available in the domain.

$ docker-compose exec nuoadmin1 nuocmd show domain
server version: 4.1.1-3-2203dab8dd, server license: Limited
server time: 2020-10-09T18:24:17.892, client token: 10e758721a939611fe18e22c03b4c6e6639d3d7a
Servers:
  [nuoadmin1] nuoadmin1:48005 [last_ack = 1.75] [member = ADDED] [raft_state = ACTIVE] (LEADER, Leader=nuoadmin1, log=0/29/29) Connected *
  [nuoadmin2] nuoadmin2:48005 [last_ack = 1.75] [member = ADDED] [raft_state = ACTIVE] (FOLLOWER, Leader=nuoadmin1, log=0/29/29) Connected
Databases:
  hockey [state = RUNNING]
    [SM] sm/192.168.176.4:48006 [start_id = 0] [server_id = nuoadmin1] [pid = 39] [node_id = 1] [last_ack =  8.85] MONITORED:RUNNING
    [TE] te/192.168.176.5:48006 [start_id = 1] [server_id = nuoadmin1] [pid = 38] [node_id = 2] [last_ack =  5.85] MONITORED:RUNNING
    [TE] te/192.168.176.6:48006 [start_id = 2] [server_id = nuoadmin1] [pid = 38] [node_id = 3] [last_ack =  4.85] MONITORED:RUNNING
    [TE] te/192.168.176.7:48006 [start_id = 3] [server_id = nuoadmin1] [pid = 40] [node_id = 4] [last_ack =  2.74] MONITORED:RUNNING

How to shut down NuoDB

To shut down a NuoDB domain that was created with Docker Compose, execute the docker-compose down command in the same directory that contains the docker-compose.yml file you created earlier in Step 1.

$ docker-compose down
Stopping nuodb_te_2        ... done
Stopping nuodb_te_1        ... done
Stopping nuodb_sm_1        ... done
Stopping nuodb_nuoadmin1_1 ... done
Stopping nuodb_nuoadmin2_1 ... done
Removing nuodb_te_2        ... done
Removing nuodb_te_1        ... done
Removing nuodb_sm_1        ... done
Removing nuodb_nuoadmin1_1 ... done
Removing nuodb_nuoadmin2_1 ... done
Removing network nuodb_default

Docker Compose will automatically clean up all resources. You can run the following commands to confirm.

networks:

$ docker network ls
NETWORK ID          NAME                      DRIVER              SCOPE

containers:

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

Volumes, on the other hand, will be kept:

$ docker volume ls
DRIVER              VOLUME NAME
local               nuodb_nuoadmin-raft-1
local               nuodb_nuoadmin-raft-2
local               nuodb_hockey-arch-vol-1

To remove unused volumes, run:

$ docker volume prune -f

That’s it! We hope you have enjoyed this quick tour to deploy Docker Compose and NuoDB!

Next Steps

For a more advanced use of NuoDB deploying with Docker Compose, for example when using NuoDB in a development or test environment, I invite you to visit our NuoDB Compose GitHub repository. Here you will find details on how to start NuoDB with NuoDB Insights monitoring, as well as many more database configuration options.