Running mongodb as a replicaSet in Docker (and upgrading it from 3.0 to 3.4)

Goal:

This post is about two things in one go:

Prerequesits (or what I have and use for my case):

  •  a virtual machine running docker
  • an application conneting to and using mongodb
  • a db dump for the application

Step One – Configure and boot a single mongo server with docker

This is influenced by the very good article Creating a MongoDB replica set using Docker.

It gives details about the basics (set up containers, start a replica set). My goal is to go a little bit furhter, though. In addtion to what the article suggests, I’d like to have the data of each container in a data volume. And I’d like to use docker-compose to keep the whole setup in order.

Using the version 3 syntax of docker-compose, I come up with a very basic initial file to start from:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    ports:
    - "30001:27017"

volumes:
  datadb01:

What it does:

  • Use version 3 of the compose syntax
  • define a first db service, based on a mongo image with the version tag 3.0
  • expose the image’s port 27017 on the host as port 30001
  • mount a named data volume datadb01 into the container at /data/db (the default path of MongoDBs data storage)

This can be run with docker-compose up -d and new we have a single instance mongodb running on port 30001, accessible from the outside.

Step Two – Extend the single mongodb instance to become a multi-host replica set

Adding a second host to the configuration is straight forward and requires some copy & paste so the docker-compose.yml file looks like this

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    ports:
    - "30001:27017"
  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    ports:
    - "30002:27017"

volumes:
  datadb01:
  datadb02:

To check if the machine is up, I connect to the second mongod instance from another machine with mongo --port 30002. Of course, this is – as of right now – only a separate single instance of mongod and not a replicaSet, as confirmed by a quick check of the replication status:


> rs.status()
{ "ok" : 0, "errmsg" : "not running with --replSet", "code" : 76 }

At this point, I decided to make this another mongo exercise and start the replicaSet with only two servers, import my data, and only later add the third machine.

So, to get this dual-setup running, we need to tell the machines what replicaSet they are part of. This can be done with a command line option on mongod (--replSet), but I wanted to make it more versatile and put some options into a config file for mongo and start the daemon by telling it where to pull the config from.

So, in a subfolder etc, the simple config file etc/mongod.conf is created:

replication:
   oplogSizeMB: 400
   replSetName: rs0

(the oplog size is a random number here and should be correctly adjusted in production environments)

Now we need to map this file into the containers and tell mongod to read it during startup:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30001:27017"
    command: ["mongod", "--config", "/etc/mongod.conf"]

  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30002:27017"
    command: ["mongod", "--config", "/etc/mongod.conf"]

volumes:
  datadb01:
  datadb02:

What we have here now in addition:

  • copy the file /etc/mongod.conf to /etc/mongod.conf in the container
  • start the container with the additional options, resulting in mongod --conf /etc/mongod.conf

Until now, I was under the impression I could spin up a mongo cluster just like that, but some research and this question on stack overflow made me aware that it won’t work withough a little bit of shell command line.

So, let’s init the replSet

To get the set working, we need to define a config in the mongo shell, for exaple like this:

> rs.initiate({
  "_id": "rs0",
  "version": 1,
  "members" : [
   {"_id": 1, "host": "db01:27017"},
   {"_id": 2, "host": "db02:27017"}
  ]
 })

(Note: as the machines connect internally, the internal ports 27017 need to be used, not the exposed ones)

However, to make this work, the containers need to be known as db01 and db02. They autom automatically got a generated name by docker-compose. So the names have to be added in the docker-compose file to be manually set:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30001:27017"
    command: ["mongod", "--config", "/etc/mongod.conf"]
    container_name: db01

  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30002:27017"
    command: ["mongod", "--config", "/etc/mongod.conf"]
    container_name: db02

volumes:
  datadb01:
  datadb02:

After another docker-compose up -d, the config above can be initialized and results in a happy replication set:

> rs.status()
{
        "set" : "rs0",
        "date" : ISODate("2017-06-21T14:38:13.720Z"),
        "myState" : 2,
        "members" : [
                {
                        "_id" : 1,
                        "name" : "db01:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 161,
                        "optime" : Timestamp(1498055732, 1),
                        "optimeDate" : ISODate("2017-06-21T14:35:32Z"),
                        "lastHeartbeat" : ISODate("2017-06-21T14:38:12.384Z"),
                        "lastHeartbeatRecv" : ISODate("2017-06-21T14:38:12.384Z"),
                        "pingMs" : 0,
                        "electionTime" : Timestamp(1498055736, 1),
                        "electionDate" : ISODate("2017-06-21T14:35:36Z"),
                        "configVersion" : 1
                },
                {
                        "_id" : 2,
                        "name" : "db02:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 216,
                        "optime" : Timestamp(1498055732, 1),
                        "optimeDate" : ISODate("2017-06-21T14:35:32Z"),
                        "configVersion" : 1,
                        "self" : true
                }
        ],
        "ok" : 1
}

Now it’s time to import some data into the cluster and try to connect my existing application to the new cluster.
It should be noted here that the application is NOT running as part of the docker setup but is intended to connect to the ports exposed.

quick break, have some coffee while we wait for mongoimport to finish

Importing the data to the cluster with a mongoimport shell script on the primary server is not a but problem, but my PHP and old \MongoClient based application seems to have a problem:

MongoConnectionException
No candidate servers found

MongoConnectionException
MongoClient::__construct(): php_network_getaddresses: getaddrinfo failed: Name or service not known

Looks like that fact that using different IPs and ports “on the outside” (the configuration exposed by docker) is not good enough for the php mongo driver.
To circumvent this, let’s try to match internal and external configurations:

First, match up internal and external mongod ports by changing the internal ones:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30001:30001"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30001"]
    container_name: db01

  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30002:30002"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30002"]
    container_name: db02

volumes:
  datadb01:
  datadb02:

The command is extended to start mongod internally on port 30001 (30002 for db02) while still exposing it on the same port.

Then the hostnames db01/db02 are added to the application server’s /etc/hosts so there is no problem resolving the name

192.168.10.20    db01
192.168.10.20    db02

After another docker-compose up -d, the changed configurations is applied; however, this breaks our cluster!! The primary and secondary have changed their internal ports, so the cluster connection is lost.

To tell the replSet about this, we need to reconfigure the cluster with the changes:

rs.reconfig({
  "_id": "rs0",
  "version": 1,
  "members" : [
   {"_id": 1, "host": "db01:30001"},
   {"_id": 2, "host": "db02:30002"}
  ]},
  {"force": true }
)

After that, my application is able to connect to the new cluster and everything seems to be fine.

A note on the php config, though:

The MongoClient configuration needs some help to know that there is a cluster and where to perform read/write operations, so the following additional information is necessary:

server-config:
  'mongodb://db01:30001,db02:30002'

client-options:
 readPreference: primary
 replicaSet: rs0

Where and how to put this depends in the individual application, in my case with doctrine_mongodb it looks something like this in the symfony’s config.yml:

doctrine_mongodb:
    connections:
        default:
            server: "mongodb://db01:30001,db02:30002"
            options:
                db: "%mongo_database%"
                readPreference: primary
                replicaSet: rs0

As this article got a little bit longer than expected, the rest will follow in another one

2 thoughts on “Running mongodb as a replicaSet in Docker (and upgrading it from 3.0 to 3.4)

  1. Pingback: Running mongodb as a replicaSet in Docker (and adding a new SECONDARY and then upgrading from 3.0 to 3.4) | Real-Time-Web, symfony and the rest

  2. The Beatles – легендарная британская рок-группа, сформированная в 1960 году в Ливерпуле. Их музыка стала символом эпохи и оказала огромное влияние на мировую культуру. Среди их лучших песен: “Hey Jude”, “Let It Be”, “Yesterday”, “Come Together”, “Here Comes the Sun”, “A Day in the Life”, “Something”, “Eleanor Rigby” и многие другие. Их творчество отличается мелодичностью, глубиной текстов и экспериментами в звуке, что сделало их одной из самых влиятельных групп в истории музыки. Музыка 2024 года слушать онлайн и скачать бесплатно mp3.

Leave a Reply