Simple Initial Sharding Architecture

Overview

This is a sample sharding architecture that is a good starting point for building your cluster.

Goal

• Two datacenters (East=primary, West=backup/DR)
• Data Tier (MongoDB)
  • 3 shards
  • 3 nodes per shard
  • 9 hosts total
• Application Tier
  • 4 application servers

Machines Locations

• e1-e6 are in 1 datacenter (East)
• w1-w3 are in another datacenter (West)

Datacenter Roles

We'll use datacenter East as the primary, and data center West as disaster recovery.

Replica sets

The replica set nodes in West will be priority 0 so they don't become master automatically.

The first thing we need to do is setup the 3 replica sets

• Replica Set A:
  • e1.acme.com : priority=1
  • e2.acme.com : priority=1
  • w1.acme.com : priority=0

Start the mongod process on each node, e.g.

e1.acme.com# mongod --shardsvr --replSet rs_a
e2.acme.com# mongod --shardsvr --replSet rs_a
w1.acme.com# mongod --shardsvr --replSet rs_a

Note: --shardsvr will default the port to 27018

In the mongo shell, create the replica set

> cfg = {
    _id : "rs_a",
    members : [
        {_id : 0, host : "e1.acme.com:27018", priority : 1},
        {_id : 1, host : "e2.acme.com:27018", priority : 1},
        {_id : 2, host : "w1.acme.com:27018", priority : 0}
    ]
}

> rs.initiate(cfg)

Repeat for each replica set as follows

• Replica Set B:
  • e3.acme.com : priority=1
  • e4.acme.com : priority=1
  • w2.acme.com : priority=0

• Replica Set C:
  • e5.acme.com : priority=1
  • e6.acme.com : priority=1
  • w3.acme.com : priority=0

Config Servers

The next thing we need is to choose 3 config server locations.

We'll pick 2 random nodes in E (primary) and 1 in W (backup/DR). Since we have multiple MongoDB process on a node we need to ensure that different ports are being used, so we will use 27019 for the Config Servers.

• Config Servers
  • e1 - c1.acme.com:27019
  • e4 - c2.acme.com:27019
  • w1 - c3.acme.com:27019

DNS Aliases We should make dns aliases for these so its easy to change later

Start up the config servers, e.g.

e1.acme.com> mongod --configsvr
e4.acme.com> mongod --configsvr
w1.acme.com> mongod --configsvr

Note: --configsvr will default the port to 27019

MongoS (routers)

The last question is where to put the mongos.

Suggested

The suggested configuration is to run an instance on each app-server (as shown by a1-a4 in diagram above).

Alternative

These are the other common options:

• On each server (all 9)
• Create a (sticky) load-balanced cluster (independent of client/shards)

Startup Options

When we start the mongos, we'll use

> mongos --configdb c1.acme.com:27019,c2.acme.com:27019,c3.acme.com:27019

Note: be sure to use DNS names for yoru configdb names, not ip addresses. Otherwise moving a config server later will be quite difficult.

Then we'll need to add the 3 replica sets as shards

> db.adminCommand( { addShard : "rs_a/e1.acme.com:27018,e2.acme.com:27018,w1.acme.com:27018" } )
> db.adminCommand( { addShard : "rs_b/e3.acme.com:27018,e4.acme.com:27018,w2.acme.com:27018" } )
> db.adminCommand( { addShard : "rs_c/e5.acme.com:27018,e6.acme.com:27018,w3.acme.com:27018" } )

Everything is running

At this point your basic architecture is ready to go. You've got 3 shards for scalability, and 3 copies of each piece of data (with one ready for DR). There are obviously many ways to configure this, but this is a pretty simple way to get started.

Your next step is to enable sharding for any database you would like to use sharded, and to then enable sharding for any collection you want sharded.  Databases and collections by default will be "unsharded" and simply reside in the first shard.

To save yourself time, don't bother sharding tiny collections, just do the big ones.

Notes

• Names (DNS) should be used everywhere, and consistently
• All client writes/reads will be isolated to the Primary Datacenter (East)
• SlaveOk can be used to allow stale/eventually-consistent reads
  • If this is done, hidden should be used with the West nodes e.g.

    > cfg = {
        _id : "rs_a",
        members : [
            {_id : 0, host : "e1.acme.com:27018", priority : 1},
            {_id : 1, host : "e2.acme.com:27018", priority : 1},
            {_id : 2, host : "w1.acme.com:27018", priority : 0, hidden : true}
        ]
    }
    
    > rs.initiate(cfg)