Several components need to work together to make sharding possible. When they’re all functioning together, this is known as a sharded cluster. To understand how MongoDB’s sharding works, you need to know about all the components that make up a sharded cluster and the role of each component in the context of the cluster as a whole.
Leave some elbow room
Although it may be tempting to wait to shard until all your disks are 100% full and all your machines are overloaded, that’s a bad idea. Sharding itself puts some load on your system because the process of automatic balancing has to move data off overloaded shards. If your system is already so overloaded that this can’t happen, your empty shards will remain empty, your overloaded shards will remain over- loaded, and your system will grind to a halt. Chapter 13 gives some practical advice for how to keep track of the important metrics, so you can scale up smoothly as your application grows.
337 Understanding components of a sharded cluster
A sharded cluster consists of shards, mongos routers, and config servers, as shown in figure 12.1.
Let’s examine each component in figure 12.1:
■ Shards (upper left) store the application data. In a sharded cluster, only the mongos routers or system administrators should be connecting directly to the shards.
Like an unsharded deployment, each shard can be a single node for develop- ment and testing, but should be a replica set in production.
■ mongos routers (center) cache the cluster metadata and use it to route opera- tions to the correct shard or shards.
■ Config servers (upper right) persistently store metadata about the cluster, includ- ing which shard has what subset of the data.
Now, let’s discuss in more detail the role each of these components plays in the cluster as a whole.
12.2.1 Shards: storage of application data
A shard, shown at the upper left of figure 12.1, is either a single mongod server or a replica set that stores a partition of the application data. In fact, the shards are the only places where the application data gets saved in a sharded cluster. For testing, a shard can be a single mongod server but should be deployed as a replica set in produc- tion because then it will have its own replication mechanism and can fail over auto- matically. You can connect to an individual shard as you would to a single node or a
mongosrouter (in-memory copy of
cluster metadata) Shard-b (Replica set)
Config server 3
Config server 2
Config server 1
Shards store application data
Metadata write (two-phase commit)
mongos reads/writes application data from/to shards
mongos routes queries and collects result
Metadata read (single server) Application sends
queries to single mongos
Application Shard-a
(Replica set)
Figure 12.1 Components in a MongoDB shard cluster
338 CHAPTER 12 Scaling your system with sharding
replica set, but if you try to run operations on that shard directly, you’ll see only a por- tion of the cluster’s total data.
12.2.2 Mongos router: router of operations
Because each shard contains only part of the cluster’s data, you need something to route operations to the appropriate shards. That’s where mongos comes in. The mongos process, shown in the center of figure 12.1, is a router that directs all reads, writes, and commands to the appropriate shard. In this way, mongos provides clients with a single point of contact with the cluster, which is what enables a sharded cluster to present the same interface as an unsharded one.
mongos processes are lightweight and nonpersistent.1 Because of this, they’re often deployed on the same machines as the application servers, ensuring that only one net- work hop is required for requests to any given shard. In other words, the application con- nects locally to a mongos, and the mongos manages connections to the individual shards.
12.2.3 Config servers: storage of metadata
mongos processes are nonpersistent, which means something must durably store the metadata needed to properly manage the cluster. That’s the job of the config servers, shown in the top right of figure 12.1. This metadata includes the global cluster config- uration; the locations of each database, collection, and the particular ranges of data therein; and a change log preserving a history of the migrations of data across shards.
The metadata held by the config servers is central to the proper functioning and upkeep of the cluster. For instance, every time a mongos process is started, the mongos fetches a copy of the metadata from the config servers. Without this data, no coherent view of the shard cluster is possible. The importance of this data, then, informs the design and deployment strategy for the config servers.
If you examine figure 12.1, you’ll see there are three config servers, but they’re not deployed as a replica set. They demand something stronger than asynchronous repli- cation; when the mongos process writes to them, it does so using a two-phase commit.
This guarantees consistency across config servers. You must run exactly three config servers in any production deployment of sharding, and these servers must reside on separate machines for redundancy.2
Now you know what a shard cluster consists of, but you’re probably still wondering about the sharding machinery itself. How is data actually distributed? We’ll explain that in the next section, first introducing the two ways to shard in MongoDB, and then covering the core sharding operations.
1 The mongos server caches a local copy of the config server metadata in memory. This metadata has a version identifier that changes when the metadata changes, so when a mongos with old metadata tries to contact a shard with a newer metadata version, it receives a notification that it must refresh its local copy.
2 You can also run a single config server, but only as a way of more easily testing sharding. Running with only one config server in production is like taking a transatlantic flight in a single-engine jet: it might get you there, but lose an engine and you’re hosed.
339 Distributing data in a sharded cluster