Using a Large Number of Collections

A technique one can use with MongoDB in certain situations is to have several collections to store information instead of a single collection.  By doing this, certain repeating data no longer needs to be stored in every object, and an index on that key may be eliminated.  More importantly for performance (depending on the problem), the data is then clustered by the grouping specified.

For example, suppose we are logging objects/documents to the database, and want to have M logs: perhaps a dev log, a debug log, an ops log, etc.  We could store them all in one collection 'logs' containing objects like:

{ log : 'dev', ts : ..., info : ...  }

However, if the number of logs is not too high, it might be better to have a collection per log.  We could have a 'logs.dev' collection, a 'logs.debug' collection, 'logs.ops', etc.:

// logs.dev:
{ ts : ..., info : ... }

Of course, this only makes sense if we do not need to query for items from multiple logs at the same time.

Generally, having a large number of collections has no significant performance penalty, and results in very good performance.

Limits

By default MongoDB has a limit of approximately 24,000 namespaces per database.  Each collection counts as a namespace, as does each index.  Thus if every collection had one index, we can create up to 12,000 collections.  Use the --nssize parameter to set a higher limit.

Be aware that there is a certain minimum overhead per collection -- a few KB.  Further, any index will require at least 8KB of data space as the b-tree page size is 8KB.

--nssize

If more collections are required, run mongod with the --nssize parameter specified.  This will make the <database>.ns file larger and support more collections.  Note that --nssize sets the size used for newly created .ns files -- if you have an existing database and wish to resize, after running the db with --nssize, run the db.repairDatabase() command from the shell to adjust the size.

Maximum .ns file size is 2GB.