Introduction
Backing up a database is a crucial task, especially in a production environment. Also, we often need to dump data from production into staging or development for debugging or testing new features before a release.
I have used UI-based tools like Compass to clone collections. While Compass is a useful tool for tasks such as explaining query plans and exporting/importing records from a single collection, it becomes inefficient when working with larger databases containing over 50 collections. At that point, we need a simpler, more scalable solution that can handle entire databases and be automated in a production environment.
MongoDB Database Tools is a command-line interface (CLI) primarily used for backing up and restoring MongoDB databases in .BSON format.
This guide will walk you through installing MongoDB Database Tools on macOS (with minimal differences for Ubuntu) and using it to dump and restore an entire database.
Installation
You can use Homebrew to install MongoDB Database Tools, but this time, I prefer to download the .zip file directly from MongoDB’s official website.
First, extract the .zip file. Then, move the files inside the /bin directory to a location where executables and command-line tools are stored. On macOS, this is typically /usr/local/bin/.
sudo mv extracted_folder/bin/* /usr/local/bin/Prep
Before backing up data, check to make sure you have the latest version of mongodump and mongorestore.
mongodump --version
mongorestore --versionYou should follow this checklist to prepare all necessary values for a smooth process.
[ ] Connection string of main DB
[ ] DB name of main DB
[ ] Connection string of backup DB
[ ] DB name of backup DB
[ ] Backup DB have enough space
Next, we create variables for connection string and DB names.
# This is a example that I have 2 DB on Heroku
# Set MongoDB Connection Variables
MONGO_PROD_URI="mongodb+srv://your_username:your_password@your_cluster.mongodb.net/heroku_abcd1fg2"
MONGO_BACKUP_URI="mongodb+srv://your_username:your_password@your_cluster.mongodb.net/heroku_back3up4"
MAIN_DB_NAME="heroku_abcd1fg2"
BACKUP_DB_NAME="heroku_back3up4"
# Set backup location
BACKUP_LOC="/Users/myuser/mongo-backup/"Dump data
Now we are ready to dump the whole database.
# Dump database
sudo mongodump --uri="$MONGO_PROD_URI" --out=$BACKUP_LOC
# Check folder of backup database
ls "$BACKUP_LOC$MAIN_DB_NAME"You’ll notice that each collection has two files: a .bson file containing the data and a .metadata.json file containing collection metadata, options, and indexes. Some options that are useful in production environments include –quiet (to reduce log output) and –query and –collection for more specific dumps. Learn more here: MongoDB Database Tools Documentation.
Restore data
My goal here is to back up the database, so I will drop all collections before restoring them. This ensures MongoDB won’t encounter errors related to duplicate unique fields.
mongorestore --drop --uri="$MONGO_BACKUP_URI" "$BACKUP_LOC$MAIN_DB_NAME"Sometimes you may want to restore just one collection. For example, I want to restore orders collection.
mongorestore --uri="$MONGO_BACKUP_URI" --nsInclude=$BACKUP_DB_NAME.orders "$BACKUP_LOC$MAIN_DB_NAME".bsonOptions for mongorestore that I think you may need :
- Debugging with –dryrun and –verbose
- Restore one or some collections –nsExclude , –nsInclude
- Restore indexing or not –keepIndexVersion , –noIndexRestore
- Bypass validation for dev env –bypassDocumentValidation
Learn more at MongoDB Database Tools Documentation
Conclusion
With this tutorial, you can now set up an automatic MongoDB backup and restore on your server with just a few adjustments. This will definitely help you dump data from production much faster for debugging.
References
Would you like to read more articles by Tekos’s Team? Everything’s here.
Author
