a

Troubleshooting and migrating your secured AWS DocumentDB inside Kubernetes

In this article, we explain a few tools to manage your AWS DocumentDB inside Kubernetes. 1) Launching a MongoDB client pod to connect to AWS DocumentDB Using the following command You will get shell access to a pod inside the k8s environment that will allow you to connect into the AWS DocumentDB database. As the […]

In this article, we explain a few tools to manage your AWS DocumentDB inside Kubernetes.

1) Launching a MongoDB client pod to connect to AWS DocumentDB

Using the following command

kubectl run -i --rm --tty mongo-client --image=mvertes/alpine-mongo --restart=Never --command -- /bin/bash

You will get shell access to a pod inside the k8s environment that will allow you to connect into the AWS DocumentDB database.

As the database requires SSL connection we need to download the public key from AWS using the following command:

wget http://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

this will download the public key file into the current folder, next, if we run the following command from the same folder

mongo --ssl
    --host <hostname>
    --sslCAFile rds-combined-ca-bundle.pem
    --username <mongo-user>
    --password <mongo-password>

After running the previous command we should be connected to the AWS DocumentDB and see a prompt like this:

MongoDB shell version v3.4.2
connecting to: mongodb://<server>:27017/
MongoDB server version: 3.6.0
WARNING: shell and server versions do not match
rs0:PRIMARY> 

Then, we can just switch to the desired db using the following command:

rs0:PRIMARY> use <db-name>;
switched to db <dn-name>

At this point we can run any mongoldb cli command we need to see/troubleshoot data issues.

2) Launching a Mongo Dump client pod to to create a backup from DocumentDB

Using the following command

kubectl run -i --rm --tty mongodump-client --image=bigtruedata/mongodump --restart=Never --command -- /bin/sh

You will get shell access to a pod inside the k8s environment that will allow you to execute mongodump against the AWS DocumentDB database.

As the database requires SSL connection we need to download the public key from AWS using the following command:

wget http://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

We are ready now to dump the database using the following command:

mongodump --ssl      --host <host>     --sslCAFile rds-combined-ca-bundle.pem     --username <db-user>     --password <db-password> --db <db-name>

At the end of the dump we will see a confirmation message like:
2019-08-14T20:22:46.547+0000 done dumping <db-name>.collectionX (12775 documents)

This should have created a new folder named dump/ with all the dump data. This means we are ready to create a tar/gz file with that folder using the following command:
tar -zcvf db-dump.tar.gz dump

Now we have the tar/gz file with all the information:

ls -ltrh *.tar.gz
-rw-r--r--    1 root     root        9.9M Aug 14 20:37 db-dump.tar.gz

Now without closing the current bash shell, open a new terminal tab and run the following command:

kubectl default/mongo-client3:/dump/db-dump.tar.gz /tmp 

This will copy the backup db-dump.tar.gz file into the tmp folder of the computer you are connected from tmp (hopefully still inside a secure network)

At this point, we can run the same cp command to copy to your second k8s environment, but we need to first launch a pod instance of the mongodump pod to be able to do a mongorestore so let’s launch it.

Finally, we can exit the mongodump pod and it will be automatically deleted.

NOTE: At this point you should switch kubectl to point to the destination k8s cluster where you want your backup to be restored.

3) Launching a Mongo Dump client pod to restore the backup from DocumentDB

Using the following command we will launch a pod that will be used to restore the database.

kubectl run -i --rm --tty mongorestore-client --image=bigtruedata/mongodump --restart=Never --command -- /bin/sh

You will get shell access to a pod inside the k8s environment that will allow you to execute mongorestore against the AWS DocumentDB database.

As the database requires SSL connection we need to download the public key from AWS using the following command:

wget http://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem

Open a new terminal tab to copy the backup file into the mongorestore pod by running:

kubectl cp /tmp/db-dump.tar.gz default/mongorestore-client:/dump

After it finishes uploading the file into our pod we can now delete our local copy of the backup.

Now, we need to extract the tar/gz file using the following command

tar xvzf db-dump.tar.gz

This will extract all the contents of the tar/gz file into our dump/ folder. We can check by running:

 ls -ltrh
total 10120
-rw-r--r--    1 501      wheel       9.9M Aug 14 20:44 db-dump.tar.gz
drwxr-xr-x    3 root     root          29 Aug 14 21:00 dump

Finally we can run the restore:

mongorestore dump/ --ssl  --host <host>  --sslCAFile rds-combined-ca-bundle.pem  --username <db-user> --password <db-password> --db <db-name>

At this point our database should have been fully restored into our new k8s environment. You can refer to the first section fo this document to connect to your AWS DocumentDB and see your data.

allcode_admin
allcode_admin

Related Articles

AWS Snowflake Data Warehouse Pricing Guide

AWS Snowflake Data Warehouse Pricing Guide

AWS Snowflake Data Warehouse – or just Snowflake – is a data cloud built for users to mobilize, centralize, and process large quantities of data. Regardless of how many sources are connected to Snowflake or the user’s preferred type of organized data used, data is easily stored and controllably shared with selectively-authorized access. Snowflake does offer extensive control over its pricing, though how it works isn’t always clear.

Guide to Cost Factors for Amazon’s RDS Pricing

Guide to Cost Factors for Amazon’s RDS Pricing

Amazon sports a complex pricing model. As simple as the pay-for-what-you-use model sounds, there are several nuances to every service from data used to computing components chosen that can easily let users blow through their entire budget. That’s not to say AWS is an inherently expensive platform and can be adapted to with proper planning and understanding of the contributing factors to billing.

Amazon Elastic Cloud Computing Pricing Guide

Amazon Elastic Cloud Computing Pricing Guide

Amazon Elastic Cloud Computing is the default option for computing on AWS. Outside of outsourced cloud computing options, it is the default service for building, running, and scaling AWS-based applications. As such, EC2 will likely be the main driving force behind AWS bills. Understanding how to control said costs is therefore the most important factor in managing your AWS environment.

Download our 10-Step Cloud Migration ChecklistYou'll get direct access to our full-length guide on Google Docs. From here, you will be able to make a copy, download the content, and share it with your team.