Collecting MongoDB TCP Traffic

Honeycomb’s TCP collector for MongoDB passively analyzes traffic between your MongoDB clients and servers to give you query-level visibility into your database workload. It doesn’t require reconfiguring MongoDB, and doesn’t affect database performance.

Setup

You can install and run the TCP collector on either your MongoDB or your application servers – the choice is yours.

Download and install the latest version of honeycomb-tcpagent by running:

deb

wget -q https://honeycomb.io/download/honeycomb-tcpagent/honeycomb-tcpagent_1.29_amd64.deb && \
      echo '44ceaf19a9a19a61c094ed529dfebef45d2d534867942c84e5d63b2741347b21 honeycomb-tcpagent_1.29_amd64.deb' | sha256sum -c && \
      sudo dpkg -i honeycomb-tcpagent_1.29_amd64.deb

wget -q https://honeycomb.io/download/honeycomb-tcpagent/honeycomb-tcpagent-1.29-1.x86_64.rpm && \
      echo 'd7ff1ddf7ec0cf3645ed3aa0ee980e28ed73cdcd9069b406b069286d709710b8 honeycomb-tcpagent-1.29-1.x86_64.rpm' | sha256sum -c && \
      sudo rpm -i honeycomb-tcpagent-1.29-1.x86_64.rpm

wget -q -O honeycomb-tcpagent https://honeycomb.io/download/honeycomb-tcpagent/1.29 && \
      echo '34f4d729d8569381b5be2327875e1940a3dcdcae8ed1181586ad10f07a43f8ba honeycomb-tcpagent' | sha256sum -c && \
      chmod 755 ./honeycomb-tcpagent && \
      sudo setcap cap_net_raw=eip honeycomb-tcpagent

The package installs honeycomb-tcpagent, a sample config file /etc/honeycomb-tcpagent/honeycomb-tcpagent.conf, and start scripts. The binary is just honeycomb-tcpagent, available if you need it in an unpackaged form or for ad-hoc use.

Usage

Start collecting data by running

honeycomb-tcpagent --writekey=YOUR_WRITE_KEY --dataset="MongoDB"

To run honeycomb-tcpagent as a service, modify the config file at /etc/honeycomb-tcpagent/honeycomb-tcpagent.conf and set:

Writekey to your team Write Key, available from your account page
Dataset to the name of the dataset you want to populate (e.g., `MongoDB’)

Then run sudo initctl start honeycomb-tcpagent (if your init system uses Upstart) or sudo systemctl start honeycomb-tcpagent (if your init system uses systemd).

Note: Are you running MongoDB on a non-standard port, or have other special configuration? Run honeycomb-tcpagent --help for advanced configuration options.

Starter Queries

A few things you might want to look at are:

Query count broken down by collection, operation, and query shape
Average and 99th-percentile query duration broken down by collection
Average and 99th-percentile request and response size broken down by query shape.

Example Field

An example find query on a MongoDB collection named “restaurants” will produce a Honeycomb event that looks like this:

field name	type	value
client_ip	string	10.2.0.45
collection	string	restaurants
command	json	{ “batchSize”: 1, “filter”: { “grade”: “A”, “score”: { “$gte”: 3 } }, “find”: “restaurants”, “limit”: 100 }
commandType	string	find
database	string	appdb
duration_ms	float	14.332
namespace	string	appdb.$cmd
normalized_query	json	{ “filter”: {“grade”: 1, “score”: { “$gte”: 1 }}, “find”: 1, “limit”: 1 }
nreturned	float	1
ninserted	float	0
request_id	float	3139
request_length	float	48
response_length	float	3442
server_ip	string	10.2.0.230

Numbers are ingested as floats by default in Honeycomb. You can change a field’s type in the Schema section of your dataset’s Overview page.

Scrubbing the Raw Command

If you have sensitive data in your queries that you don’t want to send to Honeycomb, you can use the --mongodb.scrub_command flag or set ScrubCommand = true in the configuration file. When set, the value of the command field (which contains raw queries, inserts, etc.) will be replaced with its SHA256 hash. You will still be able to do inclusion and frequency analysis on the hashed field, but the actual value won’t be sent.

Open Source

The honeycomb-tcpagent source code is available on Github and Apache 2.0 licensed.