0

I have streaming data on a Windows platform that I capture into Mongodb at fairly high rate of about 800 data points per second. I wish to have access to this data from outside the company, but the company does not wish to port forward 27017 (mongod) to the outside world. I have setup authorization and compiled mongod for ssl.

How can I expose Mongo to an external server? My server is sitting in another location in the "free" internet, and I wish it, every 10 seconds, to get the latest say, 1000 data points out of the server. How would I do this if I cannot port forward?

Can I get the mongo server to "push" data somehow to the external server (which has a fixed IP)? FTP is not a solution as the data streams too rapidly for this (I think).

Can I somehow stream it out using http, or some other protocol?

Ideally I would have liked "native" access to the mongo server as this would have allowed me to use tailable cursors, so any solution which would approximate this functionality would be good. However if this is not possible or practical, a streaming push solution from the firewall protected server, to the outside world server, would also work for me.

Thomas Browne
  • 103

3 Answers3

4

If your private server is always-on, and your company is ok with you using a VPN (that's a big if, check with IT), I would use openVPN and possibly MongoDB replica set.

OpenVPN's security/effort ratio is quite good - it's available as a standard package on most Linux distros, runs on configuration files, has many tutorials, uses static key (simple setup) or TLS (one key per client/server).
OpenVPN HOWTO
Your "external" server will be the VPN server, and the MongoDB "master" will connect to it automatically on startup.

After your servers are connected, you need to choose if you want to query the "master" via VPN, or use a secondary MongoDB server on the "external" to sync, then query it.
MongoDB's replica set allows one server to keep "in sync" with a primary server. It's usually used for fault tolerance, but you can also use it for your purpose.
MongoDB Geographically Redundant Replica Set.
Make sure you the "external" server will be non-voting and with priority 0 (means it won't be part of cluster calculations)

It's best if you confide with your IT guys about the whole solution, and test it before starting to rely on it for production-related tasks

Nitz
  • 1,078
2

One possibility is to set up an SSH tunnel that would be used for your Mongo database connections. The SSH tunnel would encrypt the Mongo traffic, and SSH tunnels are a well known technique.

See:

Community
  • 1
Jon Onstott
  • 151
1

There are good suggestions on tunnels already (of which I'd favour OpenVPN), but here I'll suggest a different approach.

Rather than exposing the whole of your mongodb data to the remote server in order to get a specific data set out, you might be better to build a more tightly focussed web API, which could run at either location.

  1. You could run a script or daemon on your mongodb server machine, or close to it (and within the firewall) which accesses mongodb natively, and then packages the data up and pushes it to an API on your remote web server. Probably REST and JSON are the sort of things you'd look at for designing the API on your web server.
  2. Alternatively you could have a web server close to the mongodb server, which is accessible from your remote web server. You'd implement a suitable API on the web server that's local to your mongodb server, and pull data from that by calling it from your remote web server.

Which approach works better depends mostly on the availability of somewhere suitable to run stuff inside your company network, and the network policies involved. You can lock down access to your API pretty tightly to satisfy concerns of your network managers. Eg lock access down by IP, require suitable authentication, and perhaps lock it down to a specific SSL key.

If possible, the most efficient approach is probably going to be running a daemon close to the mongodb server which uses the tailable cursors you mention, and sends the data to an API on your remote web server.

mc0e
  • 5,979