8

We're developing an new server to replace an old one, and want to put the side by side for before & after tests - so that we can compare the end result and find the discrepancies.

The input to both services (old and new) is HTTP requests (mostly if not all GET, but possibly also POST), we need to have traffic coming to the old server duplicated and sent also to the new server (not instead). of course we will need to do some slight rewriting on the Hostname and port in the request to avoid an endless loop.

Old server is running on Apache (PHP), new one is Jetty (Java, Dropwizard)

There must be a way to do this, I just can't find the keywords for the Google search I guess...

Moshe Eshel
  • 223

4 Answers4

3

What we chose eventually was using Gor (now GorReplay) https://github.com/buger/goreplay

this solution allows installing a listener on the original host, and what it does is record any incoming HTTP request, this is done without modifying it or blocking the production server from handling it.

It then pushes these requests to a Gor replay server which can handle all kinds of useful logics of splitting/increasing load based on the incoming requests - you can send a percentage of requests to a dev server, or a multiplication of the requests to create simulated (but from real traffic) load on your staging environment, or both...

Sadly this is at the server level, so you have to install on each production server to get all the traffic, but you don't have to, and it provides a great solution for the problem outlaid in my question.

Moshe Eshel
  • 223
1

This is probably late but for those who will need it later;

You can use mitmproxy to do that. It allows duplication and modification (changing the host in your case) of the captured requests. In fact, there is an example about this in github repo. I used it in client side, but from my understanding, it can be used at the server side by setting up a reverse proxy for your old server.

Burak Yuksel
  • 11
1

Although is not what you are asking for, I will suggest another approach to test the new server.

If you put a load balancer in front of both servers and play with the load balancing algorithms you can at the same time test the new server and gradually replace the old one. You can send 99% of the requests to the old server and the remaining one percent of requests will go to the new one where you can closely review if the service is working as expected.

If everything works fine you can increase the load gradually; 90%-10%, 80%-20%, and so on.

Hint: Check haproxy and the weight and static-rr options.

Gabriel Talavera
  • 1,377
1

There is another tool written in NodeJS and it's free https://www.npmjs.com/package/duplicator or in github https://github.com/agnoster/duplicator

As now passed too much time since the question at least hope this could help other readers if not to the original requester.

Sposmen
  • 111