Test APIs without writing tests with Twitter’s Diffy

With large APIs, it becomes difficult to automate each and every request. This makes API tests difficult to maintain, with greater risk of bugs on live systems.

As the complexity of a system grows, it very quickly becomes impossible to get adequate coverage using hand-written tests, and there’s a need for more advanced automated techniques that require minimal effort. Diffy is one such application that can be used.

Diffy behaves as a proxy and multicasts request to Staging and live application and compares the response between the two. The premise of diffy is that if the same requests results in similar response in two implementations of the application, the newer implementation is regression-free.

However, there are few inevitable “noises” in the response that can cause the tests to fail. To mitigate such scenarios, Diffy sends requests to two live servers and one qa/staging server, and does the following:

1. Differences between the QA and Live instances.
2. Noise observed between the primary and secondary Live instances. Diffy can use this to ignore parameters like timestamp, tokens, requestIds etc automatically.

Diffy Architecture

Steps to setup Diffy:

1. Download diffy from https://github.com/opendiffy/diffy or compile your own.
2. Deploy your old code to localhost:9990. This is your primary (1st Live Instance).
3. Deploy your old code to localhost:9991. This is your secondary(2nd Live Instance).
4. Deploy your new code to localhost:9992. This is your candidate(QA/Staging instance).
5. Run diffy with the following command:

java -jar diffy-server.jar \
-candidate=”localhost:9992" \
-master.primary=”localhost:9990" \
-master.secondary=”localhost:9991" \
-service.protocol=”http” \
-serviceName=”My Service” \
-proxy.port=:31900 \
-admin.port=:31159 \
-http.port=:31149 \
-rootUrl=’localhost:31149' \
-allowHttpSideEffects=true

6. Send a few test requests to your Diffy instance:
curl localhost:31900/your_application_route
7. Watch the differences show up in your browser at localhost:31149. You should see something like this:

Be sure to checkout the github repository and official twitter handle:

opendiffy/diffy

Diffy (@diffyproject) | Twitter

Questions/Suggestions are welcome in comments