Performance Testing with JMeter for REST Services - A Quick Start Guide

Started to pick up JMeter for a project that exposes a REST API. It is a quite versatile and popular performance and stress test tool. And people have built plugins to extend it.

Official guide: https://jmeter.apache.org/index.html
Extra plugins: https://github.com/undera/jmeter-plugins
Two books by Bayo Erinle: Performance Testing with JMeter 2.9 (2nd edition is coming out soon) and JMeter Cookbook

1. Install JMeter and Plugins

If you are on a Mac, using Homebrew is the easiest way, it installs both vanilla JMeter and extra plugins:

brew install jmeter --with-plugins
==> Downloading https://www.apache.org/dyn/closer.cgi?path=jmeter/binaries/apache-jmeter-2.13.tgz
==> Best Mirror http://mirrors.sonic.net/apache/jmeter/binaries/apache-jmeter-2.13.tgz
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-Standard-1.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/ServerAgent-2.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-Extras-1.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-ExtrasLibs-1.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-WebDriver-1.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-Hadoop-1.2.1.zip
######################################################################## 100.0%
🍺  /usr/local/Cellar/jmeter/2.13: 1926 files, 115M, built in 74 seconds

If you are on other platforms, do these:

1. download JMeter from the official site.
2. download and install extra plugins based on your use cases. These plugins are grouped into several packages, see the detailed plugins package content page to decide what you need. I only installed "Standard Set" and "Extra Set". Installing these plugin packages is simply unzipping them to the installation directory of JMeter, e.g. they install jars into install_dir/lib/ext, or install_dir/bin, etc., see plugin installation guide for more details and minor config changes. The download page has all the links for the plugin packages.
3. ServerAgent-X.X.X.zip contains the PerfMon Sever Agent that you need to run on the server under test. To run the agent, no special permissions are required. After the agent is running, you can use PerfMon Metrics Collector Listener to connect to the agent and monitor various metrics for CPU, Memory, Swap, Disk and Network I/O, etc. See the document for PerfMon Server Agent and Servers Performance Monitoring for more details.

2. Using JMeter and Plugins

JMeter runs in various modes, you can use with a GUI client or without, you can also set up remote test clients for distributed testing to simulate a more practical workload and traffic pattern.

Here is a basic test for HTTP GET request for a sample service API from geonames.org:

URL: http://api.geonames.org/citiesJSON?north=44.1&south=-9.9&east=-22.4&west=55.2&lang=de&username=demo

Here is the screencast (click on it), note those listeners starts with "jp@gc" are from the non-standard plugins we installed above.

3. Other Interesting Tools and Resources

gatling.io: another high performance open source load testing framework based on Scala, Akka and Netty. It has a DSL based on Scala. It also has nice integration with Jenkins.

yandex-tank: Load testing tool written in Python. For more details, check out its documents.

BlazeMeter: a hosted performance testing service, you can easily reuse your JMeter test scripts with it. It also provides integration with Jenkins CI/CD and supports mobile performance testing. Here is a quick screencast from its website:

Loadsophia: This is a service provided by BlazeMeter, it stores and visualizes the performance test results. The organic visualization in JMeter is quite limited and non-interactive. This service makes analyzing performance data intuitive and fun. It supports test results from tools like JMeter, Apache Benchmark and Yandex.Tank. You can see examples provided publicly by existing users here: http://loadosophia.org/examples/

flood.io: Cloud load testing tool, it supports JMeter and Gastling. Here is a sample report: https://flood.io/d384673f64e3a3