Monday, April 20, 2015

Performance Testing with JMeter for REST Services - A Quick Start Guide

Started to pick up JMeter for a project that exposes a REST API. It is a quite versatile and popular performance and stress test tool. And people have built plugins to extend it.

Official guide: https://jmeter.apache.org/index.html
Extra plugins: https://github.com/undera/jmeter-plugins
Two books by Bayo Erinle: Performance Testing with JMeter 2.9 (2nd edition is coming out soon) and JMeter Cookbook

1. Install JMeter and Plugins

If you are on a Mac, using Homebrew is the easiest way, it installs both vanilla JMeter and extra plugins:

brew install jmeter --with-plugins
==> Downloading https://www.apache.org/dyn/closer.cgi?path=jmeter/binaries/apache-jmeter-2.13.tgz
==> Best Mirror http://mirrors.sonic.net/apache/jmeter/binaries/apache-jmeter-2.13.tgz
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-Standard-1.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/ServerAgent-2.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-Extras-1.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-ExtrasLibs-1.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-WebDriver-1.2.1.zip
######################################################################## 100.0%
==> Downloading http://jmeter-plugins.org/downloads/file/JMeterPlugins-Hadoop-1.2.1.zip
######################################################################## 100.0%
��  /usr/local/Cellar/jmeter/2.13: 1926 files, 115M, built in 74 seconds

If you are on other platforms, do these:

  1. download JMeter from the official site.
  2. download and install extra plugins based on your use cases. These plugins are grouped into several packages, see the detailed plugins package content page to decide what you need. I only installed "Standard Set" and "Extra Set". Installing these plugin packages is simply unzipping them to the installation directory of JMeter, e.g. they install jars into install_dir/lib/ext, or install_dir/bin, etc., see plugin installation guide for more details and minor config changes. The download page has all the links for the plugin packages.
  3. ServerAgent-X.X.X.zip contains the PerfMon Sever Agent that you need to run on the server under test. To run the agent, no special permissions are required. After the agent is running, you can use PerfMon Metrics Collector Listener to connect to the agent and monitor various metrics for CPU, Memory, Swap, Disk and Network I/O, etc. See the document for PerfMon Server Agent and Servers Performance Monitoring for more details.


2. Using JMeter and Plugins

JMeter runs in various modes, you can use with a GUI client or without, you can also set up remote test clients for distributed testing to simulate a more practical workload and traffic pattern.

Here is a basic test for HTTP GET request for a sample service API from geonames.org:

URL: http://api.geonames.org/citiesJSON?north=44.1&south=-9.9&east=-22.4&west=55.2&lang=de&username=demo

Here is the screencast (click on it), note those listeners starts with "jp@gc" are from the non-standard plugins we installed above.





3. Other Interesting Tools and Resources


gatling.io: another high performance open source load testing framework based on Scala, Akka and Netty. It has a DSL based on Scala. It also has nice integration with Jenkins.

yandex-tank: Load testing tool written in Python. For more details, check out its documents.

BlazeMeter: a hosted performance testing service, you can easily reuse your JMeter test scripts with it. It also provides integration with Jenkins CI/CD and supports mobile performance testing. Here is a quick screencast from its website:





Loadsophia: This is a service provided by BlazeMeter, it stores and visualizes the performance test results. The organic visualization in JMeter is quite limited and non-interactive. This service makes analyzing performance data intuitive and fun. It supports test results from tools like JMeter, Apache Benchmark and Yandex.Tank. You can see examples provided publicly by existing users here: http://loadosophia.org/examples/

flood.io: Cloud load testing tool, it supports JMeter and Gastling. Here is a sample report: https://flood.io/d384673f64e3a3


Sunday, February 22, 2015

Insights into the success of Storm

Just read through Nathan Marz's post about the history of Storm. This is really a nice recap of how to successfully start, grow and maintain a great open source project. Technical excellence is important, but the marketing, growing the community and adoption is even more critical.

Here is his post "History of Apache Storm and Lessons Learned".

I am really looking forward to his new book on the lambda architecture: Big Data, principles and best practices of scalable realtime data systems.

Tuesday, December 30, 2014

Swift coding style guide













Found two coding styles for Swift:
Interesting notes:

1. In github style, it suggests that let should be used over var binding whenever possible. This is to explicitly show the intent that a value is supposed to or not supposed to change.


2. In raywenderlich style, when declaring protocol conformance, use separate extension instead of declaring all together. Also, do not forget to add // MARK comments.

Preferred:
class MyViewcontroller: UIViewController {
  // class stuff here
}

// MARK: - UITableViewDataSource
extension MyViewcontroller: UITableViewDataSource {
  // table view data source methods
}

// MARK: - UIScrollViewDelegate
extension MyViewcontroller: UIScrollViewDelegate {
  // scroll view delegate methods
}
Not Preferred:
class MyViewcontroller: UIViewController, UITableViewDataSource, UIScrollViewDelegate {
  // all methods
}

3. In raywenderlich style, when unwrapping the optional, shadow the original name instead of using a new name.

Preferred:
var subview: UIView?

// later on...
if let subview = subview {
  // do something with unwrapped subview
}
Not Preferred:
var optionalSubview: UIView?

if let unwrappedSubview = optionalSubview {
  // do something with unwrappedSubview
}

4. In github style, prefer implicit getters on read-only properties.

Preferred:
var myGreatProperty: Int {
    return 4
}

subscript(index: Int) -> T {
    return objects[index]
}
Not Preferred:
var myGreatProperty: Int {
    get {
        return 4
    }
}

subscript(index: Int) -> T {
    get {
        return objects[index]
    }
}

5. In github style guide, it is mentioned to prefer structs over classes. Since I am still quite new to Swift, I will need to learn a bit more to really understand it.

Saturday, November 29, 2014

book notes for Building Applications with iBeacon

I am always very interested in the location-based mobile applications, in particular indoor location apps. I've even explored a little bit about creating a mobile app that helps locating and assembling colleagues in a crowded cafeteria during lunch. At that time, what I learned is that indoor tracking is still an area being actively researched and explored, many solutions use wifi router capabilities specific to vendors such as Cisco.

Lately, I have quickly went through Matt Gast's book on iBeacon: Building Applications with iBeacon. This is a very concise book on the topic, only 80 pages. It gives a quick overview about iBeacon technology mainly focused on iOS.


Here are the book notes I took:

Companies

Estimote
One of the earliest developers of beacon technology. Estimote beacons have fixed configuration parameters and, in particular, administrators cannot set the UUID.

RadBeacon
RadBeacon is a $29 USB dongle that performs the transmission functions of an iBeacon. All you have to supply is USB power. Configuration of the beacon’s numbers is done through an app.

Kontakt
Kontakt sells an ARM-based iBeacon as well as tools for manag ing iBeacons and analyzing user interactions with them.

Also compatible with android

Gelo
Gelo’s Beacons are waterproof and designed for both indoor and outdoor use, and the batteries can be replaced manually using simple tools.





Hardware
  • Mac (Yosemite stops supporting Mac as iBeacon)
  • iOS devices
  • Raspberry Pi
  • Arduino: BLEduino, BLE Mini from RedBearLab
  • Nordic Semi nRF1822


iOS Apps

- A personal notification beacon for the iPhone. It comes with a beacon device and Geohopper for iOS. It sends notification based on your geo-location. It also integrates with web services for automated workflow.

(Developer Tools)

There are lots of developer util tooks that turns iOS devices into iBeacons. The list can go on and on, here are some apps I found in the app store:
  • Beacon Broadcaster
  • Locate Beacon (from Radius Networks)
  • Beacon Manager
  • My Beacon
  • Beacon Toolkit
  • Beacon Harvester: help you find and save iBeacons around you
  • Beacon Bits
(Games)

One interesting use case for iBeacon is the treasure hunt games. There are several demoes during the mobile developer conferences, very interesting to connect mobile and real world in a playful way. For example, Beacon Scavenger Hunt (from Radius Networks) is a treasure hunt game to collect scavenger hunt’s badges.



Mac Apps

It was very bad that Yosemite does not support turning a Mac into an iBeacon. Maybe this is one of the bugs with Yosemite?


Mavericks as an iBeacon: https://github.com/mttrb/BeaconOSX



a nice thread on StackOverflow about turning your mac into iBeacon: http://stackoverflow.com/questions/19410398/turn-macbook-into-ibeacon


Use Cases


10 things to do with iBeacon: http://blog.twocanoes.com/post/68861362715/10-awesome-things-you-can-do-today-with-ibeacons



Other Useful Resources

Apple's Official Doc on iBeacon: https://developer.apple.com/ibeacon/
Radius Networks Developer Guides: http://developer.radiusnetworks.com
beekn.net: a website dedicated to topics on beacons with Bluetooth Low Energy tech (BLE)

Friday, October 31, 2014

Exposing RDBMS over REST HTTP API

I was working on a Storm project where the work nodes run in a tightly controlled grid environment. So, directly RDBMS access through traditional mechanism is prohibited since only certain ports are open for inbound and outbound traffic. So, I ended up writing a quick and dirty HTTP proxy that exposes certain MySQL database operations using REST API calls.

Then I just found this is actually quite a common use case and several people have written generic HTTP servers to expose REST APIs for database access.

One such project is jdbc-http-server, really handy. I will see if I can replace my own version with that.

Thursday, August 28, 2014

Friday, March 14, 2014

Twisting Maven pom.xml for your legacy code

Recently, I have been working on a legacy project which was not using the standard Maven pom conventions, source code and test code are located at separate paths, the directory structure is also not follow the standard.

So, now we are starting to add more unit tests and integrate that with CI pipeline. The code base is ~200MB including everything. Instead of restructuring the whole project layout, which could block the active development and introduce unpredictable bugs, we decided to twist the pom as best as we can. Here are some nice tips I learned (I am a Maven newbie).

-1. Refactor Your Code

Testing should not be an after-thought. It should be considered along the initial code design and implementation. Apply design patterns, use modular designs and other techniques so that writing tests  become possible in the first place.

0. Writing Unit Tests

There are many unit testing frameworks. Two of the popular ones we use in the company are JUnit and TestNG. Here is a nice StackOverflow comparison for them. Note that if you are using Eclipse, you will need to install TestNG plugin, while JUnit support is built-in. Other IDEs like IntelliJ and Netbeans support both too.

There are several mocking frameworks as well: Mockito, Powermock, EasyMock, jMock, just to name a few. We are using Mockito and Powermock. Here is a nice quick guide how to use them. If you are wondering why we need both, it's because Powermock addresses several features missing from Mockito, such as mocking static methods, etc.

Vogella has several short but useful guides on unit testing, highly recommend:


1. Maven Surefire Plugin with both JUnit and TestNG tests

This is the plugin that runs the unit tests and publish test results. The plugin is documented well on its website, so I am not going to repeat anything here. Quick summary:

  • It can easily include, exclude tests, skip tests (think twice before you do), etc.
  • It support JUnit, TestNG, plain POJO tests. The report format is compatible with JUnit output, so it can be easily integrated with CI tools.
  • It also supports parallel test runs.
  • etc.
Here is a list of all the configuration options for Surefire plugin, very useful.

Normally, you would pick a testing framework and stick to it. But in our case, we have both tests written in JUnit and TestNG under the same test directory. So, how to support that?

Luckily, Marcin has found a solution already to have JUnit and TestNG tests live happily together, see his post here for details. He also has a sample pom that you can use as boilerplate. Basically, you declare dependencies inside Surefire plugin:

...
<properties>
    <surefire.version>2.16</surefire.version>
</properties>

...

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${surefire.version}</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>${surefire.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>${surefire.version}</version>
        </dependency>
    </dependencies>
</plugin>
...

2. Maven compiler plugin

Maven compiler plugin is used to compile your source code. Here are the lists of configuration options for its compiler:compile and compiler:testCompile goals:

But our problem is that it turns out the compiler plugin assumes the source code and test should live under the same root source code directory. In our case, this is not the case. I tried various options such as "testSource", "testIncludes" to specify the test path, but without luck.

Finally, I found a plugin build-helper-maven, which allows customized source and test directories. And it worked like a charm (borrowing from its website):

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>1.8</version>
        <executions>
          <execution>
            <id>add-test-source</id>
            <phase>generate-test-sources</phase>
            <goals>
              <goal>add-test-source</goal>
            </goals>
            <configuration>
              <sources>
                <source>some directory</source>
                ...
              </sources>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

3. Maven Clover Plugin for Code Coverage

To generate code coverage, you can use the Maven Clover plugin, note that Clover is free for non-commierical use, for commercial use, you will need to obtain a license and configure the plugin to point to the license file.


4. Maven FindBugs Plugin

FindBugs is a code analysis tool to find potential bugs in your code. It has nice IDE integration and it also integrates well with Maven. You can set it up as a step in CI so that if the bugs will fail the build. I actually scanned our code base and found several severe bugs (one is a switch statement without break, similar to Apple's recent SSL bug).

Please see the plugin website to set it up, quite straightforward.


5. "One Last Thing"

Another useful tip I found out is that when some of the plugin runs, e.g. test, code coverage, findbugs, etc. they require a fair amount of memory. Take the Surefire for example, depending on your configuration, it will fork separate JVM and threads to run the tests. I had JVM exited abruptly in several cases due to this reason.

Refer to the above configuration options to add customized JVM options. For example, for Maven compiler plugin:

<argLine>-Xmx2048m -XX:MaxPermSize=1024m</argLine>

For Maven compiler plugin:

</compilerArgs><arg>-Xms2048m</arg><arg>-XX:MaxPermSize=1024m</arg></compilerArgs>