JVM Tutorials - Herong's Tutorial Examples - v5.14, by Herong Yang
BenchmarkRunner.java - Benchmark Runner Program
This section provides a tutorial example on how to write a benchmark runner program that takes care of JVM warming up, JIT compilation, measuring time, and other non-test related tasks.
The best way to write a micro benchmark program is probably to separate it in 2 parts:
Here my test runner program, BenchmarkRunner.java:
/* BenchmarkRunner.java
* Copyright (c) HerongYang.com. All Rights Reserved.
*/
class BenchmarkRunner {
static java.io.PrintStream out = System.out;
static java.io.InputStream in = System.in;
long startTime = 0;
long endTime = 0;
long[] timeRecords = null;
public static void main(String[] a) {
if (a.length<5) {
out.println("Missing arguments. Usage: ");
out.println(
"BenchmarkRunner class method warmups runs steps");
return;
}
try {
String className = a[0];
String methodName = a[1];
int numberOfWarmups = Integer.parseInt(a[2]);
int numberOfRuns = Integer.parseInt(a[3]);
int numberOfSteps = Integer.parseInt(a[4]);
// Warming up the JVM
out.println("Are you ready?");
in.read(new byte[1]);
// Loading the benchmark class and method
Class<?> testClass = Class.forName(className);
java.lang.reflect.Method testMethod
= testClass.getMethod(methodName, int.class,
BenchmarkRunner.class);
BenchmarkRunner testRunner
= new BenchmarkRunner(numberOfRuns);
Object testObject
= testClass.getDeclaredConstructor().newInstance();
// JIT warmup
out.println();
out.println("Waking up the JIT compiler...");
for (int i=0; i<numberOfWarmups; i++) {
Object testResult =
testMethod.invoke(testObject, numberOfSteps, testRunner);
out.println("Run: "+i+", Time: "+testRunner.returnTime()
+", Test returns: "+testResult);
}
// Benchmark runs
out.println();
out.println("Starting benchmark test runs...");
for (int i=0; i<numberOfRuns; i++) {
Object testResult =
testMethod.invoke(testObject, numberOfSteps, testRunner);
testRunner.recordTime(i);
out.println("Run: "+i+", Time: "+testRunner.returnTime()
+", Test returns: "+testResult);
}
// Benchmark report
out.println();
out.println("Benchmark test time report...");
testRunner.report(numberOfRuns, numberOfSteps);
} catch (Exception e) {
e.printStackTrace();
}
}
// Constructor
public BenchmarkRunner(int runs) {
timeRecords = new long[runs];
}
// Starting the timer - to be called by test method
public void startTimer() {
startTime = System.nanoTime();
}
// Stopping the timer - to be called by test method
public void stopTimer() {
endTime = System.nanoTime();
}
// Returning time from the timer
public long returnTime() {
return endTime - startTime;
}
// Recording time from the timer
public void recordTime(int i) {
timeRecords[i] = endTime - startTime;
}
// Reporting benchmark result
public void report(int runs, int steps) {
long total = 0;
long minimum = Long.MAX_VALUE;
long maximum = 0;
for (int i=0; i<runs; i++) {
long t = timeRecords[i];
total += t;
if (t>maximum) maximum = t;
if (t<minimum) minimum = t;
}
long average = total/runs;
out.println("Runs: "+runs+", Ave: "+average/steps
+", Min: "+minimum/steps
+", Max: "+maximum/steps
+" - Per step in nanoseconds");
out.println("Runs: "+runs+", Ave: "+average
+", Min: "+minimum+", Max: "+maximum
+" - All steps in nanoseconds");
out.println("Runs: "+runs+", Ave: "+average/1000000
+", Min: "+minimum/1000000
+", Max: "+maximum/1000000
+" - All steps in milliseconds");
out.println("Runs: "+runs+", Ave: "+average/1000000000
+", Min: "+minimum/1000000000
+", Max: "+maximum/1000000000
+" - All steps in seconds");
}
// Constructor needed as a sample benchmark test class
public BenchmarkRunner() {
}
// A sample benchmark test method
public static long sampleTest(int steps, BenchmarkRunner runner) {
long total = 0;
runner.startTimer();
for (int i=0; i<steps; i++) {
total += i;
}
runner.stopTimer();
return total;
}
}
Note that:
Table of Contents
JVM (Java Virtual Machine) Specification
Java HotSpot VM - JVM by Oracle/Sun
java.lang.Runtime Class - The JVM Instance
java.lang.System Class - The Operating System
ClassLoader Class - Class Loaders
Class Class - Class Reflections
JVM Stack, Frame and Stack Overflow
Thread Testing Program and Result
CPU Impact of Multi-Thread Applications
I/O Impact of Multi-Thread Applications
►Micro Benchmark Runner and JVM Options
►BenchmarkRunner.java - Benchmark Runner Program
emptyLoop() - The Empty Loop Test Method
"-XX:+PrintCompilation" - Watching JIT Compilation Logs
"-XX:+PrintGC" - Watching GC (Garbage Collection) Logs
"-Xms" and "-Xmx" - Avoiding GC with Large Memory Size
Benchmark Affected by Other Running Applications
"-Xint" - Running in Interpreted-Only Mode
Micro Benchmark Tests on "int" Operations
Micro Benchmark Tests on "long" Operations
Micro Benchmark Tests in JIT Compilation Mode
Micro Benchmark Tests on "float" and "double" Operations
JRockit JVM 28.2.7 by Oracle Corporation
JVM Bytecode and Instruction Set