JVM Tutorials - Herong's Tutorial Examples - Version 4.23, by Dr. Herong Yang
MultithreadingIoRunner.java - Multiple I/O Threads
This section provides a tutorial example on running multiple I/O intensive threads on a Windows system. Multithreading on I/O intensive operations seems not improving total productivity.
Now we can build a thread launching and monitoring program to FileEditor2.java in multiple threads. Here is my version,
/* MultithreadingIoRunner.java * Copyright (c) 2014, HerongYang.com, All Rights Reserved. */ import java.io.*; import java.util.*; public class MultithreadingIoRunner { public static void main(String[] a) { // Setting configuration int param = 1000; // Default parameter for the working thread int size = 1; // Default number of working threads int interval = 1; // Default monitoring interval in seconds int priority = Thread.NORM_PRIORITY; // Default priority if (a.length>0) param = Integer.parseInt(a[0]); if (a.length>1) size = Integer.parseInt(a[1]); if (a.length>2) interval = Integer.parseInt(a[2]); if (a.length>3) priority = Integer.parseInt(a[3]); System.out.println("Thread parameter: "+param); System.out.println("Number of threads: "+size); System.out.println("Monitoring interval: "+interval); System.out.println("Thread priority: "+priority); // Launching all threads FileEditor2[] threads = new FileEditor2[size]; long[] counts = new long[size]; for (int i=0; i<size; i++) { byte[] data = new byte[param]; Random r = new Random(); r.nextBytes(data); String name = "FileEditor"+"-"+i; try { FileOutputStream fos = new FileOutputStream(name); fos.write(data); fos.close(); } catch (Exception e) { e.printStackTrace(); return; } FileEditor2 t = new FileEditor2(param,name); t.setPriority(priority); t.start(); threads[i] = t; counts[i] = 0; } // Monitoring overall productivity long startTime = System.currentTimeMillis(); long lastTime = startTime; long grandTotal = 0; System.out.println( "Seconds | Productivity per thread | Total | Average"); while (true) { try { Thread.sleep(interval*1000); } catch (InterruptedException e) { System.out.println("Monitor interrupted."); } long curTime = System.currentTimeMillis(); long delta = (curTime - lastTime); long duration = (curTime - startTime); System.out.print(duration/1000+" | "); long total = 0; for (int i=0; i<size; i++) { long c = threads[i].jobCount; long d = c - counts[i]; total += d; System.out.print((1000*d)/delta+" "); counts[i] = c; } grandTotal += total; System.out.println("| "+(1000*total)/delta +" | "+(1000*grandTotal)/duration); lastTime = curTime; } } }
Here are some results of running MultithreadingIoRunner.java with different numbers of threads with the default normal priority:
C:\>"\Program Files\Java\jdk1.7.0_45\bin\java" MultithreadingIoRunner 1000 1 Thread parameter: 1000 Number of threads: 1 Monitoring interval: 1 Thread priority: 5 Seconds | Productivity per thread | Total | Average ... 162 | 501 | 501 | 462 163 | 390 | 390 | 462 164 | 525 | 525 | 462 165 | 409 | 409 | 462 166 | 502 | 502 | 462 167 | 412 | 412 | 462 ... C:\>"\Program Files\Java\jdk1.7.0_45\bin\java" MultithreadingIoRunner 1000 2 Thread parameter: 1000 Number of threads: 2 Monitoring interval: 1 Thread priority: 5 Seconds | Productivity per thread | Total | Average ... 261 | 372 377 | 750 | 645 262 | 359 324 | 684 | 645 263 | 378 364 | 743 | 645 264 | 330 332 | 662 | 645 265 | 289 251 | 541 | 645 266 | 363 346 | 710 | 645 ... C:\>"\Program Files\Java\jdk1.7.0_45\bin\java" MultithreadingIoRunner 1000 4 Thread parameter: 1000 Number of threads: 4 Monitoring interval: 1 Thread priority: 5 Seconds | Productivity per thread | Total | Average ... 302 | 233 197 123 39 | 593 | 588 303 | 216 222 214 6 | 661 | 588 304 | 268 140 204 10 | 623 | 588 305 | 211 221 217 4 | 655 | 588 306 | 250 197 107 27 | 582 | 588 307 | 200 141 114 10 | 466 | 588 ... C:\>"\Program Files\Java\jdk1.7.0_45\bin\java" MultithreadingIoRunner 1000 8 Thread parameter: 1000 Number of threads: 8 Monitoring interval: 1 Thread priority: 5 Seconds | Productivity per thread | Total | Average ... 386 | 106 103 113 104 108 117 114 132 | 900 | 807 387 | 104 133 100 136 103 133 101 131 | 943 | 807 388 | 81 118 90 111 89 121 86 117 | 817 | 807 389 | 114 127 111 90 79 91 73 145 | 835 | 807 390 | 100 128 106 134 106 118 103 128 | 926 | 807 391 | 89 113 102 114 81 96 83 118 | 800 | 807 ... C:\>"\Program Files\Java\jdk1.7.0_45\bin\java" MultithreadingIoRunner 1000 16 Thread parameter: 1000 Number of threads: 16 Monitoring interval: 1 Thread priority: 5 Seconds | Productivity per thread | Total | Average ... 253 | 49 51 52 55 51 51 39 48 50 45 53 47 53 50 49 52 | 799 | 705 254 | 49 44 46 53 49 50 32 50 43 47 54 48 50 42 45 49 | 756 | 705 255 | 46 30 45 53 48 46 39 46 44 49 53 46 46 49 46 47 | 738 | 705 256 | 37 6 40 39 42 42 47 44 39 41 44 42 40 42 44 41 | 637 | 705 257 | 45 33 49 55 47 44 53 49 45 42 44 45 48 43 45 48 | 740 | 705 258 | 46 52 51 30 50 52 60 53 48 52 58 55 54 50 50 50 | 815 | 705 ... C:\>"\Program Files\Java\jdk1.7.0_45\bin\java" MultithreadingIoRunner 1000 32 Thread parameter: 1000 Number of threads: 32 Monitoring interval: 1 Thread priority: 5 Seconds | Productivity per thread | Total | Average ... 265 | 17 20 18 21 10 20 17 19 15 20 18 21 18 19 15 20 ... | 599 | 621 266 | 20 21 16 20 6 21 18 13 15 21 19 20 18 20 18 23 ... | 607 | 621 267 | 17 23 16 23 3 25 18 23 16 24 17 24 20 23 21 22 ... | 662 | 621 268 | 17 20 20 20 3 23 19 23 21 21 20 23 18 24 19 21 ... | 657 | 621 269 | 22 25 20 26 3 22 19 24 18 26 17 21 16 24 21 25 ... | 687 | 621 270 | 19 23 18 20 5 23 17 19 17 20 20 21 18 23 21 24 ... | 653 | 621 ...
The results are showing no particular pattern. It feels like all I/O operations are performed in a single thread at the hard disk level. Multiple threads at the application level does not improve the total productivity at all.
Threads Productivity Percent 1 462 100.00% 2 645 139.61% 4 588 127.27% 8 807 174.68% 16 705 152.60% 32 621 134.42%
Last update: 2014.
Table of Contents
Downloading and Installing JDK 1.8.0 on Windows
Downloading and Installing JDK 1.7.0 on Windows
java.lang.Runtime Class - The JVM Instance
java.lang.System Class - The Operating System
ClassLoader Class - Class Loaders
Class Class - Class Reflections
JRockit JVM 28.2.7 by Oracle Corporation
Memory Management and Garbage Collectors
JVM Stack, Frame and Stack Overflow
Thread Testing Program and Result
CPU Impact of Multi-Thread Applications
►I/O Impact of Multi-Thread Applications
FileEditor.java - I/O Intensive Process
FileEditor2.java - Avoiding FileNotFoundException
►MultithreadingIoRunner.java - Multiple I/O Threads
Running Multiple I/O Threads on JRockit
Micro Benchmark Runner and JVM Options
Micro Benchmark Tests on "int" Operations
Micro Benchmark Tests on "long" Operations
Micro Benchmark Tests in JIT Compilation Mode