dow.ngra.deno buzzwords allowed

At the moment I am working on reducing the performance overhead of JavaRebel. Once I got rid of all the obvious bottlenecks and optimizations I fired up the profiler and started searching and destroying ad-hoc hotspots. After some time I got to the point, when the bottleneck was in System.currentTimeMillis().

This deserves some explanation. Since JavaRebel reloads code after the class changes, it needs to poll the actual class file for changes. The way we do it is quite lazy and the class itself causes the resource to be polled. However often there will be a lot of polling involved (more or less any method call on the class will cause a check), so we need to filter those calls. What we did is to check if some amount of time (e.g. 500 ms) has passed since the last check:

if (lastCheck + 500  > System.currentTimeMillis())
  return;
lastCheck = System.currentTimeMillis();
// Resource check code

After some optimization everything else got fast enough that this call started to weigh the performance down. System.currentTimeMillis() causes a system call and often a hardware poll, which is comparatively expensive. I needed some way to eliminate it.

After some thought I decided to cache the call. It may sound crazy to cache the current time, but I only needed the resolution of about half a second, so it was OK for me if the time was lagging behind about that much. What I did is put up a separate thread that would cache the current time and update it every half a sec.

public class TimeCacheThread extends Thread {
  public static volatile long currentTimeMillis = 
    System.currentTimeMillis();
 
  public TimeCacheThread() { 
    setDaemon(true);
  }
 
  static {
    new TimeCacheThread().start();
  }
 
  public void run() {    
    while (true) {
      currentTimeMillis = System.currentTimeMillis();
 
      try {
        Thread.sleep(500);
      } catch (InterruptedException e) {
        throw new RuntimeException(e);
      }
    }
  }
}

Then the check became:

if (lastCheck + 500  > TimeCacheThread.currentTimeMillis)
  return;
lastCheck = TimeCacheThread.currentTimeMillis;
// Resource check code

This check is amazingly fast, because instead of doing a system call we are just making a memory read.

[EDIT] The original code did not have the field declared volatile with the following explanation:

You may also notice the complete lack of any synchronization (or volatile keyword) on the currentTimeMillis static field. Since the type is primitive we can only deal with stale data here and I cared about performance more than I cared about missing a couple of checks.

There is a follow-up post (What do we really know about non-blocking concurrency in Java?) that explains some more of the reasoning behind initially omitting the volatile and then still putting it in. [/EDIT]

However once I got that far I decided I could do even better. The currentTimeMillis is of type long and we are still doing some arithmetic on it. In fact the only thing we want to know is if a quantum of time has passed since the last check. And we can do that by using a very simple heartbeat counter:

public class HeartBeatThread extends Thread {  
  public static volatile int counter = 0;  
 
  public HeartBeatThread() {
    setDaemon(true);
  }
 
  static {
    new HeartBeatThread().start();
  }
 
  public void run() {    
    while (true) {      
      try {
        Thread.sleep(500);
      } catch (InterruptedException e) {
        throw new RuntimeException(e);
      }
 
      counter++;
    }
  }
}

Now the checking code becomes just:

if (counter == HeartBeatThread.counter)
  return;
counter = HeartBeatThread.counter;

The counter field is now of int type and we only do an equals check on a DWORD, which should translate directly to 3 very fast hardware instructions (two memory reads and one conditional jump). Hopefully if you ever find yourself polling something a lot this post will come of help. As for me, it improved JavaRebel performance by 70% on some benchmarks.

Tags: java, javarebel

This entry was posted on Monday, October 27th, 2008 at 2:09 pm by Jevgeni Kabanov and is filed under creative, Featured. You can follow any comments to this entry through the RSS 2.0 feed. You can leave a comment, or trackback from your own site.