Latest Posts »
Latest Comments »
Popular Posts »

Archive for the ‘creative’ Category

ClassLoaderLocal: How to avoid ClassLoader leaks on application redeploy

Written by Jevgeni Kabanov on June 15, 2009 – 10:51 am

“OutOfMemoryError: PermGen” is a very common message to see after a few redeploys. The reason why it’s so common is that it’s amazingly easy to leak a class loader. It’s enough to hold a single outside reference to an object instantiated from a class loaded by the said class loader to prevent that class loader from being GC-d.

In this post I’ll review how we solved this problem in JavaRebel, and share the solution with you. It’s not a magical solution, but it will help alleviate some of the problems introduced in both libraries and applications in Java EE.

The most common way to leak is to register some kind of a callback object and never deregister it. E.g. look at the following code:

PLAIN TEXT
JAVA:
  1. Core.addListener(new MyListener());

If Core is a part of the framework/platform/container then it will hold to MyListener long after the application was redeployed and the class loader left hanging.

Let’s see if we can do anything to solve this. The Core implementation looks something like this:

PLAIN TEXT
JAVA:
  1. public class Core {
  2.   List listeners = new ArrayList();
  3.  
  4.   void addListener(Listener l) {
  5.     listeners.add(l);
  6.   }
  7.  
  8.   void fireListeners() {
  9.     // Exercise for the reader!
  10.   }
  11. }

The problem is that listeners provides a strong reference to the Listener object. What if we replace it by a weak one?

PLAIN TEXT
JAVA:
  1. public class Core {
  2.   List listeners = new ArrayList();
  3.  
  4.   void addListener(Listener l) {
  5.     listeners.add(new WeakReference(l));
  6.   }
  7.  
  8.   void fireListeners() {
  9.     // Exercise for the reader!
  10.   }
  11. }

Unfortunately although this does solve the problem of GC-ing the class loader, it doesn’t really work. The Listener behind the weak reference will be GC-d at first opportunity and after that it’ll no longer receive any callbacks. To illustrate why it’s a problem the code above is basically equivalent to throwing the reference away altogether:

PLAIN TEXT
JAVA:
  1. public class Core {
  2.   List listeners = new ArrayList();
  3.  
  4.   void addListener(Listener l) {
  5.     // Listener is ignored and GC-d
  6.   }
  7. }

Replacing weak reference with a soft one doesn’t improve the situation, just delays the inevitable a bit further. Both are useful for caches, where objects can be recreated at will, but not in this case where we have an externally created object.

So what do we do? What we’d like to do is have the Listener reference to depend on the class loader somehow. Unfortunately, to the best of my knowledge, there isn’t a ready-made solution for that, and there’s no way to achieve it with any combinations of weak references without causing problems.

What we’d like to have is an ability to add a strong reference to the class loader: in other words have it carry a custom property:

PLAIN TEXT
JAVA:
  1. void addListener(Listener l) {
  2.   ClassLoader cl = l.getClass().getClassLoader();
  3.   List lls = (List) cl.getProperty(“CoreListeners”);
  4.   if (lls == null) {
  5.     lls = new ArrayList();
  6.     cl.putProperty(“CoreListeners”, lls);
  7.   }
  8.   lls.add(l);
  9. }

That would work, wouldn’t it? Well, not quite. We also need to save a reference to the class loaders, so that we could later go over all of them. Here the WeakHashMap is useful:

PLAIN TEXT
JAVA:
  1. Map classLoaders = new WeakHashMap();
  2.  
  3. void addListener(Listener l) {
  4.    //…
  5.   classLoaders.put(cl, Boolean.TRUE);
  6. }

There’s not WeakHashSet in Java, so we’re just using a boolean flag as the value.

So this would probably work, but unfortunately class loaders don’t have a getProperty()/putProperty() API. However, it turns out that with a bit of a hack we can simulate it, by generating a unique class per class loader to hold the properties for us. Let’s see how it’s done!

We start with a little boilerplate:

PLAIN TEXT
JAVA:
  1. class ClassLoaderLocalMap {
  2.   private static Method defineMethod;
  3.   private static Method findLoadedClass;
  4.  
  5.   static {
  6.     try {
  7.       defineMethod = ClassLoader.class.getDeclaredMethod(
  8.          “defineClass”,
  9.           new Class[] {
  10.             String.class,
  11.             byte[].class,
  12.             int.class,
  13.             int.class });
  14.       defineMethod.setAccessible(true);
  15.  
  16.       findLoadedClass =
  17.         ClassLoader.class.getDeclaredMethod(
  18.           “findLoadedClass”,
  19.           new Class[] { String.class});
  20.       findLoadedClass.setAccessible(true);   
  21.     }
  22.     catch (NoSuchMethodException e) {
  23.       throw new RuntimeException(e);
  24.     }
  25.   } 
  26. }

This will give us access to ClassLoader protected methods defineClass() and findLoadedClass() later on. Now let’s setup the basic API:

PLAIN TEXT
JAVA:
  1. public static void put(
  2.   ClassLoader cl,
  3.   Object  key,
  4.   Object value) {
  5.   // Synchronizing over ClassLoader is safest
  6.   synchronized (cl) {
  7.     getLocalMap(cl).put(key, value);
  8.   }
  9. }
  10.  
  11. public static Object get(
  13.   Object key) {
  14.   // Synchronizing over ClassLoader is safest
  15.   synchronized (cl) {
  16.     return getLocalMap(cl).get(key);
  17.   }
  18. }

getLocalMap() method should return a map of entries associated with the class loader. How should that work?

Next we introduce a map from class loaders to unique holder class names. We also introduce a nextHolderName() method that generates unique names:

PLAIN TEXT
JAVA:
  1. private static final Map classLoaderToHolderClassName =
  2.   Collections.synchronizedMap(new WeakHashMap())
  3. private static volatile int counter = 1;
  4.  
  5. private static String nextHolderName() {
  6.   return “ClassLoaderLocalMapHolder$$GEN$$” + counter++;
  7. }

Finally we can implement the getLocalMap() method (to save space I removed all exception handling):

PLAIN TEXT
JAVA:
  1. private static Map getLocalMap(ClassLoader cl) {
  2.   String holderClassName =
  3.     (String) classLoaderToHolderClassName.get(cl);
  4.   if (holderClassName == null) {
  5.     holderClassName= nextHolderName();
  6.     classLoaderToHolderClassName.put(
  7.       cl, holderClassName);
  8.   }
  9.  
  10.   Class holderClass =
  11.     (Class) findLoadedClass.invoke(
  12.       cl,
  13.       new Object[] {propertiesClassName});
  14.  
  15.   if (holderClass  == null) {
  16.     byte[] classBytes =
  17.       buildHolderByteCode(holderClassName);
  18.  
  19.     holderClass = (Class) defineMethod.invoke(cl,
  20.         new Object[] {
  21.           holderClassName,
  22.           classBytes,
  23.           new Integer(0),
  24.           new Integer(classBytes.length)});
  25.   }
  26.  
  27.   return (Map) holderClass
  28.     .getDeclaredField(“localMap”).get(null);
  29. }

The last method to implement is buildHolderByteCode. It’s quite trivial and builds the following class renamed to the unique name:

PLAIN TEXT
JAVA:
  1. public class ClassLoaderLocalMapHolder$$GEN$$X {
  2.   public static final Map localMap = new HashMap();
  3. }

The code can be derived using ASMifier with just a little customization, you can look it up in the full source code.

Although we could now easily implement the original example it makes sense to do just a little bit extra effort and introduce a ClassLoaderLocal, with behavior similar to the ThreadLocal:

PLAIN TEXT
JAVA:
  1. public class ClassLoaderLocal {
  2.   private Object key = new Object();
  3.  
  4.   public Object get(ClassLoader cl) {
  5.     if (!ClassLoaderProperties.containsKey(cl, key))
  6.       return null;
  7.     return ClassLoaderProperties.get(cl, key);
  8.   }
  9.  
  10.   public void set(ClassLoader cl, Object value) {
  11.     ClassLoaderProperties.put(cl, key, value);
  12.   }
  13. }

So the original example now becomes:

PLAIN TEXT
JAVA:
  1. Map classLoaders = new WeakHashMap();
  2. ClassLoaderLocal cll = new ClassLoaderLocal();
  3.  
  4. void addListener(Listener l) {
  5.   ClassLoader cl = l.getClass().getClassLoader();
  6.   List lls = (List) cll.get(cl);
  7.   if (lls == null) {
  8.     lls = new ArrayList();
  9.     cll.set(cl, lls);
  10.   }
  11.   lls.add(l);
  12.  
  13.   classLoaders.put(cl, Boolean.TRUE);
  14. }

In this code if any listener comes from a freed class loader, then it will be GC-d from both Core.classLoaders and ClassLoaderProperties.classLoaderToHolderClassName, as both are WeakHashMaps and there are no strong references to the class loaders. The generated ClassLoaderLocalMapHolder$$GEN$$X class will also be GC-d along with the class loader, so we have effectively eliminated a class loader leak without explicit cleanup calls from the user.

I hope this code will be useful for someone. I cannot give any guarantees whether it will work or not and it’s clearly a hack (though a solid hack). Please use it if you actually understand what is happening.

If you see a bug in the code or have a good suggestion, please be sure to comment. There could be a free JavaRebel license in it for you :)

Full source code: ClassLoaderLocalMap.java, ClassLoaderLocal.java.


Posted in Featured, creative | 31 Comments »

Where did my OOM go?

Written by Ivo Mägi on March 24, 2009 – 1:54 pm

Editor: We have a guest appearance from a CDO of a Fortune 500 company (actually not Fortune 500, but yes, a development manager for 200 devs). The twist? He has a compiler installed and apparently an editor too.

The other day I was playing around with a tool for memory analysis and wrote a small snippet using which I would be able to test the new tool. Quickly (Editor: yeah right quickly) created the following and executed it:

[code language='java']
import java.util.*;
class Leak {
static List list = new ArrayList();
public static void main(String[] args) {
for (int i = 0; i >= 0 ;) {
list.add(i++);
}
System.out.println("I will either reach here or die trying (with OOM)");
}
}
[/code]

And what do you think I was displayed by my command prompt (editor: CDOs don't have a command prompt):

Option A:
[code language='java']Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2760)
at java.util.Arrays.copyOf(Arrays.java:2734)
at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
at java.util.ArrayList.add(ArrayList.java:351)
at Leak.main(Leak.java:6)
[/code]
Option B:

PLAIN TEXT
CODE:
  1. I will either reach here or die trying (with OOM)

Well, as I found out, it doesn’t print out anything. As this is now two years from my last real Java development experience (got demoted to C-level), I went to my hardcore Java hackers. For 10 minutes we all stood with bedazzled faces, before it struck – the memory will be allocated in a way that there is no room for new OutOfMemoryError() to be created.

If you execute the above with 64MB heapsize (which is the default heapsize):

PLAIN TEXT
CODE:
  1. C:\work\snippets\leak java -Xmx64m Leak
  2. C:\work\snippets\leak

But if you increase (well actually modify) the heap size a bit:

PLAIN TEXT
CODE:
  1. C:\work\snippets\leak java -Xmx65m Leak
  2. Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
  3.         at java.util.Arrays.copyOf(Arrays.java:2760)
  4.         at java.util.Arrays.copyOf(Arrays.java:2734)
  5.         at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
  6.         at java.util.ArrayList.add(ArrayList.java:351)
  7.         at Leak.main(Leak.java:6)
  8. C:\work\snippets\leak

Moral of the story? I definitely would not want to be fixing a production system dying like this (Editor: lucky you, you don't have to) – most likely I would be mad way before I could have grasped anything about the reasons …


Tags: , ,
Posted in cool, creative, humour | 2 Comments »

The Mysteries of “Protected”

Written by Jevgeni Kabanov on March 13, 2009 – 3:08 am

This post is a follow up to the Ultimate Java Puzzler. I was going to name it “The Most Penultimate Java Puzzler Ever to Exist”, but it just doesn’t have the same ring to it anymore :)

The previous post was about the weird relation between default visibility and method overriding, which in the end meant that @Override is broken in some cases. Today we’ll take a look at protected.

The main issue with protected in Java is that unlike C++ and most other OO languages it can pose in one of two incarnations — either as visibility from subtypes or as visibility in the same package (the latter is just like the default visibility). This can sometime lead to some confusion, but I have a specific case of interest to offer you.

In the previous post there already was one example involving protected:

PLAIN TEXT
JAVA:
  1. public class /*p1.*/Main {
  2.   public static void main(String[] args) {
  3.     C1 c = new p2.C3();
  4.     System.out.println(c.m());
  5.   }
  6. }
  7. public class /*p1.*/C1 {
  8.   /*default*/ int m() {return 1;}
  9. }
  10. public class /*p1.*/C2 extends C1 {
  11.   protected int m() {return 2;}
  12. }
  13. public class /*p2.*/C3 extends p1.C2 {
  14.   protected int m() {return 3;}
  15. }

The relevant part in this example is that protected method is invoked via default visibility, but overridden via subtyping visibility. In case you don’t remember anymore the output is “3″. This shows that visibilities and overriding can be mixed pretty much freely. It is also a good example as it demonstrates why the original phrasing in the JVM spec was wrong. In the original phrasing for the method C3.m to override a method C1.m the latter had to be accessible from the former. In our example this is obviously not the case.

The corrected phrasing, available in the addendum, refers to the JLS notion of overriding, which adds transitivity. That is if C3.m overrides C2.m and C2.m overrides C1.m then C3.m overrides C1.m.

But getting back to the topic — when playing around with JavaRebel I stumbled on an even more interesting example where this dual nature of protected makes it act unexpectedly. The setup starts like this:

PLAIN TEXT
JAVA:
  1. public class /*p1.*/Main {
  2.   public static void main(String[] args) {
  3.     p2.C2  c = new p2.C2();
  4.     System.out.println(c.m());
  5.   }
  6. }
  7. public class /*p1.*/C1 {
  8.   protected  int m() {return 1;}
  9. }
  10. public class /*p2.*/C2 extends p1.C1 {
  11. }

This compiles and behaves as expected outputting “1″. A question I’d like to ask is: what happens if for some reason later you override the method m() in C2?

PLAIN TEXT
JAVA:
  1. public class /*p1.*/Main {
  2.   public static void main(String[] args) {
  3.     p2.C2  c = new p2.C2();
  4.     System.out.println(c.m());
  5.   }
  6. }
  7. public class /*p1.*/C1 {
  8.   protected  int m() {return 1;}
  9. }
  10. public class /*p2.*/C2 extends p1.C1 {
  11.   protected  int m() {return 2;}
  12. }

Stop here for a second, take a look at it and hazard a guess…

roulette

The example seems similar to the previous one, so you’d expect that the output is “2″. But is it?

Let’s evaluate this from the point of compiler and JVM. Initially the call to C2.m from Main resolved to C1.m(), which is in the same package with Main. When we override the method this is no longer the case and actually the method isn’t accessible anymore. So the correct answer is that the code won’t compile.

This is strongly counterintuitive to me, mainly because I always assumed that overriding a method should never affect its caller in such a way. This also isn’t limited to a compile-time error, as it’s quite possible to compile the C2 without recompiling the rest of the classes and get an IllegalAccessError at runtime. So in my book this reads quite unnatural.

There are definitely more creative ways to f**k up with protected you can come up with, by moving from the “default” incarnation to the subtype one. If you know a good one do post it in the comments.

P.S. I’d love to see what weirdness would come if they’d actually introduce the module keyword as well. Luckily it’d be limited mostly to compile time, but as you saw even there things can get tricky.


Posted in creative | No Comments »

Is the Relational Database Not an Option in Cloud Computing?

Written by Jevgeni Kabanov on February 20, 2009 – 10:16 pm

InfoQ: Though scalability is a key factor, cloud computing has other advantages that makes it attractive for vendors that do not need to deliver highly scalable applications or services. — you know when something is way overhyped, when it starts to be presented as a solution to all of your problems, even the ones it was never intended for. Also the keywords like “doomed” give a strong hint…


Posted in creative | No Comments »

Correcting the Billion Dollar Mistake

Written by Jevgeni Kabanov on February 1, 2009 – 11:04 pm

Last week I visited Stockholm to speak at the JFokus 2009. The event was quite spectacular, but for me the most interesting part occurred on the evening before the conference. I was sitting at the speaker’s dinner with Rickard Öberg, Kirk Pepperdine, Simon Ritter and a couple of others. For some reason or other I started talking to Simon about the problem that’s recently been on my mind. Perhaps it’s been eating me after legendary Tony Hoare said this:

I call it my billion-dollar mistake. It was the invention of the null reference in 1965. [...] This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years.

Now since I have a background in functional languages, I know that null references are not necessary. Null value is a member of all types, but some types don’t have or need a natural notion of absence. In fact in Haskell there is no such thing as an “absent” value generic to all types — types either have to declare it explicitly or wrap the underlying value into a Maybe type that has a dedicated Nothing value.

So e.g. the List type has a notion of an empty list encoded into the type:

// Roughly: a list is an empty list
// or a pair of a value and a list
data List a = Nil | Cons a (List a)

While the only way to encode an absent structure is to use Maybe:

// To build a car you need at most one engine
// and zero or more wheels :)
buildCar :: Maybe Engine -> [Wheel]  -> Car

So while I was discussing that with Simon, Kirk banged me on the head to get my attention (just joking, banging me on the head is nowhere near enough to get my attention). Apparently they were discussing more or less the same topic with Rickard, and he had a similar solution for Java. I assumed it was the widely discussed @NotNull annotation, but it was way cooler than that.

Basically in the beginning Rickard started by adding @NotNull support to Qi4J (BTW an amazing piece of engineering, definitely check it out), so that they could automatically insert runtime assertions for the parameters having the @NotNull annotations, something like that:

PLAIN TEXT
JAVA:
  1. class Test {
  2.   void transfer(
  3.     @NotNull Account from,
  4.     Account to,
  5.     double amount) {
  6.       //…
  7.   }
  8.  
  9.   public static void main(String[] args) {
  10.     // Succeeds
  11.     transfer(
  12.       new Account(“Bugs Bunny”), null, 1000000.0);
  13.     // Throws NullPointerException
  14.     transfer(
  15.       null, new Account(“Bugs Bunny”), 1000000.0);
  16.   }
  17. }

However soon after that they discovered that they were inserting @NotNull-s everywhere. So they reverted the notion and decided that they will generate the not null assertions for all parameters except the ones marked as @Optional. And this was the point when I thought to myself “Jevgeni, this is exactly what you were looking for!” So I turn to Rickard and I say “Rickard, this is exactly what I was looking for! This is amazing!” In fact this truly is amazing as it’s exactly captures the semantics of the Maybe type that I liked so much in the functional languages.

After that we go into a discussion whether or not it is possible to validate this assertion during compile-time. Opinions were mixed on this one, though I personally am convinced that it shouldn’t be too easy (your opinion on the topic is welcome, also it will make a great master’s thesis topic). Eventually I get an idea and I say “Rickard, I bet I could implement a JavaRebel plugin that would check this at runtime in about half an hour in 50 lines of code”. And of course Rickard goes “No way!”, so I’m challenged. Next day I sit down for half an hour, then for another half an hour (there was no internet!) and voila — I have a working JavaRebel plugin (in less that 50 lines of code) that will make your methods throw an exception if you try to pass a null reference to a parameter not marked as @Optional (the plugin code deserves another post altogether). Of course I have to show this to Rickard (and he goes “No way!”) and we agree to somehow join forces to promote this sane approach (starting with having a single namespace for the @Optional annotation).

So what do we get? The previous example now looks like this:

PLAIN TEXT
JAVA:
  1. @OptionalCheck
  2. class Test {
  3.   void transfer(
  4.     Account from,
  5.     @Optional Account to,
  6.     double amount) {
  7.       //…
  8.   }
  9.  
  10.   public static void main(String[] args) {
  11.     // Succeeds
  12.     transfer(
  13.       new Account(“Bugs Bunny”), null, 1000000.0);
  14.     // Throws NullPointerException
  15.     transfer(
  16.       null, new Account(“Bugs Bunny”), 1000000.0);
  17.   }
  18. }

As you can understand if we make all of the code without the @Optional annotation to throw a NullPointerException for nulls we’ll break a lot of existing code. Therefore at the moment you also have to annotate the class where you want to enable such semantics with @OptionalCheck.

That’s pretty much it — you can download the plugin right away and just drop it in the classpath when JavaRebel is enabled (you’ll need a 2.0 milestone as 1.x was missing the necessary APIs). At the moment both annotations are in the org.optionalalliance package, but when it changes all you have to do is Organize Imports, so I won’t sweat the naming too much. Please do let us know what do you think of the approach and feel free to advocate it further :)

Cheers,
Jevgeni Kabanov

P.S. The plugin along with the source is also avalable in our Maven repository:


Posted in Featured, creative | 31 Comments »
« Older Entries