cool / creative — by Toomas Römer
27
Dec 10

Fling – Game of Furballs

Couple of months ago a mere 100 lines of Python inspired me to hack on a project for hours and hours from my free time for fun and profit, spend 10 USD on a domain name and play a prank on an investor.

This post is an intro as how this all started out. I sure hope I get around to write the other posts (see the end of the post for a list of potential subjects).

I stumbled upon an iPhone game called Fling that I still play every now and then (I purchased it about a year ago). The game is a puzzle game and your mission (should you choose to accept it) is to fling furballs into each other and the ones closest to the edges are flung off the board.

See the animated GIF for a live demo or check out the video on Youtube. Once the board has just a single furball you’ve won!

As you progress the number of Furballs per level increases and finding a solution takes more time. At certain levels it takes a lot of time. Kind of reminds me of really difficult chess puzzles that can steal an hour or two from you.

The puzzle got so popular amongst colleagues of mine from ZeroTurnaround that during lunch we would place an iPad on the table with one of the levels and who ever found it interesting would give it a shot at some difficult level.

One time over lunch we theorized on how to solve a puzzle of this sort in algorithmic terms. We had different approaches and time estimates but one of the colleagues showed up with a Python solution the next day. Took him less than 90 minutes (apparently attended a not so interesting lecture). And in terms of time complexity we figured that on every move every furball has len(furballs) - 1 furballs to kick out so (n-1)! should be a rough upper bound estimate.

These 100 lines of Python code started a chain reaction of learning more Python, detecting furballs from screenshots, solving the puzzle by outputting images with the moves, email exchange with the author and prank pitching him as an investor, writing an email frontend (you can send in your screenshot), profiling Python, checking out Jython and hopefully some more blog posts on the subject.

opinion / rant — by Toomas Römer
14
Dec 10

JEE OSS Container Startup Times – Apples vs Oranges

NOTE: This is the first part of a series (hopefully) that we’re trying out to see if you like it, find it interesting, helpful, whatever. Doing timings actually consumes a lot of time so we would really appreciate any feedback.

I still have vivid memories of the time when JBoss 4 started for 26s on my devel machine with only its vanilla apps deployed (JBoss 5 was twice as slow). I then thought that I’ll rant about it but for no apparent reason I did not.

Time has passed and I got some extra motivation to do a startup timings for containers with their default configurations. The motivation came from a colleague of mine, Anton Arhipov who also helped me time the containers and write this post.

Our plan

Take all containers and app servers out there and compare apples to oranges. Startup a AWS Large Instance (4 EC2 compute units, high I/O performance, 7.5GB of memory) and time the startup of vanilla containers with the latest JDK (1.6.0_22 at the time of running the tests). You can check out the configuration of Ubuntu and run your own tests via launching the ami-84fd0bed at Amazon AWS.

WTF, why? Don’t you have anything better to do?

Well, hopefully this will be a start of something bigger. Actually Anton’s idea was to make something similar to Matt Raible’s web framework comparison. Of course for a similar comparison of containers a lot more is needed than just startup time timings. Clustering, session replication, availability of services comes to mind at the moment. See comparison of application servers from Wikipedia.

I’m personally interested in the startup times because there are developers out there who don’t know that the startup times can have an order of magnitudes differences and vendors don’t care too much about developers’ productivity as just pushing out new features.

And we do have better stuff to do but we do need time off from our regular activities.

Results

Let’s start by observing the overall results of the startup times. As we can see on the graph below, the pure servlet containers, Tomcat and Jetty, are way ahead of the full blown containers, which is obvious (remember, we’re comparing apples to oranges here!). Maybe this data doesn’t help you very much, but actually you can make some conclusions out of it: try to implement your application in a way that it doesn’t depend on a heavyweight container with the slow startup time! Obviously, this is not always possible, but it is worth trying.

Next, lets see how the winners compare. Tomcat seems to be a fraction of a second faster than Jetty with Tomcat 7 being the fastest with under a second for the startup time. Comparing Tomcat to Jetty is pretty fair but not very reasonable as both are very fast. The only area where Tomcat really beats Jetty is the shutdown time. Jetty executes a shutdown hook when you try to stop it, and it takes almost as much as it takes to start the container. Tomcat, in opposite shuts down immediately.

The next chart shows the startup times for the heavyweight guys. We’re missing Geronimo in our test, this will be corrected later.

Let’s have a look at the outsider of the shootout – JBoss. We have to mention here is that the measurements we’re done with the default profile. JBoss is a very configurable application server in terms of services it runs, but we measured the default profile just because it seems that it is the most commonly used by developers (our wild guess, do comment if we’re way off).

You can notice a bad tendency for the startup times starting from JBoss 4.2 and growing until JBoss 5.1. We can see an improvement for JBoss 6.0, and there’s a remarkable improvement in startup time for JBoss 7.0 You might wonder what has changed for JBoss 7.0 so that the time improved an order of magnitude? The answer is that JBoss 7.0 is OSGi-enabled and it seems that it is not starting all its services on the startup modular! Again, this is what happens when you compare apples to oranges! To make the results look more real, we’d have to deploy a mid-size application to the container which would enable some of the services on the startup. We’ll do that in our upcoming tests.

Besides the graphs we’ve also have the data in a spreadsheet.

What next?

We see that OSGI containers are really fast to startup. We’re thinking of running the tests with an app deployed so that more modules are started up on startup (read: real life tests). Also we’ve left out commercial containers at the moment. This is due to we wanting to release the Amazon AMIs and not deal with licensing issues. We’ll definitely run tests with commercial containers if there is interested out there (please comment which container would you like to see in the shootout).

tips — by Toomas Römer
28
Oct 10

WP Super Cache and 0 Byte Downloads

You’re using WP Super Cache on your WordPress site and some of your downloads end up 0 bytes? Check your Apache log to see if you’re running out of memory. Look for a line similar to:

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 34953465 bytes) in /var/www/yourwebsite.com/public_html/wp-content/plugins/wp-super-cache/wp-cache-phase2.php on line 286

The reason is that WP Super Cache is doing its best to save your server from some load and the entry point for most of the requests is actually a PHP file from the WP Super Cache. Check your .htaccess file for more details. That script tries to cache your download file which might not fit into your memory limit. Download files are anyways static (unless you add some binary cookies, which you’re probably not) and don’t need another layer of caching.

Solution

In WP Super Cache options locate “Accepted Filenames & Rejected URIs” section and add some extensions there. In this example I’ve added zip jar and exe. Mind the syntax.

rant / tips — by Toomas Römer
26
Oct 10

How to waste couple of hours with the Play! Framework

Quick rant (downgrade) of the Play! Framework, just wasted another hour trying to find out why my configuration is not applied. This is the second time this month so I need to let out some steam.

Generally I’m a big fan of Play! It is easy to use, easy to setup and fast to develop with. Its like Struts with JRebel but feels more fresh. Okay, back to the rant.

Lets say you want to make sure that each time you run your application, the HSQLDB is initialized into the same location.

Well, you go to application.conf. Find the db.url field, comment it in, change it to

db.url=jdbc:hsqldb:file:my/fav/static/location

Done! 10 minutes you notice that

/private/var/folders/Em/EmtX3-Q+EgOCv9GUy0k8dU+++TI/-Tmp-/Jetty_0_0_0_0_8080_webapp____.7fbo9s/webapp/WEB-INF/application/db/db

is used as the DB location. After grepping, doublechecking, debugging and swearing you find in the internals of play that db.url, db.pass, db.user, db.driver are ignored if db=fs is set. WTF? No warning, no nothing that your configuration gets overwritten, aaargh!

Lets say your deployment is not Play! (they advise you to use play run to run apps written using Play! instead of a classic servlet container). Lets say you target Jetty instead.

Well, you deploy your WAR archive to Jetty and develop as you would before but then you discover that your files are no longer reloaded. You also discover that you get tons of database exceptions. After grepping, doublechecking, debuggin and swearing you find in the internals of play that once you ditch the play run and move to a classic container Play! will presume that you must have been smoking crack when you wrote application.mode=dev in your configuration file. It will just override it and make sure that you run in production mode this time. WTF? No warning, no nothing that your configuration gets overwritten, aaargh.

So next time you’re having problems with Play!, do some grepping, doublechecking, debuggin and swearing and find the location where your configuration options are overwritten by some Java code inside play.jar.

cool — by Jevgeni Kabanov
04
Aug 10

Master Debugging Class: RMI

Today we had a most interesting encounter with the quirks of our environment and RMI. We use RMI in our new LiveRebel product to connect the management console to the remote agents. Commonly, the agent code is obfuscated by Proguard. However as obfuscation renames the classes it prevents us from doing any debugging, so I made a vanilla build and started the debugger.

BOOM! The particular method I needed to call throws an exception “java.rmi.UnmarshalException: unrecognized method hash: method not supported by remote object”. Googling tells me that this usually means that different classes are deployed on the client and server, so I spend the next half an hour making sure that the JAR on the client and server is exactly the same. A complicating condition is that this particular method returns an object from a third-party library, and to avoid conflicts with the user code we rename the package of that library using JarJar.

As I establish beyond reasonable or unreasonable doubt that the JAR really is the same I start looking into other potential issues. As the exception reports the hash to be missing I start looking into how the hash is calculated for the RMI calls. A fair amount of debugger stepping (a useful tip is that you can put a method breakpoint even if you don’t have the code) reveals that sun.rmi.server.Util.computeMethodHash() calculates the hash using the digest of the method name and signature. Not much room for mistakes there, but nevertheless I put a method breakpoint on UnicastServerRef.dispatch(), which is the last call in the stack trace and dump hashes for all methods in the target class from a static map.

Next I make sure that the hashes are the same on client and server. Decompiling the RMI Stub shows that it includes the precalculated hashes in the call. First check on a different method that I know to work shows that the hashes are the same. But the hash for the problematic method is different in the Stub than in the server map. WTF is wrong?

Here I probably should have verified the next assumptions, but luckily an epiphany interferes. Turns out that the rmic compiler is called before the third-party library is renamed, so the hash is calculated with the wrong packages in the signature. As the JarJar will then rename the signature in the Stub class, it is almost impossible to notice. However the last remaining question is why the hell did it work at all? The hashes should have been different all the time, not just in the vanilla build. A little digging reveals, that as Proguard commonly renames the classes in RMI signatures it will go and fix the RMI hash for you.

EOF

← Older Entries
Newer Entries →