As I was reading this week’s issue (55th) and clicking the links I stumbled upon couple of presentations that I thought I’ll mention. Some interesting facts and fiction from the presentations

• Interesting stats about Etsy
  • > 1 billion page views / month
  • $300+ million in sales in 2010
  • $41.1 million in December alone
  • 721 deploys in November
  • Engineering team grew 4x during past 15months
  • Great startup definition
• Work in trunk, branch in code
• Focusing on performance
• Logging and graphing almost everything
• Cool pictures and screenshots
• Perf-related data belongs in your server logs

Here are the presentations.

Configuring and running

There is a skype-bot project at github. It is written in Java and has support for simple functionality. You can clone it via:

git clone https://github.com/toomasr/skype-bot.git

The project is maven based and once you import the project into your favourite IDE that has maven support it should compile out of the box. Before we can actually run the project we need to make couple of adjustments. First make a copy of the project.properties file and make it personal.properties. Then change the username, password and pemfile attributes in the file. Make sure that the folder that has the pem file also has a der file that has the same name (except the extension of course).

Next lets build the project. It is sufficient to just issue a mvn install command. You will end up with a skype-bot-0.0.1-SNAPSHOT.jar in the target folder. Before we can run that we need to start the runtime. See Running the Runtime from the previous post.

Now we are ready to start the bot. Issue a java -jar target/skype-bot-0.0.1-SNAPSHOT.jar. Now expecting that the Runtime is still running and that you updated the credentials in the personal.properties file you should see your need friend come online and you can start a chat.

The bot has the commands that he reacts to defined in the org.zeroturnaround.commands package. But for starters you can tell him “bot: time” and it will tell you the time.

Web Server

The bot also launches a web server on port 2500 that has defined handler on the url http://0.0.0.0:2500/post. You can send a POST message there with the parameters message=? and group=?. This will instruct the bot to issue the message to the group. The group is looked up from the bot’s list of groups that matches the Conversation.PROPERTY.displayname.

So what can we do with this? Most of the todays services have webhooks. For example GitHub has Service Hooks, FogBugz have Web Hooks and others have similar services. This means that you can register a web hook and when an event happens the bot will tell you over Skype (in a group chat for example).

What next?

This is up to you, hopefully you have a friendly bot. Make him tell you important information that you care about and don’t let him waste your time. Check back to the original article if I’ve added anything to the series.

Join the Skype Developer Program

A lot of necessary documentation and also software is behind a login screen at Skype Developer Program. You will need to join the program to continue and log in. This will cost you 10$. I personally don’t like the monetary fee for documentation and a SDK but I’m happy this is not 100$.

Now the biggest challenge has been conquered and you can also open up Skype’s own Getting Started Guide. That guide will go into more details with some of the next steps, so if you get stuck here be sure to check their documentation also.

Create a new project at Skype

Lets create a new project at the developer page. Navigate to My Projects submenu.There you can choose between multiple projects to create.

Right now you need to choose SkypeKit for Desktop. You can name the project something along the lines of chat-bot-demo and fill the rest of the form how ever you like (or they require). Now we have a project in place and we can move on.

Download the SDK and the Runtime

Navigate to your just created project and lets get the SDK and Runtime. SDK is the library (com.skype.*) that you will use to talk to the runtime (headless Skype client). The SDK you can download right away but for the Runtime you need to put in a request and it can take up to several hours to get the download link (usually just 5-10 minutes).

Build a sid-java-wrapper.jar file

The SDK comes in source format only, before we can continue we need to get this into a single JAR file and also install it in your local maven repository. To build it we need ANT to be installed (I presume you do). Issue the following commands now:

toomasr@cigar:~/$ cd skypekit-sdk_sdk/interfaces/skype/java/api/
toomasr@cigar:~/skypekit-sdk_sdk/interfaces/skype/java/api$ ant
Buildfile: ~/skypekit-sdk_sdk/interfaces/skype/java/api/build.xml
 
clean:
   [delete] Deleting directory ~/skypekit-sdk_sdk/interfaces/skype/java/api/bin
 
compile:
    [mkdir] Created dir: ~/skypekit-sdk_sdk/interfaces/skype/java/api/bin
    [javac] Compiling 36 source files to ~/skypekit-sdk_sdk/interfaces/skype/java/api/bin
 
jar:
      [jar] Building jar: ~/skypekit-sdk_sdk/interfaces/skype/java/api/sid-java-wrapper.jar
 
clean-build:
 
main:
 
BUILD SUCCESSFUL
Total time: 3 seconds

We have the sid-java-wrapper.jar file now in that api folder. For easier project setup we will be using maven (and probably for some of you shoot you in the foot as an added bonus). We will need to install the JAR file into our local repository. I will use com.skype as the groupId and skype-sdk as the artifactId. Issue the following command to do the same.

toomasr@cigar:~/$ cd ~/skypekit-sdk_sdk/interfaces/skype/java/api/
mvn install:install-file -Dfile=sid-java-wrapper.jar -DgroupId=com.skype -DartifactId=skype-sdk -Dversion=1.0 -Dpackaging=jar

Run the runtime

Just to check if the runtime starts lets unpack and start the binary. If everything goes successfully you should see something similar. On a Linux machine I had to download couple of different binaries before I got a client that did not die (at least on EC2).

toomasr@cigar:~/downloadss/mac-x86-skypekit-novideo_3.5.1.1041_493734/bin/mac-x86$ ./mac-x86-skypekit-novideo 
SkypeRuntime Copyright (C) 2003-2011 Skype Technologies S.A.
SkypeRuntime Version: 3.4.1/mac-x86-skypekit-novideo_3.5.1.1041_493734
Proprietary and confidential, do not share this application.

Development keypair

Next we need a development keypair. It is a signed RSA private key that you need to request from the Skype developer website. See the screenshot.

Once you have the file downloaded lets convert it to a format that is used by the Skype SDK. This is the openssl command for this. Be smarter than me and don’t accidentally overwrite the downloaded PEM file because they let you download it only once. The keypair is valid for 60 days!

openssl pkcs8 -topk8 -in download-pair.pem -outform DER -out downloaded-pair.der -nocrypt

Create a Skype account for the bot

Go to skype.com and create an account for you bot. Also add this bot to your friend list and log in once with the bot and accept the request.

Lets sum it up

By now we should have the following in place

• Have 10USD less on you bank account
• Skype Developer account
• A project created at Skype Developer page
• Working Skype Runtime
• Skype SDK jar file
• Development Keypair in PEM and DER format
• New contact on your contact list and its username/password

If you are missing any of those or had problems getting them then check out the more in depth documentation at developer.skype.com or post a comment.

Where to go from now? Check out the next part, Getting it Running.

The Plan

Image by Simon Abrams

The Skype bot needs a bit more work than your average IRC bot because of the proprietary nature of Skype. There is much clicking and downloading of certificate stuff to get you started for example. Also I want to show you how to put the bot into action, so we will go beyond just the bot.

Getting started

In this Part I – Getting Started we will go through the steps to get you started with the Skype developer program and download all the necessary software for the bot. The registration process, necessary accounts etc. Once you complete these steps you can develop any other bot out there.

Hacking on the Java part

In this Part II – Getting It Running we will take a Java Skype bot and get it running. It requires a little more than just compile and run so it has its own chapter.

We will also look into how to use this bot to notify about important information to Skype chat rooms.

And maybe something more

If everything went well then you have your own bot running now. There are many ways to go forward and improve the bot and the services. Lets see if we have time to cover those.

Hopefully next time that I have a dumb crazy idea I go have a beer instead.

So in the previous part I gave you an intro of the game and how I stumbled upon the game and how it took over our lunches (now we have recovered and are back to random discussions). In this part we’ll look how to detect the furballs from Fling screenshots.

Abstract

Lets build a web service that accepts screenshots of Fling and sends you back a webpage link of the solution to the image or just bunch of attachments.

Solution

Lets divide and conquer.

• Image/Color recognition to detect the Furballs (covered in this piece)
• Algorithm to solve the puzzles (not covered, but code available)
• Algorithm to draw the 1 step screens (not covered, but code available)
• Email handling (not covered, but code available)

Detecting Furballs

When I started out with this one I had no idea how to do it. I sent an email to one of my friends who has done image recognition and asked him how to get started. He took a look at the screenshot and asked, why do I need to recognize anything, just detect the color and you can tell if it is an empty square or a furball! It sounded so obvious that I was somewhat ashamed for not thinking of this myself :)

Python and PIL

Go get the latest version of PIL installed to have the examples work. Also we’ll be working from the root of the fling solver project. Just clone my BitBucket repo. So lets get started and load up the image and verify that PIL is working. Or you can skip this and take a look at blog_demo_1.py.

shot = Image.open(os.path.join(os.path.dirname(os.path.abspath(sys.argv[0])),       
                               "etc/shots/sample001.png"))                          
shot_pix = shot.load()                                                              
shot.show()

Once you execute this an image should pop up, like the one on the right. Now lets isolate a square and dwell into it.

Because the grid is static it is a matter of figuring out the width and height and the offsets as where the squares start. You can use your top notch vision, pixel ruler or little bit of Python to get this right. Run the following and you will see how I’m using PIL to find the offsets (trial and error). You should see something like this.

x, y, w, h = 2, 70, 45, 45                                                          
# lets choose the leftmost square and visualise it                                  
draw = ImageDraw.Draw(shot)                                                         
draw.rectangle([x, y, x+w, y+h], outline=(255, 255, 255))                           
del draw                                                                            
shot.show()

Ok, excellent. We now know the coordinates. Lets now average the colors. We will iterate through every pixel and take an average of the R, G, B values. We’ll also display the image for debugging purposes.

# lets find the average color
avgR, avgG, avgB = 0,0,0
counter = 0
# load pixels of the image
shot_pix = shot.load()
 
for i in xrange(1, w):
    for j in xrange(1, h):
        counter = counter + 1
        avgR = avgR+shot_pix[x+i, y+j][0]
        avgG = avgG+shot_pix[x+i, y+j][1]
        avgB = avgB+shot_pix[x+i, y+j][2]
 
avgR = avgR/counter
avgB = avgB/counter
avgG = avgG/counter
 
avgC = (avgR,avgG,avgB)
 
draw = ImageDraw.Draw(shot)
draw.rectangle([x, y, x+w, y+h], outline=(255, 255, 255), fill=avgC)
del draw
shot.show()

Okay, looks about right. Now lets do this for the whole image. Here I will be reusing code from the flingsolver package. You can do the same.

fip = ImProcessor('etc/shots/sample002.png')
fip.debug = True
state = fip.process()
fip.show()

Voila, all squares are marked with their average color. Now detecting furballs and not furballs is quite easy. Just make a conditional based on some color. For example I used:

def isBall(...)
...
if avgR in range(60, 90) and avgG in range(90, 130) and avgB in range(36,55):        
	return False
else:                         
	return True

For the complete demo you can run blog_demo_1.py from the flingsolver repository. It should generate all the images shown in this post. Happy hacking your own images!

Next time when I have time I will continue with the solving Fling and writing the email handling.

Problem

You are running Mac OS X (maybe the problem is relevant on other platforms too) and even if you’ve started Tomcat and you can see the Java process in the Activity Monitor or with ps you can’t see the process with jps. You are not able to connect to the process with VisualVM nor JConsole. Bummer.

Solution

Tomcat scripts have a variable called CATALINA_TMPDIR and the java process is forked with

-Djava.io.tmpdir="$CATALINA_TMPDIR"

What happens is that jps and VisualVM/JConsole look for information about processes from the system TMP folder. Once you redefine this they cannot find their information. The solution is to eliminate this variable from the process.

Either execute the commandline that you can copy and paste from

ps aux | grep -i tomcat

and exclude the -Djava.io.tmpdir part or modify the script for your debug session. Thats it.

You’re welcome.

toomasr@cigarillo-2:~$ java -server -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version | grep -i heapsize
uintx ErgoHeapSizeLimit                    = 0                {product}
uintx InitialHeapSize                     := 65011712         {product}
uintx LargePageHeapSizeThreshold           = 134217728        {product}
uintx MaxHeapSize                         := 1040187392       {product}
java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261)
Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode)
toomasr@cigarillo-2:~$ java -client -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version | grep -i heapsize
uintx ErgoHeapSizeLimit                    = 0                {product}
uintx InitialHeapSize                      = 0                {product}
uintx LargePageHeapSizeThreshold           = 134217728        {product}
uintx MaxHeapSize                          = 130862280        {product}
java version "1.6.0_22"
Java(TM) SE Runtime Environment (build 1.6.0_22-b04-307-10M3261)
Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03-307, mixed mode)

If there is an easier way, let me know. Will put this one on my tool belt at the moment.

This post is an intro as how this all started out. I sure hope I get around to write the other posts (see the end of the post for a list of potential subjects).

I stumbled upon an iPhone game called Fling that I still play every now and then (I purchased it about a year ago). The game is a puzzle game and your mission (should you choose to accept it) is to fling furballs into each other and the ones closest to the edges are flung off the board.

See the animated GIF for a live demo or check out the video on Youtube. Once the board has just a single furball you’ve won!

As you progress the number of Furballs per level increases and finding a solution takes more time. At certain levels it takes a lot of time. Kind of reminds me of really difficult chess puzzles that can steal an hour or two from you.

The puzzle got so popular amongst colleagues of mine from ZeroTurnaround that during lunch we would place an iPad on the table with one of the levels and who ever found it interesting would give it a shot at some difficult level.

One time over lunch we theorized on how to solve a puzzle of this sort in algorithmic terms. We had different approaches and time estimates but one of the colleagues showed up with a Python solution the next day. Took him less than 90 minutes (apparently attended a not so interesting lecture). And in terms of time complexity we figured that on every move every furball has len(furballs) - 1 furballs to kick out so (n-1)! should be a rough upper bound estimate.

These 100 lines of Python code started a chain reaction of learning more Python, detecting furballs from screenshots, solving the puzzle by outputting images with the moves, email exchange with the author and prank pitching him as an investor, writing an email frontend (you can send in your screenshot), profiling Python, checking out Jython and hopefully some more blog posts on the subject.

BOOM! The particular method I needed to call throws an exception “java.rmi.UnmarshalException: unrecognized method hash: method not supported by remote object”. Googling tells me that this usually means that different classes are deployed on the client and server, so I spend the next half an hour making sure that the JAR on the client and server is exactly the same. A complicating condition is that this particular method returns an object from a third-party library, and to avoid conflicts with the user code we rename the package of that library using JarJar.

As I establish beyond reasonable or unreasonable doubt that the JAR really is the same I start looking into other potential issues. As the exception reports the hash to be missing I start looking into how the hash is calculated for the RMI calls. A fair amount of debugger stepping (a useful tip is that you can put a method breakpoint even if you don’t have the code) reveals that sun.rmi.server.Util.computeMethodHash() calculates the hash using the digest of the method name and signature. Not much room for mistakes there, but nevertheless I put a method breakpoint on UnicastServerRef.dispatch(), which is the last call in the stack trace and dump hashes for all methods in the target class from a static map.

Next I make sure that the hashes are the same on client and server. Decompiling the RMI Stub shows that it includes the precalculated hashes in the call. First check on a different method that I know to work shows that the hashes are the same. But the hash for the problematic method is different in the Stub than in the server map. WTF is wrong?

Here I probably should have verified the next assumptions, but luckily an epiphany interferes. Turns out that the rmic compiler is called before the third-party library is renamed, so the hash is calculated with the wrong packages in the signature. As the JarJar will then rename the signature in the Stub class, it is almost impossible to notice. However the last remaining question is why the hell did it work at all? The hashes should have been different all the time, not just in the vanilla build. A little digging reveals, that as Proguard commonly renames the classes in RMI signatures it will go and fix the RMI hash for you.

EOF

Furious as we were, we managed to keep our manners and reached out (tried to at least) to the author of of jbrute.info and their hosting provider. Sadly, the only response we ever had other than SMTP MAIL CANNOT BE DELIVERED is a canned email instructing us how to write a formal DMCA takedown notice.

We then turned to starting a blog post and asking your help on what you would do in our shoes. As Karl Marx would put it, “the power of the people is strong”, and before long the community came up with a few nothing-less-than-brilliant ideas. (We are not Communists and are not related to the Tsars in anyway, I promise, even though our operational base in Central Estonia might lead you to think otherwise :P)

In particular, Joe Walnes pointed out that since jbrute.info hotlinks .php files and JavaScript from our website, by “check[ing] the HTTP_REFERER, [we] can conditionally change the contents, and with JavaScript [we] can make the page look however [we] want.”

Sadly, life got busy and almost three months suddenly went by, and this little incident was nearly forgotten. Last week, we stumbled upon JBrute once again… Finally we got around to acting on Joe’s advice and, in the most hackish fashion, decided to take it upon ourselves to deliver a truly custom “DMCA Takedown notice” over two glasses of beer. Here is a screenshot of the finished work :) We encourage you to check out jbrute.info (this time I linked :( ) for yourself.

Just to prove our point, we bring you the pseudo xkcd comic.

I hope you enjoyed our little mischief. We’ll keep you posted on the status of jbrute over the coming days following this post and we’ll follow up next week with the technical details – How to un-hack a website! Keep tuned :)