Wednesday, May 22, 2013

Java Serialization Performance

The Java program I work on in my day job maintains its data model in a tree-like structure of Java beans. The data volume is not big but still it is significant - about 500K property values in about 30K bean objects.
To ensure our program can restart from point of failure we often have to save this data model to disk. We use custom XML serialization. As this takes quite some time (several seconds) I started researching different alternatives.

It turned out that JAXB is not appropriate for the job as it does not cope well with cyclic dependencies and cross-references. It also requires adding a lot of annotations and our data model employs about 50 classes.

Next I tried standard Java object serialization but it turned out to preform even worse - it saved the data model in 15s in a 10MB file.

A quick search about Java serialization showed an open source library Kryo. I was able to test it very easily as it requires no annotations and almost no code changes. Kryo turned out to be lightning fast! It saved the same data model in just 400ms in a 5MB file. At first I did not believe all our data was saved, so I loaded it back and compared it to the original data but there were no differences.
One of the main reasons for this performance is that instead of reflection Kryo uses dynamic byte code generation via ReflectASM.
I also found another project run by Nate - Spine to be very interesting. It is related to game development. Obviously a very skilled developer.
Kudos Nate, great job!

Wednesday, May 15, 2013

Batch convert MOV to MP4 in Ubuntu

for i in *.MOV; do avconv -i $i -b:v 3000k ${i%.MOV}.MP4; done

Video: mjpeg -> libx264
Audio: pcm_s16be -> libvo_aacenc
Compression: 10 -> 1

These are video clips from my Panasonic photo camera