Category Archives: Code Snippets

Of Neo4j and Docker

So here’s a fun one. We’re using Docker, because, you know, DevOps, startup and all that jazz; and we’re using Neo4j because graphs. Until recently Neo4j didn’t have an official Docker repository, but we’re enterprising people and we know how to build Docker images. We also know how to Google, so lets just use someone else’s Docker image. Which is fine, until you come to use the shiny new official Neo4j Docker image with all the lovely new features it brings.

And then this happens:

Caused by: org.neo4j.kernel.impl.storemigration.StoreUpgrader$UnexpectedUpgradingStoreVersionException: '/var/lib/neo4j/data/graph.db/neostore.nodestore.db' has a store version number that we cannot upgrade from. Expected 'v0.A.3' but file is version ''.

Which is problematic, because you need your 2.1.8 datastore updating to 2.3.1 to run with this image. And Google isn’t being helpful in providing a solution (which is mostly the reason behind this post). Long story short, Neo4j prior to 2.3.0 didn’t play well with Docker. You run docker stop neo4j and it casually sits there for a few seconds before issuing a kill -9 and taking the database down at it’s knees. This became apparent when I took my 2.1.8 database files and ran it using a local (non Docker) install of Neo4j 2.1.8 and it told me:

Detected incorrectly shut down database, performing recovery..

Once I’d done that the official Docker image was more than happy to start up and upgrade the files. So if you’re sat staring at whacking great big stack trace, that culminates in an exception with a blank (or garbled) version string, go fire up a local copy of Neo4j (ideally the same version you’re running in Docker), hand it your data files (or edit the neo4j-server.properties file to point to the files), run neo4j console, wait for it to repair and startup, hit CTRL-c and go use the official Docker image – which shouldn’t suffer from this issue with future upgrades.

Good luck ­čśë

goto fail

If you’ve ever worked with me (or had me review your code), you’ll know that I am a stickler for good coding standards1. I consider a Checkstyle failure a build failure and it does cause the build to break. The reason for this is that we’re human, we make mistakes, and by adhering to good coding standards we can minimise the number of mistakes we make.

One of my absolute bugbears is single line statements with no braces2. My preference is:

if (condition) {
    action();
}

Since this isn’t a post about the ongoing Holy War on where the braces should go I will also accept:

if (condition)
{
    action();
}

I will grudgingly accept:

if (condition) { action(); }

I’m more inclined to accept the above with languages like Javascript where this is a commonly used convention.

Unless you’re coding in Python or similar I will not accept:

if (condition)
    action();

Why not? Well it’s just asking for failure. It’s so easy to just add another line and suddenly have something accidentally be executed that you didn’t want. Python gets round this with indention, which is significant, other languages just trip you up.

How bad can it be, I hear you ask? Well lets consider the following snippet of C:

if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)
    goto fail;
    goto fail;

We are quite literally going to fail. This is a snippet from sslKeyEchange.c which is responsible for this little nasty. Ignoring the fact that the code is a whole series of WTF’s piled on one another it’s a bug that could have been so easily spotted by just using braces.


1 The actual phrase used would imply that my policies on coding belong in WWII era Germany.

2 For languages that support them.

OPML to Markdown

I’ve started using mind maps for various things and wanted to use them for taking minutes in meetings. The issue is that we usually record minutes in a markdown text format so I needed to be able to convert from a mind map to markdown. A quick Google gave me an example Gist in python, but it didn’t work on my Mac. On top of that, once I’d fixed those issues, it wasn’t really outputting things in markdown, just plain text.

I fixed up the code so it now goes from OMPL to Markdown using the root node as the document title, and presents the rest of the nodes as a nested list.

For a given example map:

Example

We get the output:

# Example Map

* 1st Level Child
    * 2nd Level Sibling
    * Another 2nd Level Sibling
* Another 1st Level Child
    * Sibling on Level 2
        * Sibling on Level 3

I’ve actually gone one further locally by setting #!/usr/local/bin/python at the top, moving the script to /usr/local/bin/opml2md, setting chmod 755 and putting some basic argument checking in place. I’ll leave those changes as an exercise for the reader as some bits are specific to my OSX install.

Issues with GlassFish on OSX

I’ve been trying to get GlassFish 4 to install on my laptop on and off for the past two days now. Needless to say, it’s not being going well. Initially I tried the native install (export DISPLAY=localhost:0.0 btw ­čÖé ), it hung trying to configure the domain. Trying to configure a new domain from the command line yielded:

./asadmin create-domain --adminport 4848 --instanceport 8000 domain2
You do not have permission to use port 4848 for domain2. Try a different port number or login to a more privileged account.

Nothing on port 4848 and even running the command as sudo root didn’t work. Next up, the zip install. The comes with a preconfigured domain and a new error trying to start it:

./asadmin start-domain domain1
There is a process already using the admin port of 4848 -- it probably is another instance of a GlassFish Server

Resorting to Google and I finally worked out that my hostname wasn’t in /etc/hosts. Being on the work network I’ve been assigned a hostname by the DHCP server rather than the sulaco.local it usually is. A quick google for setting the hostname on a Mac got me [this][http://blog.psyrendust.com/2008/05/23/change-the-hostname-in-mac-os-x-osx/] and I was able to run

sudo scutil --set HostName sulaco.local
./asadmin start-domain domain1

Lo and behold, GlassFish has started.

Headless Java Process

Someone1 has updated things on OSX so that when a Java process runs it spawns a little coffee cup icon in the task bar which then steals focus. I’m sure whoever did this thought it would be a wonderful idea which no doubt has some benefits if you think really really hard. If, on the other hand, you’re a dev with a build process that spawns JUnit as a forked process many times during the course of the build, then this new behaviour can be considered a tad on the annoying side. Thankfully a quick moan at Google yielded the following magic:

-Djava.awt.headless=true

I’ve added this little gem into our build scripts by using:

<junit printsummary="on" haltonfailure="true" fork="yes">
    <jvmarg value="-Djava.awt.headless=true" />
    :    :    :
</junit>

And I even hear rumour that you can set export JAVA_TOOL_OPTIONS='-Djava.awt.headless=true', although that’s uncorroborated hearsay at the moment – depends how many scripts I need to update as to whether I go for the global solution.


1Oracle, I’m looking at you… and Apple, don’t think I don’t hold you at least partly responsible.

Sleep Sort

I may be late to the party here, but I was introduced to the Sleep Sort algorithm at Sync Conf last week. The code is deceptively simple:

#!/bin/bash
function f() {
    sleep "$1"
    echo "$1"
}
while [ -n "$1" ]
do
    f "$1" &
    shift
done
wait

Of course it’s also hideously inefficient. Using it to sort 3 1 4 1 5 is fine as it only takes 5 seconds, but I wouldn’t like to sort any number much over 10 as you could be there for some time. Does make me wonder if you could speed it up by using milliseconds rather than seconds… and at what point you start getting race conditions.