Author Archives: Dom Davis

About Dom Davis

Geek, pedant and sarcastic git.

Where I go find Go is like Go

I’ve been playing with Go recently; writing a little REST server that will do some stuff on hitting various end points. So far so copy-and-paste-from-Stack-Overflow. The issue is that, while my application works, I know at a visceral level that it sucks. I’m using go run to test it, and everything is lumped into one file. As an experienced developer I know this is bad – and it’s going to get worse.

And herein lies the problem. Architecturally I know what I should be doing, but as a Go newbie I don’t know if I should be splitting my file into many files in the main package, or splitting it into lots of little single file libraries, or something else entirely. I’ve picked up the syntax of the language, but the paradigms and idiosyncrasies are still, for the most part, unknown to me.

Remove the architectural knowledge that a few decades bashing away at various languages has given me and what you’re left with is a program that compiles, runs and does some nifty stuff – so it must be good, right? Look at me, I’m a Go developer!

I’m all for teaching people about how computers work, and the fundamentals of programming them, but lets not kid ourselves that we’re raising a generation of software engineers. There’s a world of difference between barfing a small, toy project into existence and writing performant, fault tolerant, scalable software that can drive business. Seriously, if I’d presented what I wrote yesterday as some production code I’d fire me. And yet too many people out there don’t know that they don’t know how to be a software developer.

Sadly, I have no idea what the solution is. Computers are fairly ubiquitous, learning to code is easy, and putting together need little programs that, on the surface of it, look impressive, is reasonably trivial. A good parallel would be the game ‘go‘. It’s trivial to pick up and learn, but takes a lifetime to master.

Now, do I know any friendly Go developers who would be willing to give a gentle code review of some truly awful code? Now that I’ve learned Go I need to learn Go.

Of Neo4j and Docker

So here’s a fun one. We’re using Docker, because, you know, DevOps, startup and all that jazz; and we’re using Neo4j because graphs. Until recently Neo4j didn’t have an official Docker repository, but we’re enterprising people and we know how to build Docker images. We also know how to Google, so lets just use someone else’s Docker image. Which is fine, until you come to use the shiny new official Neo4j Docker image with all the lovely new features it brings.

And then this happens:

Caused by: org.neo4j.kernel.impl.storemigration.StoreUpgrader$UnexpectedUpgradingStoreVersionException: '/var/lib/neo4j/data/graph.db/neostore.nodestore.db' has a store version number that we cannot upgrade from. Expected 'v0.A.3' but file is version ''.

Which is problematic, because you need your 2.1.8 datastore updating to 2.3.1 to run with this image. And Google isn’t being helpful in providing a solution (which is mostly the reason behind this post). Long story short, Neo4j prior to 2.3.0 didn’t play well with Docker. You run docker stop neo4j and it casually sits there for a few seconds before issuing a kill -9 and taking the database down at it’s knees. This became apparent when I took my 2.1.8 database files and ran it using a local (non Docker) install of Neo4j 2.1.8 and it told me:

Detected incorrectly shut down database, performing recovery..

Once I’d done that the official Docker image was more than happy to start up and upgrade the files. So if you’re sat staring at whacking great big stack trace, that culminates in an exception with a blank (or garbled) version string, go fire up a local copy of Neo4j (ideally the same version you’re running in Docker), hand it your data files (or edit the file to point to the files), run neo4j console, wait for it to repair and startup, hit CTRL-c and go use the official Docker image – which shouldn’t suffer from this issue with future upgrades.

Good luck 😉

The communal food and drink area

Deja Vu

Almost exactly a year ago I was stood on the platform at Roughton Road station, my life packed into bags, ready to embark on one of the biggest adventures of my life. It’s hard to believe a year has passed, the time has just flown by. That year has seen Rainbird more than double in size, gain traction, and move to swanky new offices. It’s also seen us accepted onto the MasterCard Start Path program, which is why I’m once again sat on a bed that isn’t mine, away from my family.

The Start Path programme is 6 months long, but it’s a very different beast to Techstars. For starters, only a couple of weeks is spent away from home, kicking off with an Immersion Week in Berlin. Secondly, I don’t think the mental working hours will apply. And thirdly, daily blogging isn’t going to happen (although maybe for the days when I’m away from home).

Despite being run by a big corporate like MasterCard the programme has a startup feel to it, right down to the space we’re using. Today has mostly been introductions, both to MasterCard and the Start Path programme, and to each other as we’re one of 7 startups in this cohort. Tomorrow I get to learn all about payments. In the mean time I actually need to get some work done, something that could be fun given the poor mobile signal and flaky hotel WiFi.

Our offices for the next week

Since arriving in Germany I’ve learned that you need to validate your train tickets (Ben and I got told off for not doing that by the conductor on the train from the airport), that hotels here don’t believe in proper pillows (may require some improvisation), and that pedestrian crossings don’t quite work like they do in the UK (turning traffic seems to still drive at you, and you just have to take it on faith they’re going to yield for you). Oh, and German gummy bears (and gummy sweets in general) are just awesome.

Despite the not-so-mental working hours I seriously doubt we’ll get to see much of the city. It’s autumn, so it’s going to be dark when we leave each evening, and it’s only a short walk from the hotel to the venue with not a huge amount to see en-route, or to explore in the surrounding area. That said, we did get to see a few sights on the train this morning when we weren’t being told off for accidentally fare dodging.

Quantified Terror

So here’s an interesting thought. as part of the always on, always connected quantified self we can now measure terror. As I sit in my bulkhead seat of this RyanAir flight to Berlin, climbing out from the takeoff, I can watch in real time as my heart rate rises well above 100BPM, despite my deploying a barrage of tricks and diversion tactics to remain calm.

My rate spikes as the plane banks, my head studiously down, ignoring the fact that I’m in a pressurised tube hurtling through the sky in a display of what my brain feels to be Brute Force Over Ignorance.

And this state of mild panic actually me coping well. I can think straight enough to write. I don’t yet have that sinking feeling in the pit of my stomach. Clearly I’ve got better at flying.

Not great though. An elevated heart rate and nervous shuffling around at the airport for 15 minutes while waiting for my travelling companion registered as a full 15 minutes of exercise. As we settle into the flight my resting heart rate is now in the high 90s, low 100s

And for the next hour and however many terrifying minutes are left in this flight I can know, with data to back it up, that I do not fly well. Of course, I knew that before, but now it’s quantified.

There now follows a party political rant on behalf of the Common Sense Party

When my Dad died it fell to me to sort out his online presence. I sat down at his fancy HP Touch Screen Media Centre monstrosity and promptly demonstrated why full disk encryption is so vital.

Within 5 minutes I had full access to his computer, and everything on it. This included Outlook Express, which dutifully connected to his email account using the credentials it had handily stored away. 30 minutes later and I owned almost his entire digital life.

Once you have access to someones email, you have access to a huge number of websites they’re registered with. Don’t know the password? No problem, we’ll handily email you a link to reset it to something you do know via an email sent to the compromised account you now own. Oh goody!

But that’s OK, no one is going to steal your computer and get access to your email client. Maybe not (although you might just want to enable full disk encryption, just in case), but then perhaps someone like TalkTalk will slip up and release account details which give them access to your email – because, hey, remembering lots of passwords is hard and what’s the harm in using the same password everywhere?

Now lets just pretend that companies like TalkTalk need to suddenly start storing lots more personal information because the government said so. What’s going to happen here? Are you going to get a well architected, well implemented system with security being on of the basic tenants of its design? Or are they just going to ram something into production that does what’s required and then forget about it?

What we’re talking about is a data store that does nothing for the company’s business model, makes them no money, and is a cost to build and maintain. It’ll also be a treasure trove that hackers will want to target, if only to say they did. I have very little doubt that the whole thing will furnish me with a sequel to my talk on massive cockups in IT.

And this data store does what? It allows you to pin some extra evidence on the sloppy who, lets face it, probably have made enough mistakes to be convicted anyway. Anyone with half a clue and nefarious intentions can easily sidestep this store, flood it with spurious data, or both. Incidentally, anyone with half a clue and perfectly honest intentions can also sidestep the store. It’s very hard to tell the difference.

But it’s just metadata! Yeah… putting the word “meta” in front of it doesn’t make it OK. Happy for me to see your browsing history? Of course you are, you’ve got nothing to hide! What if I told you I could also see all those sites you went to in that incognito window you had open last night? Not so great any more is it? Still happy to hand over “metadata”? Then consider this: if the data is the results of your STD test, then the metadata is that you’ve had a test in the first place. I could go on.

It’s also dangerous. So perhaps your browser connects to a compromised server because someone has injected a malicious advert into an innocuous page you’re browsing. Perhaps that server also contains material the authorities deem to be something you shouldn’t be looking at. They now have proof that you were looking at it, but no context as to why. You’re simply the victim of a drive-by download, but the metadata convicts you of viewing forbidden data. Don’t try to argue, it’s all there in the logs, we know you’re guilty.

But it stops the terrorists! Ah yes, that old chestnut. We must all make sacrifices to our liberties, and to our rights so that we can Maintain Our Way Of Life, because otherwise The Terrorists Win. There is a brutal irony there. The only way it could get more ironic is if it’s the terrorists who hack all this data and use it for nefarious purposes… you honestly couldn’t make this stuff up.

Anyway, on a totally unrelated topic, who can recommend me a good VPN and some strong cryptography packages?

Apple upgrade treadmill

It’s the Apple upgrade season which means I’ve got a lot of shiny new software, and a little bit of shiny new hardware. As a rabid Apple fanboi it’s not been a stellar year. Here’s my breakdown:

iOS9: “Meh”, unless you’ve got an iPhone 6S in which case still “meh”, albeit with a few extra features I’ll likely use rarely.

Watch OS2: “…”, although that’s not a bad thing, I’m very happy with my watch and remain very happy with it.

OSX El Capitan: My exact words were “Did they let Jony Ive out of his white room to look at this before they released it?”. There are some questionable design decisions around Expose.

iPhone 6s: An unexpected upgrade. My wife broke her phone, so I upgraded and gave her my 6. 3D touch is… interesting. The haptic feedback makes me think I’ve broken something in the phone by pressing too hard. As more apps use it it could become a useful feature. As it is I wouldn’t have upgraded if we didn’t need a new phone in the house.

iPad Pro: Want. Badly. I had no idea how I was going to get one last week. Since then I’ve had an unexpected iPhone purchase and need to get a laptop repaired (my wife has not had a good week) so it’s even more unlikely I’ll be getting one. I may need to smile sweetly at work.

My coding role models

There were some interesting tweets the other day about programming role models when we were kids, or, more specifically the lack of them. The generation I’m talking about grew up with the BBC Micro, a computer that I credit for getting me where I am now. Growing up in the Middle East meant limited television and playing inside during the hottest parts of the day. Imagination and playing with friends staved off boredom, as did a fair few hours playing computer games.

I’ll be honest here, mostly I’d go round my friends house to play computer games. He had a Commodore 64 a much better games. Of the games I had for my BBC only Repton 2 and Elite were really any good.

Elite was, in some respects, the best game ever written. This may not be immediately obvious if you look at it in a modern context. The graphics are clunky, the sound is lo-fi and the game relied heavily on your imagination. Look at it in the context of what computers could do in those days and it was a technical tour de force.

As a young kid I was able to fly, trade and fight my way across thousands of planets spanning 8 galaxies. I could upgrade my ship and compete in what felt like a living, breathing universe. This was not a game on rails, this was a sandbox you could explore. Yes, you had to bring your own imagination, but as a kid of 10 I had that in bucketloads.

The fact that Braben and Bell were able to make my lowly BBC simulate all that was astounding for me. I wanted to know how. I was inspired to explore, and learn. And through that exploration I taught myself to program. It means that as the current incarnation of Elite pushes technical boundaries once again, I have a good appreciation of how it’s been done. I can also contribute to the community in the form of joystick maps and video tutorials. The very fact I could afford the gaming rig I use to play Elite: Dangerous is directly related to learning to program and taking the career path I did.

So I would argue there were programming role models when I was a kid. David and Ian, I salute you.

Why tech needs context

6am, and an alarm goes off on my phone. It’s time to get up for work. A discreet tap on my wrist at 7am from my watch alerts me to the fact it’s bin day. Today it’s garden waste, so the brown bin. 7:10, Another alarm on my watch: 5 minutes before I need to leave. Time to get my bag and put on my shoes.

I’m forgetful. I get caught up in things. And there is stuff that I really need to do at certain times. Like leave to catch trains. Or put the bins out. Mundane stuff, but important nonetheless.

But these things are contextual. And other than limited control over time and date I can’t really add context to them. I have 6 alarms relating to work which I can only limit to weekdays. I’d like to further limit them by special events (so only go off on weekdays that are not public holidays, and where I am not on holiday or otherwise out of the office). I’d also like to make them smart when it comes to location; “You are not at home, should I adjust your morning alarm?”

This leads to being able to schedule alarms to go off only if an event has occurred. Remind me to take out the brown bin every other week at 7am on a Monday, but only if I have put something in it since it was last put out. When I’ve done some gardening, “Hey, Siri! I’ve put some stuff in he brown bin”, and I know that I’ll get a reminder to put the bin out at the correct time.

Conceptually these are all easy things to do when considered in isolation, but become much harder when you start considering how it would work in the real world for all the different combinations people would want. Just knowing if it’s a public holiday can be confusing enough for me since my calendars insist on showing ones that are observed in Scotland but not England. Any system making informed decisions off this data would need to know about regional nuances like that. And that’s one of the simpler problems.

Smart reminders are coming though. I can envisage a time when I can just say “Hey, Siri! Remind me I need to buy some more tea for the office tomorrow,” and for it to know to remind me as I pull into the station that I need to deviate to a shop, plus tell me which is closest to my usual route. Then, knowing I paid using the company card, Siri will them prompt me when I get to the office to scan the receipt for my expenses.

We’re tantalisingly close, and I can’t wait until it starts becoming mainstream. In the mean time I’m just glad I managed to wake up on time the first day back after my holiday – I forgot to turn all my alarms back on, which could have been problematic.

Clumsy is the new [FR]agile

I recently gave a quick talk at Agile on the Bench in Norwich entitled ’Clumsy is the new [FR]agile’. The following is a rough transcript of that talk:

Hi, I’m Dom Davis, CTO at Rainbird, and lapsed Agile Evangelist.

I’m going to be honest here. When Paul asked me about coming to Agile On The Bench I fully expected to be sitting in the audience watching someone else talking. You see, I’ve rather fallen out with agile. I don’t recognise what people call Agile these days. I couldn’t even tell you what Agile is anymore.

In my last role I was confidently informed that I didn’t do agile because I didn’t do scrum – which was an interesting statement. Agile isn’t scrum. And Scrum spelled with a capital “Waterfall” isn’t agile either. It may sound harsh, but I can guarantee you that’s what they were practicing.

Part of this persons philosophy was that stand-ups were important, but hard to organise every day. So instead of five short daily stand-ups there was one hour long weekly standup. That’s not a stand-up. That’s an uncomfortable team meeting.

And it’s not just that one person. I’d go so far as to say that most people using Scrum aren’t really being Agile. At best they’re being Clumsy. At worst they’re deluding themselves. The same holds for Kanban, Lean and pretty much any flavour of Agile you care to mention.

The problem is that you cannot prescribe a one size fits all approach to software development. People are different. Teams are different. Companies are different. Trying to force people to work in a specific way simply because it has been prescribed as “Agile” is agile with a capital FR.

What works at company X may not work at company Y. And while it may look all fine and dandy at company X I’m willing to bet they also have issues – they’ve just got an evangelist who’s willing to stand up and talk about the good bits, while glossing over the problems.

This carbon copy approach shows a fundamental misunderstanding of what it means to be agile. Scrum, Kanban, Lean; these are all frameworks, philosophies you can weave into the very fabric of your company, not rigid processes to be mandated and enforced. Constraint and agility are diametrically opposed. One cannot allow the other. A rigid process, by definition, cannot be Agile.

And yet the virtues of Agile have been sung from the highest parapets. So much so that we all now know, at a visceral level, that Agile Is The Way. So if it doesn’t work for some reason we’ll just try another flavour of it. We’ll employ certified scrum masters and hope that if we believe hard enough, and follow the plan rigidly enough, it will all be OK.

It won’t, because you’re addressing the wrong problem. This is not the Agile way. It’s the Clumsy way. It’s only when you need to react fast, to be truly Agile, that you find out you’re actually Clumsy. And you find out the hard way, tripping over your processes and getting tied in knots.

Agile is not a process, it’s a state of being. It means you can act with agility. That you can react to the needs of the business, and to the pitfalls of software development in a timely manner. And that’s it. Everything else is process.

Most truly agile teams will review and change their process regularly: keep what is working at this time, shed what is not. What works today may not work in 6 months because the problem space is dynamic, constantly shifting. They understand this and embrace it. There is no perfect solution, just something that works well enough for now.

Strip away the buzzwords and we’re just talking about project management. Except project management is a dirty word. It’s enterprise-y, and we’re all ninja-rockstar-full-stack developers deploying fleets of containerised micro-services to the cloud. We don’t want to be constrained by Process.

But we do need something. It doesn’t need to be Process with a capital P, but simply building a Kanban board, running daily stand-ups, and declaring ourselves to be “agile” isn’t going to work. There are some fundamentals we need to get right. Without those fundamentals you’re setting yourself up to fail.

There are no quick wins. But there is basic starting point to get to the solutions that works for you, in your team, in your company. In the end it all boils down to communication. How do we communicate the requirements from our stakeholders and users to the development team? How do we communicate progress back to the stakeholders and users?

This could be anything from Post-It notes on a whiteboard and informal meetings when required, all the way up to full blown project management systems. Informal doesn’t scale well, and the more formalised the system the less agility it has, so there is a trade off.

But don’t start with the tools, or the process. Start with what you want to communicate. How do new issues enter the pipeline? How do we make sure that what is developed is what is required? How do we feedback progress?

Get that sorted and you’re well on the way to winning, regardless of how it’s done, or what you call it.

If you’re interested I use a process loosely based on Xanpan at Rainbird. I call this process ’Fred’, purely because people keep asking me what I use and I needed to give it a name. Fred, as practiced now, has little resemblance to Fred when I first called it that. It’s not without its issues.

If you want to hear more I’ll be ranting about Agile and Agile Processes in a longer session called ’Agile Smagile’ at NorDevCon.

Rural Internet

I recently changed both my broadband and mobile phone providers. The reason was simple: I wanted a faster, more stable connection, both at home and while out and about.

Increasingly our lives are revolving around a strong internet connection, and there are certain aspects of my job I can’t even do without being online. Everything is in the cloud; local storage is becoming little more than a cache. And yet, living in Rural North Norfolk 4g coverage is spotty, 3g is rare and GPRS (or even the dreaded No Service) is not uncommon. Even my fibre optic broadband tops out at 13mb/s.

I was on the train with a colleague of mine the other week. They stated that “it’s amazing how useless a laptop is without an internet connection”. I found this to be quite amusing. Perhaps it’s because I’m old enough to remember a time before The Internet and constant, near ubiquitous connections, that I find my laptop (and iPad, and even phone) perfectly usable offline. Unfortunately I am little hamstrung when it comes to looking up information, which can be a practically constant need if I’m trying to do development – hurrah for Dash and offline documentation.

This near ubiquitous connection is a problem though. Too many services assume constant connection and become problematic at best when you drop offline. It annoys me that I have to use the iPad version of Trello on the train because the website doesn’t support offline properly. It’s annoying me to the point where I may actually write a native Kanban style app for OSX as an excuse to learn Swift.

There is still a need for offline version of apps, tools and utilities, but that need is shrinking. In some respects that’s turning those of us who can’t be online constantly into second class netizens.