I’ve just come home from this year’s All Your Base conference in London’s Barbican Centre. It’s a fascinating place with brutal concrete and tropical gardens looking over skyscrapers and the Guild Hall: I’d love a proper tour some time, but this is rapidly getting off-topic - you’re here to read about databases and the people who manage them.
Once upon a time the DBA was a gatekeeper, tightly controlling access to data. They owned the databases, running on servers that were special snowflakes that nobody was allowed to touch and there were very few of them. Optimisations were hand coded in and deployed manually. The DBA had a superhero complex: they could make things go faster by tweaking indexes or changing where clauses - but they didn’t know or care for anything outside of their databases. This kind of attitude is no longer acceptable: let’s look at some of the ideas discussed at this year’s conference about what a database engineer should be.
The modern database engineer’s manifesto should be all about the mission, protecting the data, eliminating waste, and using data to make decisions. Like their children, their databases aren’t special snowflakes, and can be replaced. We should therefore be eliminating the barriers between software and ops, DBAs and developers, sharing knowledge and learning each others’ tools.
With databases no longer treated as special snowflakes, we can introduce the whole of the ops team to taking care of them through automation, monitoring, and continuous integration and testing. We work together towards removing failure scenarios to the point where failed database servers won’t be noticed by the customer. We might go as far as to introduce Chaos Engineering where we deliberately cause things to fail in order to better understand weaknesses.
By sharing knowledge and prioritising the company’s mission over and above the database servers, we can work together to perform proper risk assessments of database upgrades. By looking at how critical the data is, how clients will interact with the new version, how mature the software and organisation are, we make a much better judgement call on the risks of any given upgrade and we know what to expect when it happens. There should be fewer surprises.
We can apply Maslow’s hierarchy of needs to our databases to better understand how far we are along this road. From the physiological: “is the data available?”, through safety: “are there replicas?”, to love and belonging: “is configuration management and continuous integration used?”, right up to self-actualisation: “is this the right technology to be using?” - and this fits nicely with an infrastructure team’s core set of values.
A set of core values might be a prioritised set of goals from basic data integrity though automation, up to zero-touch ops and reducing the amount of moving parts. We can’t always meet all of these goals, we might need to introduce a few more moving parts to automate things, but they suit as a reminder of what we should be working towards.
Technology has come a long way over the years. MySQL once popular because of its support for multiple storage engines, and is now being replaced with more specialist databases. A wise man once told me that databases are generally only stable after five years, and so all these NoSQL databases that appeared ten years ago have an awful lot of traction in fairly conservative businesses around the world. But we’re still missing many of the tools we want to support this modern way of thinking, and so many of our database engines expect a single primary. Each company ends up writing their own tooling to test, automate and increase resiliency. If we look at the future of databases, it’s the appearance of mature tooling to allow us to treat databases as just another server.
If you’re looking for your next multi-million pound idea, it’s modern tooling for databases - let’s get to the point where databases are a commodity which we can just use. In twenty years, I’d like databases to be just like electricity - I don’t need to know how it works, I just want to use it.
I can highly recommend All Your Base, it’s a really fun event condensed into a single day. The single-track polyglot nature of the conference means there’s something for everybody, and you’ll learn things you didn’t know you needed to. I’ll see you there in 2016.