open source

Elastic is an open-source software company which makes a product, Elasticsearch, that companies use to make cloud widgets do stuff better.

(It's not actually that complex, it's software that lets you build a search engine for your site or app, but that's not really important, and so long as there are no further questions let's all agree that I understand it completely)

Anyway, this week, the company did something that made me stand up and pay attention: it changed its licensing specifically to nobble AWS, Amazon's cloud computing division.

I'm going to back up a bit here to lay the groundwork, because I think this matters, but also know that some of you will be skipping to the next email in your inbox already. No hard feelings. My mum replies whenever I write about games to remind me that she doesn't really care, so it's not the worst criticism I've had.

Open source software is a tricky business. Foundational to the sector is the idea that the actual source code – the basic software itself – is given away free. And not just "free as in beer", to use the cliche, but free to use, re-use and adapt as you see fit. There's a few reasons why you would want to do that, that tend to fall somewhere between politico-philosophical (information wants to be free; software development is inherently collaborative; more coders than you think are literal communists) and bluntly pragmatic (if your software is largely used by other coders, then by giving them access to the source code, you also gain the ability to incorporate their work into your product. Free bug fixes for life! [it does not tend to work out that way]). 

The problem, of course, is that if you give your code away for free, you may struggle to eat. So there are a few solutions that the industry has settled on there, too. Some are inherently low-stakes, but that's fine for some projects: you could just work on it in your spare time and get a day job to pay the bills, or you could ask for donations to fund development. You could, if you're at the consumer end of the spectrum, give the code away for free, but charge for the polished final product. You could – and this is actually quite a popular one – just get hired by a much larger company which relies on your software, and get paid a salary to work on that software anyway, since it's not what the company's in the business of making and it's better for them that it has a full-time developer on it.

But the most popular option that many have settled on is a sort of carrot-and-stick approach. The carrot is that you offer something in return for payments. If you can't make that software – since you're giving it away for free – you make it service. Businesses who use your software can, if they want, pay you for a "support package". These tend to be less like an extended warranty from the consumer world, and more like a guarantee that someone will pick up the phone if you're having a problem, walk you through how to fix it, and, if it can't be fixed, commit to developer time to fix it on their end. It might even come with the promise of dedicated staff, private patches to tweak the software to your needs, or even input into the product roadmap. Heady stuff.

The stick is the software license. Because while open source software is definitionally free to use, it can come with conditions attached. The most famous of those is GPL, the General Public License, which says that you may reuse the code in your own products as you see fit, but you must in turn release anything you make under the GPL. It's a great concept, a viral meme for spreading open source ideas: if you are a hobbyist developer just tooling around with ideas, then incorporating some GPL code into your project means that, even if you don't really care about open source software yourself, you're committed to sharing your changes and expanding the sector accordingly. 

If, on the other hand, you're a big commercial entity, you look at the GPL code and you decide that you want to keep what you build private anyway. So you contact the developers and negotiate a special license to use their code without applying GPL terms: you pay them for that nice support contract, perhaps, or similar.

It's easy to see one problem with the GPL already, for what it's worth: it puts limits on that option. If you're the sole owner of the code you're licensing, you of course have the option to change the license however you see fit. But if you've incorporated someone else's code into your work, then you're bound by the terms of the GPL regardless of how good an offer others might make you.

For that reason, and many others, GPL is now only one of a number of 'public' licenses, a spectrum that extends all the way from effectively-public-domain licenses at one end, to heavily limited "copyleft" agreements like those enabled by the Creative Commons non-commercial, attribution, no-derivatives agreements at the other.

But until recently, this approach worked fine. A company like Elastic could, and did, give away its software for free, and make a sizeable living from selling these support packages to help larger companies achieve their goals with it. 

Then came The Cloud.

If you are using AWS to host your app's backend (say), you might want to have a search engine in it, and you might want that search engine to be powered by ElasticSearch. Great: Amazon, in its goal of making everything smooth, has already incorporated ElasticSearch into its tools! You just have to flick a switch on the AWS dashboard, and you're good to go. If there are any problems, you can write to Amazon. But in practice, you know the scale of AWS: if there are any problems, your job is to change how you work to fit around what AWS offers.

What you lose from having to work around Amazon, you gain in flexibility, of course. You can serve one customer or one million without really doing any work at your back end, without paying for unnecessary capacity, and without having to ever research any providers other than Amazon.

This is a disaster for Elastic.

Amazon doesn't need to pay for support, because it's big enough that it can do it itself. In fact, it's big enough that even if it want to pay for support, it would probably be cheaper just to poach staff from Elastic itself. 

And Amazon's customers don't need to pay for support, because they're not even doing the hard work of installing and maintaining ElasticSearch. They're just flicking a switch on a dashboard. 

Which means that Elastic, and others in the same position, suddenly make no money.

The solution, and the way Elastic is trying to nobble AWS, is something called the 'server-side public license'. To be clear, this isn't Elastic's innovation: it actually came from another open source software company, MongoDB, which found itself in the same situation with its eponymous database software. What the SSPL says is that anyone can use the code for free, but if they sell access to it (like Amazon does) then they have to release all the changes they've made for free, just like with the GPL. But as well as that, they have to release the source code to their "management layers". What that means is buried in legalise, but in practice, it means that Amazon would need to either pay Elastic a licensing fee, or make almost all of AWS open-source. 

Needless to say, Amazon is not eager to make almost all of AWS open-source. The company is emphatically not in favour of vaguely communal sharing of labour, and it is very much in possession of trade secrets that it would like to keep that way.

This is a gambit on Elastic's part. For the time being, there's a third option for Amazon: it can carry on using the old versions of Elastic's software, under the old license, indefinitely. It can even "fork" the software, updating its own version of ElasticSearch using its in house staff. It could try and match the real version's features, or it could just throw up its arms and declare that it's now doing a new product roadmap. If it backfires, Elastic could have signed its death warrant.

But as powerful as Amazon is, Elastic is still the company that makes the better-known internal search engine. There's always the risk that a sort of "AmazonBasics" offering ends up being created, but Elastic can be fairly confident that would-be customers will notice the difference.

"AWS … have been doing things that we think are just NOT OK since 2015 and it has only gotten worse," Elastic's founder, Shay Banon, wrote earlier this week. "If we don’t stand up to them now, as a successful company and leader in the market, who will?"

There's a bigger reason why this all matters, though. Open source software is literally what the internet runs on, but the entire industry is surviving on the goodwill of a few ancient nerds and some midsized companies that are being pummelled by the giants. 

The most graphic demonstration of this was almost six years ago, when the Heartbleed vulnerability was discovered – a catastrophic flaw in OpenSSL, a security technology used on a substantial proportion of the entire web. 

Heartbleed was front page news, and while we will never know how much damage it did (the vulnerability was so bad that it could be completely silently exploited), just the cost of fixing it alone will have run into millions of hours of labour. But, without laying blame at their feet, it's easy to see how it happened: the core development team of OpenSSL is two full time employees, with a total budget of less than $1m a year, mostly in donations.

That's about 90 seconds of Amazon revenue. Amazon, at least in 2014, was using OpenSSL and offering to all AWS customers. As best I can discover, it does not contribute to the project, either in cash or kind.

Til next time,

Alex