Let me start by making an arrogant statement:
There is no such thing as eventual consistency. There is only ‘Immediate Consistency’ and ‘Wrong’.
A lot of people would regard me as being unreasonable, but from my perspective I have a chain of reasoning that I don’t think you can fault:
- The purpose of any database is to hold data so someone (or something) can read it. They may not read it directly – it might be part of a total, for example. Or they might indirectly read it – you can check uniqueness by attempting to insert a key, knowing that a failure means we’ve seen this key before.
- Eventual consistency is a technique used to make updates to large databases scale. It means that there is a period of time when different observers could see different values after an update, but eventually the system will settle on a correct value.
- But as per point ‘1,’ the whole point of a database system is that someone will read the data. And the whole point of reading data is to do something. So if someone has read it before a settled value has been reached, they have presumably taken some kind of decision or acted in some kind of non-trivial way. If you are using Eventual Consistency without understanding its limitations, sooner or later you – or a user of your information – will therefore take a decision that is wrong.
- Once they’ve taken a wrong decision it becomes impossible to ‘fix’ the situation simply by getting the database to settle on a correct value. Like the proverbial butterfly causing a hurricane, the consequences of the providing the wrong information will already be spreading throughout the real world. Settling on a correct value in the database after the fact is a case of closing the stable door after the horse has bolted. The only way to fix this involves a time machine.
Given the above, the only way to safely use an Eventual Consistency system is to wait until the ‘eventual’ part has happened. This will work in cases where the data is unimportant (social media) or where it’s not practically possible for users to query the data before it has settled. But in other scenarios Eventual Consistency poses major problems:
- How long do I wait? In a world where latency expectations are increasingly in the single digit millisecond range you can’t sit there and wait before asking for the correct answer. Both the IoT and the 5G telco standard will be working in the low single digit MS range. Since as a general rule EC won’t commit to an absolute time of correctness, you have to take your chances on this one. Anyone who has ever worked on a high performance OLTP system will tell you that having each transaction wait is a really bad idea.
- At what time do I start waiting? If I do a read how do I know it’s not subject to an eventual consistency reconciliation algorithm? According to the point above I have to ‘wait’, but starting when? I can’t rely on a timestamp in the data because that too could be subject to eventual consistency…
- Arbitrary behavior. Eventual consistency systems often use arbitrary algorithms such as ‘earliest transaction’ to decide correctness. In a scenario where two people are trying to spend the last unit of credit this is simply wrong – somebody will end up getting something for nothing. This is not an obscure edge case. I am currently working to help an unnamed and somewhat embarrassed customer because they need to move off an EC technology that leaks revenue.
- Murphy’s Law. EC vendors will point to how rare conflicts are in real life. In both IoT and telco we can easily expect billions of transactions a day, generated by devices in a wild variety of real-world situations. When you get to billions of transactions a day everything that can happen will happen. Things which in theory can’t happen will happen. Explaining that the system is right most of the time won’t cut any ice if the audience is a judge in a class action law suit about disappearing customer credit.
This brings me back to my original point. If the only safe way to use an Eventually Consistent database is to wait an arbitrary amount of time for it to settle on an agreed value, and I don’t know when that time starts – but it’s simply not safe to look at the data before then – for practical purposes “Eventually Consistent” is useless if your system requires accurate data. There are lots of cases where you can get away with it, e.g. social media applications, applications where no updates take place, situations where by sheer luck nobody will ever read the record just after it’s been updated. But in cases where you need accurate and fast data it’s hard to see how it can be used without creating a swarm of problems.
Next time I’ll be making another wild claim: You want fewer features in your next database…
