In 2020, Volt Active Data became easier to operate, easier to develop on, easier to integrate into your data pipeline, and, thanks to our latest Long-Term Support (LTS) release, V9.3, more resilient in production.
Table Of Contents
Easier to Operate in Public and Private Clouds
Volt Active Data V10 introduces the Volt Active Data Kubernetes Operator, Helm charts, and a Prometheus agent. It is now easier than ever to deploy and run Volt Active Data clusters and rely on Kubernetes orchestration to keep them running and to handle routine maintenance. The Operator works for single clusters or pairs of clusters using Cross Datacenter Replication (XDCR). We also introduced support for IPv6 clusters this year.
New Features to Speed Development and Integration
In 2020, we added several features that make it easier and faster to write applications for Volt Active Data and added a new, powerful feature —Topics — that allows you to easily integrate Volt Active Data into your streaming data pipeline:
- Custom Tasks and Schedules (doc, javadoc) — Last year, when we introduced Time To Live (TTL) on tables, we recognized that scheduled data deletions were just the tip of the iceberg in terms of automating data operations within the database, so we added Custom Tasks and Schedules.In V9.3, we gave you total control with an automation API that allows you to fully customize the logic for tasks. A customized task schedule can use the results of the last run(s) to influence future decisions, allowing creation of complex workflows and “If This, Then That” type work-scheduling. Take a look at our DR latency-monitoring example.
- Topics (doc) — V10.1 introduced Topics as a way to easily integrate Volt Active Data into your data pipelines and take advantage of Volt Active Data’s ability to apply complex, stateful logic to your data streams. Topics are easy to use and work with industry standard protocols. Unlike simple streaming systems, Volt Active Data Topics offer a rich vocabulary for low-latency, complex applications, giving you the full range of SQL and Java to express your logic, plus full access to database state to inform your real-time decisions. Topics also allow multiple, concurrent topic readers, each able to work at its own pace. Contact Volt Active Data sales if you would like to schedule a demo of Volt Active Data Topics.
Stability Improvements and a New LTS Release
Many customers have already upgraded to V8.4 LTS, the LTS version we introduced in 2019. Our LTS versions are supported for three years and are proactively updated with critical defect and security fixes. We also added support for software dependencies. For example, in 2020 we added Java 11 support to V8.4LTS.
In mid-2020, we announced V9.3 LTS, giving you a production-worthy version with a longer support term for all the new features from V9 plus two significant advances for Volt Active Data’s stability and uptime.
1) Improved Uptime when Transactions Result in Differences between Replicas
In V9.3, Volt Active Data no longer shuts down to preserve data consistency when inconsistency is detected. Volt Active Data checks a hash of all SQL queries, inputs, and results for transactions run on the partition replicas.
When divergence is detected, previous versions of Volt Active Data brought down the whole database cluster in order to preserve data consistency. In V9.3, Volt Active Data now closes extra data replicas, keeping the database totally consistent by running in a single-replica state, preserving availability with no compromise to durability, performance, or consistency. This gives your team the ability to fix the non-deterministic code and restore the database to high-availability on your timetable.
2) Improved Scalability and Resource Utilization in Export
In V9.3, we re-engineered the Streaming Export subsystem to improve its reliability and performance. With all the new streaming features introduced in V9 (TTL Migrate, Export Tables, Alter Stream, and configurable flush intervals), it was time to overhaul the foundations so that you could get real-time streaming of your fast decisions and processing from Volt Active Data. As part of this overhaul, we also greatly decreased the thread counts and increased the throughput. We also use this same subsystem for our newer streaming feature in V10, Topics.
What We’re Looking Forward to in 2021
We have lots of great plans for 2021.
Must Read: Data Streaming
First off, we are going to continue our work in Kubernetes. Look for support for three-datacenter XDCR, enhancements to the Helm update capabilities, support for newer Kubernetes releases, and security updates.
We also plan to make Topics generally available once we incorporate feedback from the beta, finish our stability testing, and make a few tweaks to improve the speed of streaming.
Mid-year, we will release a V10 LTS product with all the Kubernetes and Topics work stabilized, hardened, and ready to run in production for years.
Finally, with V11, we have a big overhaul of Volt Active Data memory management coming. This rewrite, which we call “Deterministic Storage”, will lay out data in the same order on all replicas and has three primary benefits to Volt Active Data users:
- SQL queries will get deterministic results on all replicas without the need to add extra ‘ORDER BY’ clauses and the extra indexes they require. This will be a huge simplification during application development and a memory savings for many applications.
- There will be fewer determinism problems in production. In V9.3 LTS, determinism no longer causes a fatal error but it will cause Volt Active Data to operate in single-replica mode. With Deterministic Storage, the lack of an ORDER BY on a SQL statement will no longer trip you up in production.
- As part of this rewrite we are also eliminating memory compaction stalls — a frequent source of long-tail latency problems for real-time applications.
As you can see, we accomplished quite a lot in 2020 and have plans to accomplish quite a lot more this year. In the end, 2020 was a very productive year for Volt. As we continue to shape and improve our product, we are always listening to you, the customer, and ready to make the changes you need to make Volt work well for you and your fast data needs.
