We’re happy to announce the release of Volt Active Data V9.3. In this release, we improved the uptime and stability of Volt Active Data, re-engineered the Streaming Export subsystem with a faster version that uses far less resources, upped the size of JAR files you can load into Volt Active Data (something you might need if you want to include large ML models in your stored procedure logic), and provided a custom task API for creating flexible logic for repetitive tasks within the database.
Table Of Contents
Improved Uptime when Transactions Result in Differences between Replicas
Volt Active Data is designed to be highly-available with synchronous replication within a cluster (k-safety) and will always maintain data consistency. That is, in the CAP Theorem, a single Volt Active Data cluster is a CP system: partition-tolerant and always consistent. In other words, prior to V9.3, when Volt Active Data detected a consistency error, it could stop the database to avoid data corruption.
In V9.3, Volt Active Data will no longer shut down automatically when inconsistency is detected. If you’ve ever seen the dreaded “FATAL: Hash Mismatch” error you know what this is. Volt Active Data checks a hash of all the SQL queries, inputs, and results for transactions run on the partition replicas. When divergence is detected, previous versions of Volt Active Data brought down the whole database in order to preserve data consistency. In V9.3, Volt Active Data now closes extra data replicas, keeping the database running in a degraded, single-replica state. Availability is preserved, with no compromise to durability, performance, or consistency. This gives your team the ability to fix the non-deterministic code and restore the database to high-availability on your timetable.
Streaming Export Throughput and Reliability Improvements
We’ve re-engineered the Streaming Export subsystem in order to improve its reliability and performance. With all the new streaming features introduced recently (TTL Migrate, Export Tables, Alter Stream, configurable flush intervals), it was time to overhaul the foundations so that you could get real-time streaming of your fast decisions and processing from Volt Active Data . As part of this overhaul we greatly decreased the thread counts and increased the throughput. Keep an eye out for even more improvements that will help you easily integrate Volt Active Data into your fast data pipelines.
Automate From Within Volt Active Data: Custom Task Scheduling
In V9.2, Volt Active Data introduced the Beta of two new features that can speed up your application development:
- Scheduled Tasks – a way to call a stored procedure at regular intervals, with fixed delays between runs, or by specifying a cron-style schedule.
- Directed Procedures – a new type of procedure that runs as a separate transaction on each Volt Active Data partition without having to specify a partition key in the call.
These 2 features are now generally available. Take a look at our blog post with a nice example of how to use these features. In addition, we’ve made Schedules and Tasks totally customizable.
Custom task schedulers allow you to create flexible logic for repetitive tasks. A customized task schedule can use the results of the last run(s) to influence future decisions, allowing creation of complex workflows and “If This, Then That (IFTTT)” type work-scheduling. Customizations fall into three areas:
1. Customize the schedule.
The cron-type scheduling is very flexible, but sometimes you need something more. For example, you could write a custom scheduler that has a ‘holidays list’ and will postpone jobs by a day if they fall on a holiday.
2. Customize the procedure calls.
The built-in schedulers will always use the same procedure and call it with the originally specified inputs, but you might want to check the return values from previous procedure calls to change inputs to the next run or change the procedure that gets run. As an example, a ‘batchupdater’ task could check the number of rows that were updated in the last call and make subsequent calls with a larger or smaller ‘batchsize’ input for the next run.
3. Customize both schedule and procedure calls.
You can fully customize both schedule and calls in order to implement sophisticated workflows and schedules. With full customization, the ‘batchupdater’ task could vary both the batch size and frequency of the call. It could even implement a full workflow to call multiple stored procedures.
We’ve started a new repository for custom task examples. Our first example task watches DR lag time, and writes a warning message to the logfile when the lag reaches a threshold value. If you create a custom task that you think others might find useful, create a pull request. For more information on how to build your own custom tasks, see the Custom Tasks chapter in the Performance and Customization Guide.
Support for Uploading Large Classfiles into Volt Active Data
Some customers use large Java packages and Java ML models in their stored procedure logic. When these JAR files are uploaded into Volt Active Data, they are stored with the schema into a catalog. Prior to V9.3, catalogs were required to be under 50MB. It is now possible to exceed that limit.
I encourage you to take a look at the release notes to get a full view of the improvements delivered in V9.3. Download Volt Active Data V9.3 and give it a try. If you have any questions about Volt Active Data, send us a note at support voltactivedata.com.
