Introduction
Applications typically use a data cache to increase performance, especially where the application predominantly uses read-only transactions. These applications directly update the database for changes in the data. The issue here is that as the load increases then the response time on these updates grows. Databases are not good at executing lots of concurrent transactions with a small number of records per transaction. Databases are much better at executing batched transactions.
Eventually, the database will saturate the CPU or disks and at that point the response time will rise as additional load is added. Conventional in-memory caches are also limited to only storing what can fit in the free memory of a JVM. Once we need to cache more than this amount of data then thrashing occurs where the cache continuously evicts data to make room for other data. The required record must then be read continually thereby making the cache useless and exposing the database to the full read load.
There are several products currently available, including IBM® WebSphere® eXtreme Scale, Oracle Coherence, and Gigaspaces that allow all of the free memory of a cluster of JVMs to be used as a cache rather than just the free memory of a single JVM. This allows the capacity of the cache to scale as more JVMs are incorporated. If these JVMs are on additional physical servers with CPU, memory and network, then this allows scalable servicing of read requests. They can also provide scalable servicing of update requests by leveraging their write-behind technology. The scalability of write-behind caching makes it ideal to handle extreme transaction processing (XTP) scenarios. XTP is defined by Gartner as "an application style aimed at supporting the design, development, deployment, management and maintenance of distributed TP applications characterized by exceptionally demanding performance, scalability, availability, security, manageability and dependability requirements.”(1)
In this paper, we will illustrate how to optimize the performance of an application by leveraging the write-behind pattern with examples using IBM WebSphere eXtreme Scale. The write-behind function batches updates to the back-end database asynchronously within a user configurable interval of time. The obvious advantage of this scenario is reduced database calls and therefore reduced transaction load and faster access to objects in the grid. This scenario also has faster response times than the write-through caching scenario where an update to the cache results in an immediate update to the database. In the write-behind case, transactions no longer have to wait for the database write operation to finish. Additionally, it protects the application from database failure as the write-behind buffer will hold changes through memory replication until it can propagate them to the database.
Defining and Configuring Key Concepts
What is a “write-behind” cache?
In a write-behind cache, data reads and updates are all serviced by the cache, but unlike a write-through cache, updates are not immediately propagated to the data store. Instead, updates occur in the cache, the cache tracks the list of dirty records and periodically flushes the current set of dirty records to the data store. As an additional performance improvement, the cache does conflation on these dirty records. Conflation means if the same record is updated or dirtied multiple times within the buffering period then it only keeps the last update. This can significantly improve performance in scenarios where values change very frequently such as stock prices in financial markets. If a stock price changes 100 times a second then normally that would have meant 30 x 100 updates to the loader every 30 seconds. Conflation reduces that to one update.
With WebSphere eXtreme Scale, the list of dirty records is replicated to ensure its survival if a JVM exits and the customer can specify the level of replication by setting the number of synchronous and asynchronous replicas. Synchronous replication means no data loss when a JVM exits but it is slower as the primary must wait for the replicas to acknowledge they have received the change. Asynchronous replication is much faster but changes from the very latest transactions may be lost if a JVM exits before they are replicated. The dirty record list is written to the data source using a large batch transaction. If the data source is not available, then the grid continues processing requests and will try again later. The grid can offer reduced response times as it scales for changes because changes are committed to the grid alone and transactions can commit even if the database is down
A write-behind cache may not be suitable for all situations. The nature of write-behind means that, for a time, changes which the user sees as committed are not reflected in the database. This time delay is called “cache write latency” or “database staleness”; the delay between database changes and the cache being updated (or invalidated) to reflect them is called “cache read latency” or “cache staleness”. If all parts of the system access the data through the cache (e.g., through a common interface) then write-behind will be acceptable as the cache will always have the correct latest record. It is expected that for a system using write behind that all changes are made through the cache and through no other path.
Either a sparse cache or complete cache could be used with the write-behind feature. The sparse cache only stores a subset of the data and can be populated lazily. Sparse caches are normally accessed using keys since not all of the data is available in the cache and thus queries cannot be done using the cache. A complete cache contains all the data but could take a long time to load initially. A third method is available that is a compromise between these two options. It preloads the cache with a subset of the data in a short amount of time and then lazily loads the rest. The subset that is preloaded is roughly 20% of the total number of records but it fulfills 80% of the requests.
An application using a cache in this manner is typically only used for scenarios which access partitionable data models using simple CRUD (Create, Read, Update, and Delete) patterns.
Configuring the write-behind function
The write-behind function is enabled (in the case of WebSphere eXtreme Scale) in the objectgrid.xml configuration by adding the writeBehind attribute to the backingMap element as shown below. The value of the parameter uses the syntax “"[T(time)][;][C(count)]" which specifies when the database updates occur. The updates are written to the persistent store when either the specified time in seconds has passed or the number of changes in the queue map has reached the count value.
Listing 1. An example of write-behind configuration
<objectGrid name="UserGrid"> <backingMap name="Map" pluginCollectionRef="User" lockStrategy="PESSIMISTIC" writeBehind="T180;C1000"/>
JPA loader
WebSphere eXtreme Scale uses loaders to read data from and write data to the database from the in-memory cache. Starting with WebSphere eXtreme Scale 6.1.0.3, there are two built-in loaders that interact with JPA providers to map relational data to the ObjectGrid maps, the JPALoader and JPAEntityLoader. The JPALoader is used for caches that store POJO’s and the JPAEntityLoader is used for caches that store ObjectGrid entities.
To configure a JPA loader, changes must be made to the objectgrid.xml and a persistence.xml file must be added to the META-INF directory. A loader bean is added to the objectgrid.xml along with the necessary entity class names. A transaction callback must be defined in the objectgrid.xml to receive transaction commit or rollback events and send it to the JPA layer. The persistence.xml file denotes a particular JPA provider for the persistence unit along with provider specific properties.
Exploring an Example Business Case
An online banking website with a growing number of users is experiencing slow response times and scalability issues with their environment. They need a way to support their clients with the existing hardware. Next we’ll walk you through this use case to see how the write-behind feature can help resolve their issue.
Use Case: Portal Personalization
Instead of pulling the user profile information directly from the database, they will preload the cache with the profiles from the database. This means the cache can service the read requests instead of the database. Profile updates were also written directly to the database in the old system. This limited the number of concurrent updates/second with an acceptable response time as the database machine would saturate. The new system writes profile changes to the grid and then these changes are pushed to the database using the write-behind technology. This allows the grid to service these with the usual grid quality of service and performance and it completely decouples the single instance database from the read and write profile operations. The customer can now scale up the profile service simply by adding more JVMs/servers to the grid. The database is no longer a bottleneck as there are vastly fewer transactions sent to the back-end. The quicker response leads to faster page loads and results in a better user experience as well as cost effective scaling of the profile server and better availability as the database is no longer a single point of failure.
In this case, we are using WebSphere eXtreme Scale with a DB2Ò database and the OpenJPA provider. The data model for this scenario is a User which contains one-to-many UserAccount’s and UserTransaction’s. The following code snippet from the User class shows this relationship:
Listing 2: Use case entity relationships
@OneToMany(mappedBy = "user", fetch = FetchType.EAGER, cascade = { Cas-cadeType.ALL })
private Set<UserAccount> accounts = new HashSet<UserAccount>();
@OneToMany(mappedBy = "user", fetch = FetchType.EAGER, cascade = { Cas-cadeType.ALL })
private Set<UserTransaction> transactions = new HashSet<UserTransaction>();
1. Populating the database
The sample code includes the class PopulateDB which loads some user data into the database. The DB2 database connection information is defined in the persistence.xml shown earlier. The persistence unit name listed in the persistence.xml is used to create the JPA EntityManagerFactory. User objects are created and then persisted to the database in batches.
2. Warming the cache
After the database is loaded, the cache is preloaded using data grid agents. The records are written to the cache in batches so there are fewer trips between the client and server. Multiple clients should also be used to speed up the warm-up time. The cache can be warmed up with a “hot” set of data which is a subset of the all the records and the remaining data will be loaded lazily. Preloading the cache increases the chances of a cache hit and reduces the need to retrieve data from back-end tiers. For this example, data matching the database records was inserted into the cache rather than loaded from the database to expedite the execution time.
3. Generating load on the grid
The sample code includes a client driver that mimics operations on the grid to demonstrate how the write-behind caching function increases performance. The client has several options to tweak the load behavior. The following command will load 500K records to the “UserGrid” grid using 10 threads with a rate of 200 requests per thread.
$JAVA_HOME/bin/java -Xms1024M -Xmx1024M -verbose:gc -classpath $TEST_CLASSPATH ogdriver.ClientDriver -m Map -n 500000 -g UserGrid -nt 10 -r 200 -c $CATALOG_1
4. Results
Use of the write-behind feature can lead to an improvement in performance. We ran the sample code using write-through and write-behind for a comparison in response time and database CPU utilization. Data was inserted into the cache that matched the records in the database to avoid the warm up time and produce a consistent read response time so we can compare the write response times. Listing 3 shows the responses times for the write-through scenario with reads taking less than 10 ms compared to the writes which are taking around 450-550 ms. The additional network hop and disk I/O is adding significant time to the transactions. Listing 4 shows the response times for the write-behind scenario where all transactions are committed to the grid. This leads to similar read and write response times as shown in the chart where both operations are taking 2.5-5 ms. The two charts demonstrate how the write-through scenario results in higher response times for updates whereas the write-behind scenario has update times that are nearly the same as the reads. More JVMs can be added to increase the capacity of the cache without changing the response times as there’s no longer a bottleneck with the database.
Listing 3: Chart of response times for write-through cache scenario
Listing 4: Chart of response times for write-behind cache scenario
The database CPU utilization charts illustrate the improvement in back-end load when using write-behind. Listing 5 is a graph of the database CPU usage with the write-through use case. You can see the CPU usage is continuous throughout the run with all updates being immediately written to the database. The CPU oscillates between idle state around 4% to 10-12% when it’s in use. Rather than having constant load on the back-end like the write-through scenario, the write-behind case results in low CPU with load on the back-end only when buffer interval is reached as shown in listing 6. The graph shows that the database is fairly idle at 4% CPU until the three minute interval or change count is hit and then the CPU usage increases for a short time to 12-16% when the batched changes are written to the disk. The write-behind configuration should be tuned to best match your environment with regards to the ratio of write transactions, the same record update frequency, and database update latency.
Listing 5: Chart of database CPU utilization for write-through cache scenario
Listing 6: Chart of database CPU utilization for write-behind cache scenario
Conclusion
This article reviewed the write-behind caching scenario and showed how this feature is implemented with WebSphere eXtreme Scale. The write-behind caching function reduces back-end load, decreases transaction response time, and isolates the application from back-end failure. These benefits and its simple configuration make the write-behind caching a truly powerful feature.
References
1. Gartner RAS Core Research Note G00131036, Gartner Group - http://www.gartner.com/
2. User’s Guide to WebSphere eXtreme Scale
3. WebSphere eXtreme Scale Wiki
Resources
1. WebSphere eXtreme Scale free trial download
2. WebSphere eXtreme Scale V7.0 on Amazon Elastic Compute Cloud (EC2)
Acknowledgements
Thanks to Billy Newport, Jian Tang, Thuc Nguyen, Tom Alcott, and Art Jolin for their help with this article.
