In a recent blog post, Amazon announced changes to its NoSQL database service, DynamoDB, that includes support for Time to Live (TTL) on data stored in the service. Deleting data, based upon time-based thresholds, provides opportunities for organizations to reduce storage costs for data that decreases in value over time.
Having a serverless database service that can scale lends itself to use cases in Ad Tech, Gaming, IoT and applications requiring low latency data access. One use case that Amazon calls out is Lyft and how they have migrated to DynamoDB to persist GPS data for its Ride Location Tracking System.
A challenge that organizations have with applications that have short-term data bursts is data retention. While tracking GPS data may be extremely valuable while a ride is taking place, it loses its value over time, which places a cost-burden on organizations storing this data for long durations.
By using the new TTL feature, organizations can set time thresholds that will automatically delete data once it has reached its expiration timeframe. Jeff Barr, chief evangelist AWS, explains how the TTL feature works:
You can enable this feature on a table-by-table basis, specifying an item attribute that contains the expiration time for the item. Once the attribute has been specified and TTL management has been enabled (a single API call takes care of both operations), DynamoDB will find and delete items that have expired. This processing takes place automatically and in the background and does not affect read or write traffic to the table.
Developers, or Administrators, can set the TTL attribute from the AWS Management Console by providing a value in a DynamoDB Number format which will be interpreted in seconds based upon the Unix Epoch time system. The TTL threshold can be updated from the AWS Command Line Interface (CLI) by calling the update-time-to-live command or the UpdateTimeToLive function can be called from your code.
Image Source: https://aws.amazon.com/blogs/aws/new-manage-dynamodb-items-using-time-to-live-ttl/
You can move items, that have been expired due to TTL, to cold storage or update other DynamoDB tables using AWS Lambda and DynamoDB Triggers. You can also use DynamoDB streams to process or archive the actual deletions.
Some other considerations to be aware of when using DynamoDB TTL include:
- The TTL attribute can be applied to new or existing tables, but it cannot be an element of a JSON document as it must be a DynamoDB Number type.
- Administrators have the ability to govern access to the TTL attribute through AWS Identity and Access Management (IAM).
- There are no performance penalties for using the TTL attribute. The scans and deletions take place in the background.
- There are no additional charges for using TTL. Customers continue to pay for the storage that the item occupies until it is deleted.
- The TTL feature is available for use as of February 27th, 2017.