InfoQ Homepage S3 Content on InfoQ
-
Open Source Skyplane Targets Faster and Cheaper Data Transfers between Clouds
A team of researchers at the RISELab at UC Berkeley has recently released Skyplane, an open source tool to optimize the transfer of large datasets between cloud providers, reducing transfer times and costs.
-
Amazon File Cache Now Generally Available
Amazon recently announced the general availability of File Cache, a managed high-speed cache for processing file data stored in disparate locations. The new service can be linked to multiple sources including on premises network file systems and managed AWS services like Amazon FSx or S3.
-
Cloudflare R2 Storage Generally Available
Cloudflare recently announced the general availability of R2 storage, an S3-compatible object storage without egress charges. The new service provides dynamic functionalities integrating with Cloudflare Workers.
-
Azure Data Explorer Supports Native Ingestion from Amazon S3
Microsoft recently announced the ability to natively ingest data from Amazon S3 into Azure Data Explorer (ADX). The new feature simplifies multi-cloud data analytics deployments, bringing data from Amazon S3 to Azure, without relying on custom ETL pipelines.
-
AWS Backup for Amazon S3 Generally Available
Amazon recently announced the general availability of AWS Backup for Amazon S3, a managed option to centralize data protection for applications storing data on S3. The new feature automates the backup and restore of application data on S3 and combines it with the backup of other services, including instances and databases.
-
Amazon S3 Supports New Checksum Algorithms for Integrity Checking
Amazon S3 recently introduced support of four checksum algorithms for data integrity checking on upload and download requests. Amazon claims that the enhancements to the AWS SDK and S3 API accelerates integrity checking of the S3 requests by up to 90%.
-
Amazon Introduces S3 Batch Replication to Replicate Existing Objects
Amazon recently introduced Batch Replication for S3, an option to replicate existing objects and synchronize buckets. The new feature is designed for use cases such as disaster recovery setup, reduce latency or transfer ownership of existing data.
-
HashiCorp Terraform AWS Provider Introduces Significant Changes to Amazon S3 Bucket Resource
HashiCorp has announced the release of version 4.0 of their Terraform AWS provider. This release introduces significant, breaking changes to the Amazon S3 bucket resource. The release also includes full lifecycle control over default resources, changes to the provider configuration, and improvements to handling plural data sources.
-
Incorrect IAM Policy Raised Questions about AWS Access to S3 Data
An unexpected change in the policy used by AWS Support raised concerns about access to customers' S3 data. The cloud provider reverted the change, stating that the permissions were not and could not be used and published a security bulletin. Security experts suggest steps to detect and prevent similar issues in the future.
-
Amazon S3 Adds New Storage Class for Long-Lived Data and Simplifies Access Management
During the latest re:Invent Amazon announced the S3 Glacier Instant Retrieval storage class, a new storage class for rarely-accessed data that requires milliseconds retrieval. A new bucket owner enforced option lets customers disable the ACLs associated with the bucket and the objects.
-
Cloudflare Challenges AWS with R2 Storage and No Egress Fees
Cloudflare has recently announced R2 storage, a S3-compatible service to store large amounts of data with no egress bandwidth fees associated. An automatic migration of objects from Amazon S3 to Cloudflare R2 will be offered to facilitate the transition or integration for existing AWS deployments.
-
AWS Introduces New Intelligent Tier Options for S3 and EFS
AWS announced a few features and changes to their storage approach for S3 and Amazon Elastic File System (EFS) regarding Intelligent Tiering options during the recent AWS Storage Day. The updates and changes will allow customers to leverage S3 and EFS more efficiently and cost-effectively.
-
Amazon Announces S3 Object Lambda: Run Code Dynamically During Retrieval
Amazon has recently announced S3 Object Lambda, a new serverless feature to add customized code and process data from S3 before returning it to an application. S3 Object Lambda works with S3 GET requests and uses AWS Lambda functions to modify data as it is being retrieved from the object storage.
-
Designing for Failure in the BBC's Analytics Platform
Last week at InfoQ Live, Blanca Garcia-Gil, principal systems engineer at BBC, gave a session on Evolving Analytics in the Data Platform. During this session, Garcia-Gil focused on how her team prepared and designed for two types of failure - "known unknowns" and "unknown unknowns."
-
AWS Outposts Now Supports Amazon Elastic Block Store Local Snapshots
Recently AWS announced that its Outposts service now supports Amazon Elastic Block Store (ESB) local snapshots. With Amazon EBS Local Snapshots on Outposts, customers can store snapshots of their Amazon EBS volumes locally on Amazon S3 on Outposts to meet their data residency and local data processing needs.