Hey HN, AWS S3 Express One Zone promises improved latency and throughput but comes with several limitations including only being available in one availability zone and the directory bucket structure.
We developed Accelerated Cloud Storage to offer an object storage system with high performance (up to 10X better than S3 Express One Zone in latency and throughput) and global availability that scales and moves data based on demand. The service is accessible through an S3 compatible Go/Python SDK. ACS has enterprise-grade security compliance (SOC 2 Type 2 pending), and a transparent pay-for-usage pricing model with no egress fees.
You can sign-up on the website or schedule a call with one of our engineers to discuss further.
Why do you say GCS performance isn’t an issue? I would imagine a highly parallel compute system would require higher throughput from object storage? I’m surprised you aren’t I/O bound.
If you want object storage faster than S3 Express One Zone or GCP Rapid Storage without the zonal limitation check out ACS: https://acceleratedcloudstorage.com
You can bring data in and out of the GPU quickly and improve utilization.
Is S3 Express One Zone performance greatly improved to standard S3 like GCP rapid storage? My understanding is S3 Express One Zone is just more cost effective.
> 20x faster random-read data loading than a Cloud Storage regional bucket.
Update: Just read this article[1] which clarifies S3 Express One Zone. Yes, performance is greatly improved, but actually storage costs are 8x more than a standard S3 bucket. The naming S3 Express One Zone is terrible and a bit misleading on pricing changes.
I understand your belief that One Zone implies less expensive, but I’m staunchly in favor of them having it in the name so people know that their data is in a single AZ. The storage class succinctly summarizes faster with lower availability.
Fair, how about instead of S3 Express they call it S3 Max (One Zone). It doesn’t take a rocket scientist to come up with good product names, just copy Apple. Though I suppose what happens when engineers are left up to the marketing. :-)
Yep, I love Apple, follow them closely, own a Mac Studio with an M3 Ultra and a MacBook Pro with an M4 Max, and it's still confusing. :)
I mean, surely a Mac Studio with an M4 Max must be the best, right? It's an entire CPU generation ahead and it's maximum! Of course, it's not... the M3 Ultra is the best.
Hey, there are a couple differences between ACS and S3 Express One Zone.
One is speed, ACS provides up to 5X better performance (latency/throughput) than S3 Express One Zone. We have benchmarks repo here: https://github.com/Accelerated-Cloud-Storage/Benchmarks. Please let me know if you have any questions on this.
Another aspect of this is that ACS buckets are modeled on the traditional S3 bucket structure not the directory buckets of S3 Express One Zone. ACS buckets are not stored in only one availability zone and can be accessed in a performant manner from a region different than the region/az of bucket creation.
The service is currently available in US-East-1 on AWS. We plan on launching multi-region AWS support and SOC2 compliance soon. This should give companies the assurance to use ACS to store their data and the flexibility to use it wherever they are deploying their GPUs in AWS.
I understand your frustration with some of these AI startups, and there is a lot of hype in this space. However, there are a couple meaningful differences between ACS and Redis.
One difference is the engineering effort for integration. If an SWE was using Redis they would have to create custom polices on when to move the data between S3 and Redis, how to store it, for how long and when to move newly written data back to S3. This can impose a significant amount of engineering effort for a team. Redis is also mostly single-threaded whereas ACS is designed from the ground up to be multi-threaded.
Another difference is the interface. ACS is accessible through an S3 compatible Python/Go SDK or a high-performance FUSE mount for a POSIX-like file system interface. With Redis, you would still have to build an interface around it for other engineering teams or customers to use.
The purpose is different in that ACS is meant for datacenter workloads. ACS is optimized for both fast reads and writes to Object Storage. The architecture is different than a typical CDN.
We developed Accelerated Cloud Storage to offer an object storage system with high performance (up to 10X better than S3 Express One Zone in latency and throughput) and global availability that scales and moves data based on demand. The service is accessible through an S3 compatible Go/Python SDK. ACS has enterprise-grade security compliance (SOC 2 Type 2 pending), and a transparent pay-for-usage pricing model with no egress fees.
You can sign-up on the website or schedule a call with one of our engineers to discuss further.