Author of the post here, figured I'd clarify some things since there seem to be some major misconceptions present.
First off, I don't claim to be an expert, I find that a pretty arrogant title for anyone to use.
I'd like to think I know a thing or two about building highly scalable webservices however, and of course I'm always open to the opportunity to learn if I'm doing things incorrectly.
That said, Picsum is what I use to play around with new technologies and try new things since it's high-traffic enough that I can get some real data on how things perform.
Is it very over-engineered? Absolutely, but that's part of the fun.
When it comes to Picsum, the reason for not pre-processing all the images is that there are simply too many variations with the sizes and variations you can request through the API. For every image, there are 5001 * 5001 * 22 variations that can be requested, and in total, we have just under a thousand source images.
As for running Varnish behind our CDN, this is done for a couple of reasons:
- We can make sure that an image is only processed once simultaneously, even tho the CDN might request it multiple times before it's cache has been filled.
- We can apply optimizations, such as sorting and filtering the query parameters for variations, to achieve a better cache rate. This is not possible to do with the CDN provider we use.
The resources it uses are negligible, the extra latency within the cluster is vanishingly small, and it saves us a lot of extra processing.
Every service within the Kubernetes cluster runs at least two replicas, varnish included, for redundancy and to distribute the load. We're not using separate servers for each layer/component, that'd be wasteful.
As for bandwidth costs, there's no cost for the bandwidth between the CDN and the load balancer, as DigitalOcean does not charge for load balancer bandwidth. There's also no cost for anything behind the load balancer, as this is all internal traffic, either within DigitalOcean or within the Kubernetes cluster itself.
Talking about Spaces, I think you might be confused. Spaces is an object storage, which also happens to have optional CDN capability built-in. Picsum only uses the object storage part, for storing the source images that are used for processing. The reason we use Redis to cache said source images is to avoid having to fetch them from Spaces on every request, as this is rather slow comparatively.
An important distinction here is that Spaces/Redis stores and caches the source images, not the processed ones, which are cached by Varnish and the CDN.
As an aside, since you seem to think that comparing numbers for services with vastly different needs and usecases is worthwhile, Picsum serves a bit over 8TB of traffic a month, and costs less then your setup to run.
It does indeed.
DigitalOcean is kind enough to sponsor the infrastructure, and BelugaCDN provides us with CDN services.
Any other costs I cover out of pocket / via carbon ads.
How did this sponsorship take place?I have a free server from a goverment program for University Students,but i am curious if you contacted them and talked about your project and they agreed.Same for Beluga
We switched over from our old NodeJS backend to a new Go backend recently, and gave the website an overhaul as well, things were a bit shaky during the transition.
I wrote the old codebase ~4 years ago when I was just learning NodeJS, and didn't touch it too much after that, so it was in dire need of being replaced, in particular since it wasn't really written to scale horizontally.
Didn't have any particular issues with NodeJS in terms of performance, just felt like using Go when I was rewriting it.
This site pre-dates the Unsplash license (and before Unsplash had their own API/website even, it started when they were still a tumblr blog), the images used on it is from back when Unsplash still licensed images under CC0.
We're on good terms with the Unsplash team, and think they're awesome.
There are plenty of other good options indeed.
I created it in 2014 since I wanted a service that would give me nice looking pictures, rather than just coloured boxes or similar, for placeholders, there's not anything else to it.
First off, I don't claim to be an expert, I find that a pretty arrogant title for anyone to use. I'd like to think I know a thing or two about building highly scalable webservices however, and of course I'm always open to the opportunity to learn if I'm doing things incorrectly.
That said, Picsum is what I use to play around with new technologies and try new things since it's high-traffic enough that I can get some real data on how things perform. Is it very over-engineered? Absolutely, but that's part of the fun.
When it comes to Picsum, the reason for not pre-processing all the images is that there are simply too many variations with the sizes and variations you can request through the API. For every image, there are 5001 * 5001 * 22 variations that can be requested, and in total, we have just under a thousand source images.
As for running Varnish behind our CDN, this is done for a couple of reasons:
- We can make sure that an image is only processed once simultaneously, even tho the CDN might request it multiple times before it's cache has been filled.
- We can apply optimizations, such as sorting and filtering the query parameters for variations, to achieve a better cache rate. This is not possible to do with the CDN provider we use.
The resources it uses are negligible, the extra latency within the cluster is vanishingly small, and it saves us a lot of extra processing. Every service within the Kubernetes cluster runs at least two replicas, varnish included, for redundancy and to distribute the load. We're not using separate servers for each layer/component, that'd be wasteful.
As for bandwidth costs, there's no cost for the bandwidth between the CDN and the load balancer, as DigitalOcean does not charge for load balancer bandwidth. There's also no cost for anything behind the load balancer, as this is all internal traffic, either within DigitalOcean or within the Kubernetes cluster itself.
Talking about Spaces, I think you might be confused. Spaces is an object storage, which also happens to have optional CDN capability built-in. Picsum only uses the object storage part, for storing the source images that are used for processing. The reason we use Redis to cache said source images is to avoid having to fetch them from Spaces on every request, as this is rather slow comparatively. An important distinction here is that Spaces/Redis stores and caches the source images, not the processed ones, which are cached by Varnish and the CDN.
As an aside, since you seem to think that comparing numbers for services with vastly different needs and usecases is worthwhile, Picsum serves a bit over 8TB of traffic a month, and costs less then your setup to run.