Very informative and well-written post, thank you. Nice evolution of the MLK quote.
What I'm wondering next is, What is the practical take-away for startups and relatively small efforts that are looking to scale? Regardless of tools-stack, what should a forward-thinking developer do? Is the answer to design around a RESTful API specification right from the beginning, then building layers of server-side and client-side code exclusively using that API? etc. etc.
about software that is alive because it's extensible. That is true of your startup too. You don't want to be a "site", you want to be a "service". And that means you want to be an authority for a unique kind of data, that you want your users to create and use.
I think the Google+ data is pretty unique and cool. I like the user experience. But you can't call it a service, which is bad news until they get their crap together.
I'm a strong believer that flowing data puts pressure on software to work correctly. You want a public API because you don't assume that you and your team are world class geniuses who have exhausted the search space of valid use cases for your data... but your customers can, close enough. (A very Amazon virtue: start with the customer.)
You want to have a well-designed interface for yourself and your users because it's so painful to scale, migrate, control security, etc. without it. So sure, I would say start with it as early as you can stand. Make it public as soon as you can. Allow your users to contribute and build on your data and service.
You'll probably treat your public-facing interfaces with different levels of scrutiny than internal-only ones. This is convenient, but it might be a mistake. You don't want to put off security or user data integrity until it's too late.
Having multiple services means that you can scale them independently. This costs some overhead but you'll be able to right-size your hardware, say with appropriate fleets in EC2.
Sorry that's all kind of generic, but that's about as deep as I would go without a real-world example to talk about.
The High Scalability blog is one I would recommend at the leading edge of this thing. I see posts on the front page alone that cover all I've been talking about and more.
Thanks for both the links and the advice - this is making more sense.
It would be fantastic if someone - maybe you, since you've got a high-level 10,000' view down to the on-the-ground detailed experience - could consolidate this into an article and provide a guide on how to build software as a platform from the get-go. Any chance of that? :)
What I'm wondering next is, What is the practical take-away for startups and relatively small efforts that are looking to scale? Regardless of tools-stack, what should a forward-thinking developer do? Is the answer to design around a RESTful API specification right from the beginning, then building layers of server-side and client-side code exclusively using that API? etc. etc.