Working at Small Town Heroes building apps for TV shows, we used services for everything: Fastly CDN, Pusher for real-time events, Imgix for image optimization, DataDog for monitoring, CircleCI for builds. Almost no servers of our own. When mobile exploded, even more services joined the stack. The maintenance burden dropped but the risk of external dependencies shot up – NPM outages, CircleCI undocumented changes, DynamoDB inconsistencies, Amazon ELB too slow for our TV show spikes.
Lambda gave us the ability to generate animated GIFs and custom images for kids shows at massive scale with no idle infrastructure. We used it for load testing too – bikini-girls-with-machine-guns spins up PhantomJS instances as headless browsers to hammer our API. But the real thinking framework came from promise theory. A promise is not a contract: it is verifiable, documented, mutually agreed, and the intention is best-effort. You cannot make promises on behalf of other agents. When Amazon goes down, you cannot blame them – keeping your promise to your customer is your responsibility.
The super agent pattern is dangerous. Relying on one provider creates a single point of failure regardless of their uptime. Their internal communication slows as they grow – it took us ages to get Amazon to deploy a new Lambda runtime version. When we asked for 30,000 concurrent Lambda executions, Amazon said they had to check European region capacity and it would take until Monday. That is the abstraction leaking.
The emerging practices are all outward-facing: status pages, exposing internal metrics and error rates to customers, subscribing to outage notifications, exporting logs to your own S3, documenting error codes, publishing post-mortems externally, proactive degradation notices, changelogs, blog posts, conference talks, direct engineer access on Slack, and giving customers a voice in the product roadmap. External services are the next silo to break. The collaboration we built inside companies with DevOps needs to extend to every third-party dependency.
Watch on YouTube – available on the jedi4ever channel
This summary was generated using AI based on the auto-generated transcript.