Patrick makes the case that GenAI adoption in organizations follows a familiar pattern seen with previous technology waves such as cloud, mobile, and DevOps itself. He draws on the Team Topologies framework to argue that a dedicated platform team is the natural organizational structure for scaling GenAI beyond a single pilot team. The talk covers three pillars: platform infrastructure, enablement, and governance, each treated as essential for sustainable adoption.
The platform infrastructure section provides a detailed inventory of the shared services a GenAI platform team should offer. This includes centralized model access with a unified API interface across providers, vector databases for indexing unstructured data, RAG connectors for pulling information from sources like Confluence, Jira, and file shares, abstraction proxies for multi-model routing, caching to reduce cost, and comprehensive observability. Patrick describes metrics specific to LLM operations such as time to first token, token throughput, and cost per request, and argues for prompt tracing, continuous data quality monitoring, and feedback-as-a-service as shared platform capabilities. He also covers model and prompt registries for reuse across the organization.
On enablement, Patrick emphasizes overcoming the fear that engineers feel when confronted with AI, reassuring them that GenAI work is about integration rather than data science. He recommends providing controlled playgrounds for prompt experimentation, local development environments using tools like Ollama and LM Studio, and visual prototyping tools for flow engineering. He addresses the framework debate honestly, acknowledging that current GenAI frameworks change so fast they frustrate developers, but predicting this will stabilize. A significant portion of the enablement discussion focuses on testing and evaluation, where Patrick lays out a progression from exact matching and pattern testing through helper-model evaluation, semantic distance checks, and LLM-as-a-judge approaches.
The governance section covers the trust, licensing, and compliance dimensions of GenAI adoption. Patrick discusses model card inspection for understanding training data and bias, opt-in and opt-out policies for AI training, licensing implications of open-source models, and the European AI Act’s risk-level framework. He introduces the concept of “humble AI” and stresses the need for PII monitoring, prompt injection protection, and guard rails as a service. The talk closes with Patrick positioning the AI engineer as an emerging role that bridges data science and production engineering, and advocating for placing the GenAI platform team alongside existing cloud, DevOps, and developer experience platform teams.
Watch on YouTube — available on the jedi4ever channel
This summary was generated using AI based on the auto-generated transcript.