Jay Gordon

In this slide-free, code and agent heavy walkthrough, we will build an agentic app with the GitHub Copilot SDK.

We will begin by understanding with the Copilot Agent Runtime harness and then progressively build a museum exhibit curator agent. Attendees will create and manage sessions, stream events, expose application-owned data through a custom tool, connect an MCP server, and produce a structured, evidence-based report.

GitHub Copilot SDKs include Java, Node.js, Python, .NET, Rust, and Go support. Although each uses its language conventions and type systems, they all implement the same concepts.

Attendees will see with a working application, a practical understanding of the agent and tool-execution loop, and reusable patterns for building secure, testable Copilot-powered apps.

AI coding agents are becoming users of your APIs, documentation, tools, and services, and they don’t behave like human developers. Experiences that seem intuitive to people can send an agent down the wrong path, but traditional developer-experience testing won’t reveal why.

Agent behavior can be evaluated at scale: you can run the same developer scenario in a sandbox and inspect every action, tool call, error, and outcome. This gives teams a way to test how changes to their guidance affect an agent’s ability to complete real tasks.

In this talk, we’ll introduce Project SCOPE, an open-source system for evaluating developer scenarios across combinations of models, coding-agent harnesses, and extensions such as skills, MCP servers, and instruction files. Drawing on twelve months of experiments, we’ll share how SCOPE helped the Azure Cosmos DB team improve developer experience by identifying gaps in agent skills and documentation, refining that guidance, and evaluating the results.

You’ll leave with a practical approach for measuring agent experience, diagnosing failures, and improving the skills and documentation that developers and their coding agents depend on.