I spent time today with our team’s head of engineering for Google-sponsored languages. Dart has an interesting history, and is critical to some key services from Google. I’m going to hack around with it this weekend.
[article] Cognitive Surrender. This is when the AI’s output becomes your output. It leads to cognitive debt. Where might you allow yourself your surrender, and how do you protect against it?
[article] The “AI Job Apocalypse” Is a Complete Fantasy. You might roll your eyes at this headline in the face of the stream of “the business is great but we’re laying people off because of AI” stories (see: Cloudflare). But the amount of work to be done isn’t fixed, and this post brings some receipts from the recent past.
[article] Tech job postings hit 3-year high. Tons of open tech jobs, with more added each month. A reconfiguration doesn’t mean the work goes away.
[article] The company that made RAG mainstream is now betting against it. The half-life of a “best practice” in this industry is like 3 months. Pinecone who got famous as a vector database serving RAG use cases, now pushes knowledge upstream into artifacts used by agents.
[blog] Write-Only Code. Is this the final-boss of agentic coding? Maybe. If you are comfortable with this, you’ve knocked down every concern. Or you’ve embraced YOLO engineering.
[article] Designing front-end systems for cloud failure. Since I’m not a frontend guy, I haven’t thought much about this topic. But I liked the insights about degraded frontend experiences and the breadth of things that you need to guard against.
[blog] 10 Lessons for Agentic Coding. Today. Lessons for today. Who knows what the “good practices” will look like tomorrow?
[blog] Monitoring reliably at scale. What if your observability stack depends on the same systems that it monitors? Yikes. Airbnb wanted to break these circular dependencies.
Today’s list has a terrific mix of opinions, not all in agreement. But what’s important is that we’re learning things together.
[article] Agent Skills. Developers are loving these twenty agent skills. They serve the SDLC as a whole and force your agents through the same stages an engineer goes through.
[article] How To Be Direct And Strategic. You can be a straight-shooter and still be thoughtful in how you deliver the message. Good post on how to be strategic and direct.
[article] When an Executive Asks You an Unexpected Question. Related to the above, in some way. It’s not just about the fast and factual answer; you need to understand where the question is coming from.
[article] Designing the AI-native engineering organization. This group says that AI-native engineering looks like shorter planning cycles, more frequent releases, smaller squads of engineers, and more types of contributors.
[article] Agentic Coding is a Trap. I definitely get the arguments here, and the author makes a reasonable case. Some is speculation and anecdotal, but no one really knows how agentic coding is going to impact the field of software engineering.
Autonomous agents are cool and all, but we all know there are plenty of circumstances where we want human review. Your custom built agent might have instructions to stop and get input, but that doesn’t guarantee it happens. The Agent Development Kit recently added a human-in-the-loop feature, and I decided to try it out.
Let’s build an agent that generates product tutorials on-demand. The agent takes in a request, and uses tools to ground its output. Before it publishes a tutorial, it requires explicit approval from outside its own workflow. ADK now offers both a simple boolean approval, and a more sophisticated option. I’ll test out both, using the built-in web UI and the raw REST API. FYI, all my source code is here.
Below is the core agent code. The agent definition (at the bottom) has instructions, a model, and some tools. I’m using the Google Search tool and our Developer Knowledge API MCP server tools. The former grounds results in Google Search results, and the latter points to Google’s core documentation.
func createTutorialAgent(ctx context.Context, llmModel model.LLM) (agent.Agent, error) {
transport, err := mcpTransport(ctx)
if err != nil {
return nil, err
}
mcpToolSet, _ := mcptoolset.New(mcptoolset.Config{Transport: transport})
publishTool, _ := functiontool.New(functiontool.Config{
Name: "publish_tutorial",
Description: "Publish the tutorial. Requires user confirmation.",
RequireConfirmation: true,
}, publishTutorial)
searchAgent, _ := llmagent.New(llmagent.Config{
Name: "search_agent",
Model: llmModel,
Tools: []tool.Tool{geminitool.GoogleSearch{}},
Instruction: "Search the web.",
})
return llmagent.New(llmagent.Config{
Name: "tutorial_agent",
Model: llmModel,
Description: "Simple tutorial agent.",
Instruction: `You are a technical writer. Draft a Google Cloud tutorial based on user requests.
Once the draft is complete, show it to the user and ask them if they would like to start the publishing process. The 'publish_tutorial' tool will then handle the review and approval.`,
Tools: []tool.Tool{agenttool.New(searchAgent, nil), publishTool},
Toolsets: []tool.Toolset{mcpToolSet},
})
}
Notice that the publish_tutorial tool has a RequireConfirmation attribute. The corresponding publishTutorial function only gets called for a true result.
To start the agent (after setting environment variables for Google Cloud project and any credentials) for the built-in web UI, I used this command:
ADK_AGENT=tutorial go run . web api webui
This provides me a localhost web UI to explore my agent. I asked for a tutorial and saw it use the right tools to generate a response.
I then ask to start the publishing process, and the confirmation flow kicks in. Notice the small free text box asking for my response.
The accepts a JSON payload of {"confirmed": true} only. When I provide that value, the control returns to the agent and it publishes the Markdown tutorial to the local folder.
Let’s do the same workflow with the REST API.
This time, I started up the agent with this command to get the API endpoint only:
ADK_AGENT=tutorial go run . web api
The http://localhost:8080/api/list-apps endpoint shows that I have one “app” named tutorial_agent. To start, I need to create a session with my agent. I’ll set the username and (optionally) the session ID. If I just post to the session endpoint, I get back a random session ID.
curl -X POST http://localhost:8080/api/apps/tutorial_agent/users/u_123/sessions/s_123 \
-H "Content-Type: application/json"
Now I can send in a prompt to my agent. Notice that I’m passing in the user ID and session ID from above.
curl -X POST http://localhost:8080/api/run \
-H "Content-Type: application/json" \
-d '{
"appName": "tutorial_agent",
"userId": "u_123",
"sessionId": "s_123",
"newMessage": {
"role": "user",
"parts": [{
"text": "create a tutorial for deploying a container to an existing GKE Autopilot cluster. use the cli."
}]
}
}'
I get back a pile of JSON that includes the tutorial itself. Like with the web UI experience, I’m asked if I want to kickstart the publishing process. I send in a follow-on message like this:
curl -X POST http://localhost:8080/api/run \
-H "Content-Type: application/json" \
-d '{
"appName": "tutorial_agent",
"userId": "u_123",
"sessionId": "s_123",
"newMessage": {
"role": "user",
"parts": [{
"text": "yes, start the process to publish the tutorial"
}]
}
}'
There’s another pile of JSON to parse through, and I need to find the ID tied to the adk_request_confirmation object.
[ ... "content":{"role":"model","parts":[{"functionCall":{"id":"adk-e6d32a67-c1d4-46b8-87f6-6922a3af3d98","name":"adk_request_confirmation","args":{"originalFunctionCall":{"id":"adk-0e6c0394-a22b-4d5f-9af6-e27284bc175f","args":{"content":"# Deploy a Container to an Existing GKE Autopilot Cluster using the gcloud CLI and kubectl\n\nThis tutorial guides you through" ...]
The next REST API call includes the ID from above and the familiar “confirmed” property.
After this call, I see the Markdown tutorial written to disk.
Advanced Tutorial Agent
This agent itself looks nearly the same as the prior one, but has a more sophisticated toolset. Here’s the agent code, with the confirmation behavior in the downstream function, not the function tool.
func publishTutorialAdvanced(ctx tool.Context, args PublishTutorialArgs) (map[string]any, error) {
confirmation := ctx.ToolConfirmation()
if confirmation == nil {
ctx.RequestConfirmation(
"Reviewer approval required to publish.",
map[string]any{
"status": "approved", // approved, rejected, update
"notes": "",
},
)
return map[string]any{"status": "Pending reviewer approval."}, nil
}
payload := confirmation.Payload.(map[string]any)
status, _ := payload["status"].(string)
notes, _ := payload["notes"].(string)
if strings.ToLower(status) == "approved" {
if !strings.HasSuffix(args.Filename, ".md") {
args.Filename += ".md"
}
if err := os.MkdirAll("tutorials", 0755); err != nil {
return nil, fmt.Errorf("failed to create tutorials directory: %w", err)
}
fullPath := filepath.Join("tutorials", filepath.Base(args.Filename))
if err := os.WriteFile(fullPath, []byte(args.Content), 0644); err != nil {
return nil, fmt.Errorf("failed to save tutorial: %w", err)
}
return map[string]any{"status": "published", "path": fullPath}, nil
}
return map[string]any{"status": status, "notes": notes}, nil
}
func createAdvancedTutorialAgent(ctx context.Context, llmModel model.LLM) (agent.Agent, error) {
transport, err := mcpTransport(ctx)
if err != nil {
return nil, err
}
mcpToolSet, err := mcptoolset.New(mcptoolset.Config{Transport: transport})
if err != nil {
return nil, err
}
publishTool, err := functiontool.New(functiontool.Config{
Name: "publish_tutorial_advanced",
Description: "Publishes the tutorial. Requires status and notes from a reviewer.",
}, publishTutorialAdvanced)
if err != nil {
return nil, err
}
searchAgent, _ := llmagent.New(llmagent.Config{
Name: "search_agent",
Model: llmModel,
Description: "Web search agent.",
Instruction: "Search the web for info.",
Tools: []tool.Tool{geminitool.GoogleSearch{}},
})
return llmagent.New(llmagent.Config{
Name: "advanced_tutorial_agent",
Model: llmModel,
Description: "Tutorial agent with advanced confirmation.",
Instruction: `You are a technical writer. Draft a Google Cloud tutorial based on user requests.
Once the draft is complete, show it to the user and ask them if they would like to start the publishing process.
The 'publish_tutorial_advanced' tool will then handle the multi-stage review and approval.
If the reviewer provides feedback (status 'update' or 'rejected'), revise the draft and try again.`,
Tools: []tool.Tool{agenttool.New(searchAgent, nil), publishTool},
Toolsets: []tool.Toolset{mcpToolSet},
})
}
Notice the RequestConfirmation command in the publishTutorialAdvanced function. Here, we have more control over sending notifications or doing whatever to solicit the confirmation approval.
Note that I tried to get this agent to work with the built-in web UI but couldn’t get it. The same text box pops up for approval confirmation, but no values seem to get the agent to proceed. The only way I got this to work was with a bunch of custom handling in the agent.
So let’s go straight to the REST API. I started the agent using this command:
ADK_AGENT=advanced go run . web api
When I check the list-apps endpoint, I see that my agent app is called advanced_tutorial_agent. So just like above, we start by creating a session with the correct agent name.
curl -X POST http://localhost:8080/api/apps/advanced_tutorial_agent/users/u_123/sessions/s_123 \
-H "Content-Type: application/json"
Now I can prompt the agent with a tutorial request.
curl -X POST http://localhost:8080/api/run \
-H "Content-Type: application/json" \
-d '{
"appName": "advanced_tutorial_agent",
"userId": "u_123",
"sessionId": "s_123",
"newMessage": {
"role": "user",
"parts": [{
"text": "create a cli tutorial for creating a pub/sub topic."
}]
}
}'
My custom tutorial comes back in a second, and I can kick off the publishing process.
After parsing the JSON response, I again find the adk_request_confirmation object and snag the ID.
[ ... "content":{"role":"model","parts":[{"functionCall":{"id":"adk-7744ece0-a82d-4796-96b7-f376d96bf925","name":"adk_request_confirmation","args":{"originalFunctionCall":{"id":"adk-30c04217-3c0b-4519-a866-b914e9357872","args":{"content":"## Creating a Pub/Sub Topic with the `gcloud` CLI\n\nThis tutorial guides you through creating a Google Cloud Pub/Sub topic using the" ...]
In the next REST call, see that I can now send a richer payload back to the agent and react accordingly.
And once again, I get a Markdown file saved to disk.
After this initial demo, I may play with putting an A2UI interface on top of the agent, or even building it again using Genkit Go. But either way, make sure you give yourself some discrete moments where your agent stops and asks for input!
I’ve been thinking more about how it’s premature to build any large custom AI engines inside companies. Use off-the-shelf products to get rolling. Too many things are changing right now, and you can get a lot done by stuffing context into available products. The Stripe item below reinforced that for me.
[article] The Map of System Topologies. Some impressive analysis of common architectures that most tech systems end up falling into.
[blog] AI Slop & the Vulnerability Treadmill. Lengthy and important piece by Kate that’s a must-read for security teams AND executives. It’s time to rethink things.
[article] Local AI. I’ll admit that open models didn’t get me that fired up a year ago. Why run one yourself when you can use a SOTA one as a service? But token usage has skyrocketed, sovereign needs are more clear now, and open models have continued to innovate. So, I get it now.
[article] Google Is A Full Stack AI Player, And Is Playing Well. A lot of long bets have paid off, or are showing signs of paying off. It’s cheaper and faster to get wins through partnerships (e.g. Microsoft), but you’re left exposed without owning more of your supply chain.
[blog] State of Routing in Model Serving. Netflix has a legit ML serving platform and needed to evolve their routing approach. Cool deep dive here.
[blog] Reading List #1. This “reading list” thing is catching on! Even if no one else read my notes every day, I’d still get a lot of value from the discipline of reading and writing it.
[article] AI finds 20-year-old bugs in PostgreSQL and MariaDB. I don’t think we should have AI “slow down” because we can’t handle all the code issues it finds. We need better ways to quickly triage issues and incorporate fixes.
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
I’m back at work today and was planning on making a quick trip up to Mountain View for a work meeting. But since I’m solo dad-ing this week and the kid just caught a cold, I’m staying home with him instead. On the plus side, my day is WIDE open tomorrow now!
[blog] Run multiple coding agents safely with git worktrees. Work on a few branches simultaneously. This matters even more now when one person might be coordinating a handful of agents working on the same codebase.
Happy May! I’m expecting another exciting month of tech updates and real-world stories of people learning the best ways to do modern work. Buckle up.
[blog] Databases Were Not Designed For This. Predictable queries and deterministic code? That’s not what databases encounter today. Are defensively designing your data layer for agents? This piece has specific advice.
[article] The Psychological Costs of Adopting AI. Let’s ensure we intentionally build the human infrastructure needed to get the most value from these AI tools.
Back on vacation and had a day-date with my wife. A bunch of fresh crazy kicks in at work next week, so I’m thrilled to get a breather before we run at full speed again.
[blog] Long-running Agents. Another killer post from Addy. What changes when you move from single-turn stateless agents to agents that need memories and coordination over time? This post has the patterns and solution options.
[article] AI productivity gains: More modest than expected. So far. But as the agentic operating model takes hold, team shapes change, and better platforms stretch from build-to-prod, you’ll see these gains skyrocket.
We had quite the earnings call today, and I’m proud of the work Google Cloud has done to deliver a platform people want to use. I had a mini-break from vacation this week to do customer calls, but taking Thursday and Friday to finish out my recharge week. Reading lists still part of the plan!
[blog] Meetings are forcing functions. We all complain about meetings, but there’s a reason they exist. In some cases, the standing meeting actually forces people to do things ahead of time.
[article] Generative UI explained without the hype. I really needed this. For some reason, I just didn’t “get” generative UIs. But now I understand it much better.
[article] Managing a team that didn’t choose you. You rarely get to build an entire team or situation from scratch after taking over a leadership assignment. This post looks at how an engineering manager might handle the new circumstances.
[blog] Before GitHub. Armin looks at how we managed source code and packages in a pre-GitHub world. While many of us are rooting for GitHub to get back on track, it’s useful to play through scenarios where that’s not the case.
[blog] Building a production MCP server in Go. This post starts off by saying it was useful to build the MCP server in the same language as their existing backend. Many good tips here.
[blog] You can now easily generate files in Gemini. Here we go. Create PDFs, Word docs, Google Sheets, and other artifacts directly in your Gemini app session. Available to everyone, right now.
Day two of vacation included me hitting golf balls and eating lunch overlooking the ocean. And a couple of unexpected work calls. But overall, definitely feeling better and more energized already.
[blog] AI in DevOps: Why Adoption Lags in CI/CD (and What Comes Next). It’s an interesting point of view. I’m not sure the conclusion is right. Are devs slow to add AI to CI/CD because of trust issues, or because the current landscape of products isn’t suited for AI workloads or surfacing the right AI integration points?
[article] Vibing, Harness and OODA loop. You had me at OODA loop, as I’m a big Boyd fan. Just because AI helps us “act” faster, doesn’t mean we should skip the other important parts of the loop.
[blog] Agent Memory Patterns. This post calls out a few places to store mutable memory. It focuses on files, memory blocks, and skills. I’m not sure how you’d categorize external durable memory services.
Richard Seroter's Architecture Musings 