I had a quiet day in London, meeting with friends and getting work done. The event I’m here for starts tomorrow morning. In the meantime, i got to build some things today, and read a fair bit. Check out the list below.
[article] From Potential to Profit with GenAI. Lots of interesting data here. The investment in generative AI is big, but so are expectations. Many are looking for cost savings in 2024, which may be unrealistic.
[article] HOWTO: Change your behavior. Are there things you’d like to start doing or stop doing? Probably, if you’re a human being. Read this for some perspective on what it takes to change.
[blog] Sign in with Google in Go. If you built an app that lets users sign in with existing identities, your users probably thank you. This post shows two options for incorporating Google identities.
[blog] Coming Soon: Golang 1.22. It feels to me that language changes are coming faster (it’s probably just in my head), so I like these types of posts that bubble up the changes that matter most.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Greetings from London! I arrived yesterday and spent today at our office working with our local sales engineering team. Great bunch of proper experts.
[article] Many software developers don’t trust AI – report. That’s the headline, but inside, you also see that 70% of devs think AI will help, and 60% want to use AI. But, there’s work to do for us all to trust it more.
[article] Applying the SPACE Framework. It’s less prescriptive that DORA, but the SPACE framework offers a more rich view of the “developer productivity” lens for tech teams. I suspect that many folks will use both to uplevel their teams.
[blog] 5 Steps to Debug Development and Operations Teams. If you inherit a struggle team, or wake and realize your own team is struggling, this article offers helpful guidance for figuring out where the issue is.
[article] How to Lead Across a Siloed Organization. It’s probably rare nowadays for someone to NOT work with a lot of people across different functions or organizations. If that’s hard for you—as it is for most of us—this guidance should help.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Did you have an interesting week? I hope you learned a few things and met someone new. I’m off to London tomorrow for a week of work stuff, but will continue publishing this reading list while I’m there. It’s just possible I mix in some “bloody hell” mentions while complaining about lorries and loos.
[blog] Cloud Deploy gains support for custom target types. I really like the ability to use a single continuous deployment engine to along with a variety of “targets” like an MLOps platform or GitOps-style rollout.
[blog] Deploying Go API on GKE | Google Cloud. I like this end to end walkthrough that covers account setup, Kubernetes cluster setup, and database configuration.
[article] You probably have too many SaaS apps. Seems accurate. There are lots of specialized products which are great, but there must be room to consolidate to fewer, good-enough products.
Another day of drowning in meetings, but at least I’m drowning gracefully. I enjoyed the handful of pieces I got a chance to read, and bet you’ll like one or two of them too.
[blog] Analyzing Go Binary Sizes. It’s easy to ignore the ever-growing size of our compiled components, but you’re doing yourself a favor by understanding why a binary is bloated, and how to shrink it.
[article] How Golang Manages Its Security Supply Chain. I genuinely hope the things the Go team does become standard with the other major languages. It’d go a long way to improving the security posture of countless systems.
[blog] Laying the foundation for a career in platform engineering. We see lots of companies embrace the concept of platform engineering, but what does it take to build a career as a platform engineer? I like this post and what it covers.
This is supposed to be a “light meeting” week at work. In unrelated news, I had twelve meetings today. Fortunately, there wasn’t a dud in a bunch and I could still make time to read some interesting items today.
[article] When Your Boss Gives You a Totally Unrealistic Goal. Hey, I did this to my team yesterday! But on purpose. Outlandish goals force unconstrained thinking. We do the best we can, and no one (on my team) will get dinged because we didn’t reach it. Aim high.
[docs] Perform automatic side-by-side evaluation. Wow, evaluate the performance of two LLMs, side by side to see which gives better responses? This is now part of Vertex AI.
[blog] Getting started with AI notebooks and Jupyterhub on GKE. Is your team doing more with Jupyter notebooks? If so, you likely have some platform for folks to interact with them. Here’s a way to run such an experience on Kubernetes.
[blog] Gemini, the stylist… Would you trust an LLM to offer to hairstyle suggestions? Um, maybe? Abi built a fun app that used Gemini and Imagen2 to create suggestions that she accepted.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
I cover a fair amount of “developer productivity” and “developer happiness” content in this reading list. Admittedly, I’m drawn to it. It’s not about pampered developers who do what they want and everyone caters to them. Rather, it’s about unleashing a valuable asset and holding them accountable for excellence by giving them what they need to succeed.
[article] What Is Active Listening? Being a good listener is a choice. One that takes a lot of effort, but has a material impact on those around you. I liked this article about active listening.
[blog] Software Quality. In measuring developer productivity, how does Google look at “quality” metrics? This post explores a new paper from us that digs into four dimensions of quality.
[article] 2023: The Year Open Source Security Supply Chain Grew Up. From the surveys and data I’ve seen, we have a long way to go before we’re all good at securing our software supply chains. But there’s progress, and the tech itself is getting better.
[blog] 3 predictions for AI in healthcare in 2024. I’ve gotten fairly cynical on vendor-driven predictions (even from my own employer), but these are pretty reasonable ones.
[article] It’s time to fix flaky tests in software development. Tests that sometimes work, and sometimes don’t, even when the code doesn’t change. Not fun. Trisha says you can’t ignore these flaky tests, and need to focus on resolving them.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Happy Monday! This looks like it’ll be my first 5-day workweek in a while. You too? I cleared out some of my reading queue today, and saw lots of great material. Jump in.
[blog] Product Managing to Prevent Burnout. To me, a great product manager is one who protects their team. They keep engineers focused on priorities and out of useless meetings, and encourages sustainable pace. This article does a great job explaining this.
[article] A Guide for Getting Stakeholder Buy-In for Your Agenda. Most of us eventually learn that a great idea won’t matter if you can’t get people to buy into it. This article has a good framework to think through if you want to advance our ideas within your company.
[blog] Our move to generated SDKs. GitHub is moving towards software-generated SDKs with the goal of more structured and regularly-updated bits. .NET and Go are the first two languages supported.
[blog] Advice on using LLMs wisely. My former colleague is a great follow on LinkedIn and has shared some smart perspectives on generative AI over the past few months.
[blog] Free technical writing courses. Are you an engineer who wants to be a better technical writer? These upcoming courses from Google are free to take.
[blog] Go 1.22: Interactive release notes. Not only do you learn about new features in this post, but you can try them inline without leaving the page. Very cool.
[blog] Engineering Transformations – Adopting Automated Testing As A Practice. Generative AI can help you slap more tests into your codebase, but that doesn’t magically give you a good testing culture. This post looks at why automated tests matter, failure modes, and strategies for success.
If you’re still on holiday break, drink in your remaining hours. For those of us who returned to work this week, how was it? I’m glad to be back, and am excited about the year ahead.
[article] The Hard Questions to Ask When Planning Your Strategy. This article says that strategy can be defined as answering “where are we now”, “where do we want to go”, and “what is a credible path to get there.” But many aren’t honest with themselves about the first question.
[blog] How to Choose Developer Events. Which events should you sponsor this year? Relatedly, I wonder how many of you are choosing which events to attend as well.
[blog] Angular Developer Survey 2023. User surveys are a powerful way to get ideas and feedback about your product. The Angular team uses these to help prioritize their roadmap.
[blog] Bazel 7 Release. Each language/framework has their own build system(s), but there’s something powerful about having a single build engine regardless of language, OS, or platform. This latest LTS release of Bazel offers some terrific updates.
[blog] Management. Have you hugged your manager today? I mean, don’t get fired for being creepy, but have you stopped to appreciate the role and their efforts? This post looks at the noble profession and hard work it takes to be good at it.
I had a good day getting some items completed, while also learning and writing a bit. Many of the pieces below are useful for those thinking through 2024 strategies.
[article] Observability in 2024: More OpenTelemetry, Less Confusion. “Observability” is a noisy space, and every monitoring vendor now calls themselves an observability vendor. But that doesn’t mean that there’s not a lot of important work in this area.
[blog] Choosing between Cloud Run and GKE. Here’s a short post for those deciding between the two primary container-based runtimes in Google Cloud. For me, if you have a lot of code/components and want the Kubernetes API, choose GKE Autopilot. If you have glue apps or standalone apps, pick Cloud Run.
[article] Legacy Seam. This post from Martin Fowler looks at finding seams in your legacy code base and changing behavior without editing source code.
[blog] Coding at Google. Eric was at Microsoft, then Google, now back at Microsoft, but he offers up a terrific look at the platforms and processes we give to our devs so they can do their best work.
How many chatbots do we really need? While chatbots are a terrific example app for generative AI use cases, I’ve been thinking about how developers may roll generative AI into existing “boring” apps and make them better.
As I finished all my Christmas shopping—much of it online—I thought about all the digital storefronts and how they provide recommended items based on my buying patterns, but serve up the same static item descriptions, regardless of who I am. We see the same situation with real estate listings, online restaurant menus, travel packages, or most any catalog of items! What if generative AI could create a personalized story for each item instead? Wouldn’t that create such a different shopping experience?
Maybe this is actually a terrible idea, but during the Christmas break, I wanted to code an app from scratch using nothing but Google Cloud’s Duet AI while trying out our terrific Gemini LLM, and this seemed like a fun use case.
The final app (and codebase)
The app shows three types of catalogs and offers two different personas with different interests. Everything here is written in Go and uses local files for “databases” so that it’s completely self-contained. And all the images are AI-generated from Google’s Imagen2 model.
When the user clicks on a particular catalog entry, the go to a “details” page where the generic product summary from the overview page is sent along with a description of the user’s preferences to the Google Gemini model to get a personalized, AI-powered product summary.
That’s all there is to it, but I think it demonstrates the idea.
How it works
Let’s look at what we’ve got here. Here’s the basic flow of the AI-augmented catalog request.
How did I build the app itself (GitHub repo here)? My goal was to only use LLM-based guidance either within the IDE using Duet AI in Google Cloud, or burst out to Bard where needed. No internet searches, no docs allowed.
I started at the very beginning with a basic prompt.
What are the CLI commands to create a new Go project locally?
The answer offered the correct steps for getting the project rolling.
The next commands are where AI assistance made a huge difference for me. With this series of natural language prompts in the Duet AI chat within VS Code, I got the foundation of this app set up in about five minutes. This would have easily taken me 5 or 10x longer if I did it manually.
Give me a main.go file that responds to a GET request by reading records from a local JSON file called property.json and passes the results to an existing html/template named home.html. The record should be defined in a struct with fields for ID, Name, Description, and ImageUrl.
Create an html/template for my Go app that uses Bootstrap for styling, and loops through records. For each loop, create a box with a thin border, an image at the top, and text below that. The first piece of text is "title" and is a header. Below that is a short description of the item. Ensure that there's room for four boxes in a single row.
Give me an example data.json that works with this struct
Add a second function to the class that responds to HTML requests for details for a given record. Accept a record id in the querystring and retrieve just that record from the array before sending to a different html/template
With these few prompts, I had 75% of my app completed. Wild! I took this baseline, and extended it. The final result has folders for data, personas, images, a couple HTML files, and a single main.go file.
Let’s look at the main.go file, and I’ll highlight a handful of noteworthy bits.
package main
import (
"context"
"encoding/json"
"fmt"
"html/template"
"log"
"net/http"
"os"
"strconv"
"github.com/google/generative-ai-go/genai"
"google.golang.org/api/option"
)
// Define a struct to hold the data from your JSON file
type Record struct {
ID int
Name string
Description string
ImageURL string
}
type UserPref struct {
Name string
Preferences string
}
func main() {
// Parse the HTML templates
tmpl := template.Must(template.ParseFiles("home.html", "details.html"))
//return the home page
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
var recordType string
var recordDataFile string
var personId string
//if a post-back from a change in record type or persona
if r.Method == "POST" {
// Handle POST request:
err := r.ParseForm()
if err != nil {
http.Error(w, "Error parsing form data", http.StatusInternalServerError)
return
}
// Extract values from POST data
recordType = r.FormValue("recordtype")
recordDataFile = "data/" + recordType + ".json"
personId = r.FormValue("person")
} else {
// Handle GET request (or other methods):
// Load default values
recordType = "property"
recordDataFile = "data/property.json"
personId = "person1" // Or any other default person
}
// Parse the JSON file
data, err := os.ReadFile(recordDataFile)
if err != nil {
fmt.Println("Error reading JSON file:", err)
return
}
var records []Record
err = json.Unmarshal(data, &records)
if err != nil {
fmt.Println("Error unmarshaling JSON:", err)
return
}
// Execute the template and send the results to the browser
err = tmpl.ExecuteTemplate(w, "home.html", struct {
RecordType string
Records []Record
Person string
}{
RecordType: recordType,
Records: records,
Person: personId,
})
if err != nil {
fmt.Println("Error executing template:", err)
}
})
//returns the details page using AI assistance
http.HandleFunc("/details", func(w http.ResponseWriter, r *http.Request) {
id, err := strconv.Atoi(r.URL.Query().Get("id"))
if err != nil {
fmt.Println("Error parsing ID:", err)
// Handle the error appropriately (e.g., redirect to error page)
return
}
// Extract values from querystring data
recordType := r.URL.Query().Get("recordtype")
recordDataFile := "data/" + recordType + ".json"
//declare recordtype map and extract selected entry
typeMap := make(map[string]string)
typeMap["property"] = "Create an improved home listing description that's seven sentences long and oriented towards a a person with these preferences:"
typeMap["store"] = "Create an updated paragraph-long summary of this store item that's colored by these preferences:"
typeMap["restaurant"] = "Create a two sentence summary for this menu item that factors in one or two of these preferences:"
//get the preamble for the chosen record type
aiPremble := typeMap[recordType]
// Parse the JSON file
data, err := os.ReadFile(recordDataFile)
if err != nil {
fmt.Println("Error reading JSON file:", err)
return
}
var records []Record
err = json.Unmarshal(data, &records)
if err != nil {
fmt.Println("Error unmarshaling JSON:", err)
return
}
// Find the record with the matching ID
var record Record
for _, rec := range records {
if rec.ID == id { // Assuming your struct has an "ID" field
record = rec
break
}
}
if record.ID == 0 { // Record not found
// Handle the error appropriately (e.g., redirect to error page)
return
}
//get a reference to the persona
person := "personas/" + (r.URL.Query().Get("person") + ".json")
//retrieve preference data from file name matching person variable value
preferenceData, err := os.ReadFile(person)
if err != nil {
fmt.Println("Error reading JSON file:", err)
return
}
//unmarshal the preferenceData response into an UserPref struct
var userpref UserPref
err = json.Unmarshal(preferenceData, &userpref)
if err != nil {
fmt.Println("Error unmarshaling JSON:", err)
return
}
//improve the message using Gemini
ctx := context.Background()
// Access your API key as an environment variable (see "Set up your API key" above)
client, err := genai.NewClient(ctx, option.WithAPIKey(os.Getenv("GEMINI_API_KEY")))
if err != nil {
log.Fatal(err)
}
defer client.Close()
// For text-only input, use the gemini-pro model
model := client.GenerativeModel("gemini-pro")
resp, err := model.GenerateContent(ctx, genai.Text(aiPremble+" "+userpref.Preferences+". "+record.Description))
if err != nil {
log.Fatal(err)
}
//parse the response from Gemini
bs, _ := json.Marshal(resp.Candidates[0].Content.Parts[0])
record.Description = string(bs)
//execute the template, and pass in the record
err = tmpl.ExecuteTemplate(w, "details.html", record)
if err != nil {
fmt.Println("Error executing template:", err)
}
})
fmt.Println("Server listening on port 8080")
fs := http.FileServer(http.Dir("./images"))
http.Handle("/images/", http.StripPrefix("/images/", fs))
http.ListenAndServe(":8080", nil)
}
I do not write great Go code, but it compiles, which is good enough for me!
On line 13, see that I refer to the Go package for interacting with the Gemini model. All you need is an API key, and we have a generous free tier.
On line 53, notice that I’m loading the data file based on the type of record picked on the HTML template.
On line 79, I’m executing the HTML template and sending the type of record (e.g. property, restaurant, store), the records themselves, and the persona.
On lines 108-113, I’m storing a map of prompt values to use for each type of record. These aren’t terrific, and could be written better to get smarter results, but it’ll do.
Notice on line 147 that I’m grabbing the user preferences we use for customization.
On line 163, I create a Gemini client so that I can interact with the LLM.
On line 171, see that I’m generating AI content based on the record-specific preamble, the record details, and the user preference data.
On line 177, notice that I’m extracting the payload from Gemini’s response.
Finally, on line 181 I’m executing the “details” template and passing in the AI-augmented record.
What I have here is a local example app. How would I make this more production grade?
Store catalog images in an object storage service. All my product images shouldn’t be local, of course. They belong in something like Google Cloud Storage.
Add catalog items and user preferences to a database. Likewise, JSON files aren’t a great database. The various items should all be in a relational database.
Write better prompts for the LLM. My prompts into Gemini are meh. You can run this yourself and see that I get some silly responses, like personalizing the message for a pillow by mentioning sporting events. In reality, I’d write smarter prompts that ensured the responding personalized item summary was entirely relevant.
Use Vertex AI APIs for accessing Gemini. Google AI Studio is terrific. For production scenarios, I’d use the Gemini models hosted in full-fledged MLOps platform like Vertex AI.
Run app in a proper cloud service. If I were really building this app, I’d host it in something like Google Cloud Run, or maybe GKE if it was part of a more complex set of components.
Explore whether pre-generating AI-augmented results and caching them would be more performant. It’s probably not realistic to call LLM endpoints on each “details” page. Maybe I’d pre-warm certain responses, or come up with other ways to not do everything on the fly.
This exercise helped me see the value of AI-assisted developer tooling firsthand. And, it feels like there’s something useful about LLM summarization being applied to a variety of “boring” app scenarios. What do you think?
Richard Seroter's Architecture Musings 