It’s been a good week, even though I’ve been under the weather. I’ve got a chance to rest up over this weekend and most of next week. I hope you also take a chance to recharge!
[article] In The Context Of Long Context. Here’s a wonderful post from Steven (who’s part of the NotebookLM effort) about the magic and importance of long context for LLMs.
[article] Auditors blast Pentagon over insecure, “antiquated” IT systems. Even when high-profile audits result in embarrassing findings, it’s hard to modernize the offending legacy systems. I can’t imagine how hard it is when the findings aren’t on the front page.
[blog] What’s new in Spring Modulith 1.3? I’ve been keeping an eye on this “modular monolith” framework from the Spring team. It’s making good progress.
[blog] Building a User Signals Platform at Airbnb. I always enjoy these types of posts where companies go deep into their architectures. This one looks at a stream processing system.
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
[article] Google’s Gemini chatbot now has memory. It remembers. I mean, not in a creepy way. But if you want our Gemini agent to use past info to make future suggestions, you got it.
Had another good day in Sunnyvale today and I’m enjoying these chances to plan what, and how, we work in 2025. The industry keeps moving, even if I’m in all-day meetings, and there’s some good updates below.
[blog] How we built Google Meet’s adaptive audio feature. I assumed this was just a terrible fact of life. But no, it doesn’t HAVE to sound like death when people join a conference call from the same room.
[blog] Why Spring AI: The Seamless Path to Generative AI. Don’t sleep on Spring AI as a framework that stands to get traction. There’s a lot of Spring Boot out there, and this looks like a solid option for Java devs.
[blog] Playground Wisdom: Threads Beat Async/Await. Big, interesting post about blocking code, async processing, and why threads are better than the async/await abstraction.
[blog] Meet Angular v19. Big, big release of this popular frontend framework. Dig through this to see what matters, and what’s new/improved.
[blog] Batch prediction in Gemini. You want predictions for dozens, hundreds, or thousands of items in your store, portfolio, or account? Mete has a clear post that shows how to submit batches of requests to an LLM.
[article] p-Hacking your A/B tests. You might be making a mistake when running A/B tests for your app, marketing campaigns, or whatever. Jason says you shouldn’t stop a test once it looks like you got a “conclusive” result.
[blog] Scaling Ambient In Your Sleep. Quick post, but good to read if you skipped Istio the first time around because of too much complexity or too many scaling concerns.
I’m in Sunnyvale for the week. Today was a product leadership offsite, and I had fun talking to our senior product leaders. Enjoy the reading list below.
[blog] Platform Engineering as a Service. Does platform engineering bring order to the chaos that DevOps creates? Tyler thinks so, and offers up a solution.
[report] CNCF Technology Landscape Radar. If you’re a platform builder who is looking for open tech to bake into your stack, look at this new CNCF report that explores sentiment about some leading projects.
[blog] Meta Prompt Guard. Nervous about people tricking your AI agents into doing bad things? This post explores a lightweight classifier model from Meta that looks for malicious input.
[article] Let’s End Toxic Productivity. We have more conveniences than our grandparents did, but also seem to work more hours. Our standard of living is higher, as are our expectations, but still. This deep dive looks at how to step back and not overdo everything.
[blog] Gemini in Firebase for Data Connect queries. Here’s a good use case for LLMs. Firebase is using natural language prompts to generate GraphQL instead of you needing to know the full syntax.
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
The ability to use your own codebase to customize the suggestions from an AI coding assist is a big deal. This feature—available in products like Gemini Code Assist, GitHub Copilot, and Tabnine—gives developers coding standards, data objects, error messages, and method signatures that they recognize from previous projects. Data shows that the acceptance rate for AI coding assistants goes way up when devs get back trusted results that look familiar. But I don’t just want up-to-date and familiar code that *I* wrote. How can I make sure my AI coding assistant gives me the freshest and best code possible? I used code customization in Gemini Code Assist to reference Google Cloud’s official code sample repos and now I get AI suggestions that feature the latest Cloud service updates and best practices for my preferred programming languages. Let me show you how I did it.
Last month, I showed how to use local codebase awareness in Gemini Code Assist (along with its 128,000 input token window) to “train” the model on the fly using code samples or docs that an LLM hasn’t been trained on yet. It’s a cool pattern, but also requires upfront understanding of what problem you want to solve, and work to stash examples into your code repo. Can I skip both steps?
Yes, Gemini Code Assist Enterprise is now available and I can point to existing code repos in GitHub or GitLab. When I reference a code repo, Google Cloud automatically crawls it, chunks it up, and stores it (encrypted) in a vector database within a dedicated project in my Google Cloud environment. Then, the Gemini Code Assist plugin uses that data as part of a RAG pattern when I ask for coding suggestions. By pointing at Google Cloud’s code sample repos—any best practice repo would apply here—I supercharge my recommendations with data the base LLM doesn’t have (or prioritize).
Step #0 – Prerequisites and considerations
Code customization is an “enterprise” feature of Gemini Code Assist, so it requires a subscription to that tier of service. There’s a promotional $19-per-month price until March of 2025, so tell your boss to get moving.
Also, this is currently available in US, European, and Asian regions, you may need to request geature access via a form (depending on when you read this), and today it works with GitHub.com and GitLab.com repos, although on-premises indexing is forthcoming. Good? Good. Let’s keep going.
Step #1 – Create the source repo
One wrinkle here is that you need to own the repos you ask Gemini Code Assist to index. You can’t just point at any random repo to index. Deal breaker? Nope.
I can just fork an existing repo into my own account! For example, here’s the Go samples repo from Google Cloud, and the Java one. Each one is stuffed with hundreds of coding examples for interacting with most of Google Cloud’s services. These repos are updated multiple times per week to ensure they include support for all the latest Cloud service features.
I went ahead and forked each repo in GitHub. You can do it via the CLI or in the web console.
I didn’t overthink it and kept the repository name the same.
Gemini Code Assist can index up to 950 repos (and more if really needed), so you could liberally refer to best-practice repos that will help your developers write better code.
Any time I want to refresh my fork to grab the latest code sample updates, I can do so.
Step #2 – Add a reference to the source repo
Now I needed to reference these repos for later code customization. Google Cloud Developer Connect is a service that maintains connections to source code sitting outside Google Cloud.
I started by choosing GitHub.com as my source code environment.
Then I named my Developer Connect connection.
Then I installed a GitHub app into my GitHub account. This app is what enables the loading of source data into the customization service. From here, I chose the specific repos that I wanted available to Developer Connect.
When finished, I had one of my own repos, and two best practice repos all added to Developer Connect.
That’s it! Now to point these linked repos to Gemini Code Assist.
Step #3 – Add a Gemini Code Assist customization index
I had just two CLI commands to execute.
First, I created a code customization index. You’ve got one index per Cloud project (although you can request more) and you create it with one command.
Next, I created a repository group for the index. You use these to control access to repos, and could have different ones for different dev audiences. Here’s where you actually point to a given repo that has the Developer Connect app installed.
I ran this command a few times to ensure that each of my three repos was added to the repository group (and index).
Indexing can take up to 24 hours, so here’s where you wait. After a day, I saw that all my target repos were successfully indexed.
Whenever I sync the fork with the latest updates to code samples, Gemini Code Assist will index the updated code automatically. And my IDE with Gemini Code Assist will have the freshest suggestions from our samples repo!
Step #4 – Use updated coding suggestions
Let’s prove that this worked.
I looked for a recent commit to the Go samples repos that the base Gemini Code Assist LLM wouldn’t know about yet. Here’s one that has new topic-creation parameters for our Managed Kafka service. I gave the prompt below to Gemini Code Assist. First, I used a project and account that was NOT tied to the code customization index.
//function to create a topic in Google Cloud Managed Kafka and include parameters for setting replicationfactor and partitioncount
The coding suggestion was good, but incomplete as it was missing the extra configs the service can now accept.
When I went to my Code Assist environment that did have code customization turned on, you see that the same prompt gave me a result that mirrored the latest Go sample code.
I tried a handful of Java and Go prompts, and I regularly (admittedly, not always) got back exactly what I wanted. Good prompt engineering might have helped me reach 100%, but I still appreciated the big increase in quality results. It was amazing to have hundreds of up-to-date Google-tested code samples to enrich my AI-provided suggestions!
AI coding assistants that offer code customization from your own repos are a difference maker. But don’t stop at your own code. Index other great code repos that represent the coding standards and fresh content your developers need!
Happy Friday. I’ve got lots of links again for you today! Enjoy your weekend and see y’all next week.
[blog] The 5 Cs: Configuring access to backing services. How do you set up configurations between app code and its database? Brian looks at what is needed, and wonders if there’s a better way.
[blog] Inference with Gemma using Dataflow and vLLM. I learned a least a half dozen things from reading this post. Cool look at what it takes to use an LLM in a streaming pipeline.
[article] “Reducing Complexity”. John makes great points here about how we use “complexity” as a shorthand for a lot of different problems.
[blog] Announcing .NET 9. I’m admittedly not doing much with C# right now, but these language updates are still a big milestone. I’m sure devs will pick up this version quickly.
[article] Why designing landing pages is hard. It is hard. Know who you’re targeting, and just accept that for pages with a wide audience, you can’t please everyone.
[blog] Generative epiphany. Analogies don’t have to be perfect to be helpful. I like how Katie used ideas from the containerization world to grok LLMs.
[blog] Spring Boot and Temporal. Sometimes we feel like pioneers as we navigate the mashup of technologies. Cornelia goes through an exploration to get this workflow engine to play with a Java Spring Boot app.
It’s probably just me, or confirmation bias, but I feel like I’ve seen an uptick in deep content lately. Each day I’m finding blog posts and articles that explore topics in fresh ways. Lovin’ it.
[blog] Netflix’s Distributed Counter Abstraction. Do you want a couple thousand words on building an accurate counting service? OF COURSE YOU DO. This is an excellent post from the Netflix team, and its level of detail is an example of why I love tech blogs.
[blog] Becoming an AI-Assisted Coding Convert. We each have different relationships with our dev tools and our preferred workflows. Aja found peace with using AI assistance in a way that worked for her.
[article] Will OpenTofu Dethrone Terraform in IaC? Not any time soon. But there’s shakiness in some foundational areas (IaC, IDEs, databases like Redis) that we haven’t seen in a long time.
[article] Build GenAI Prototypes with Streamlit. if you’re getting into building AI apps atop LLMs, you could do worse than starting with Streamlit to build prototypes.
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
[blog] Deployment-Driven Development. Tyler says that we should change our thinking. Instead of treating deployments as a destination, it should be prioritized from the start.
[article] Employee AI adoption cools globally. Temporarily. This survey from Slack shows that folks don’t want to be perceived by management as lazy for using AI. We can fix that with better education of the management ranks, and better sponsorship of AI initiatives from exec teams.
[article] Memory in Agent Systems. This is a good topic to learn about. With all this talk about agents, how do we think about durable storage of “memory”?
There was so much thought-provoking content today! I enjoyed the pieces on refining strategy, giving feedback, running local LLMs, shipping products, and more.
[blog] Using systems modeling to refine strategy. how do you work through an engineering strategy? Will proposes using the ideas behind systems modeling and explains how to do it.
[article] High Performers Need Feedback, Too. Can we give more constructive feedback to our best teammates? Yes. This article has some examples of how to improve the quality of your feedback.
[blog] LLM Guard and Vertex AI. Mete has been looking at evaluation frameworks and this one is a security-related one. Make sure that you’ve got frameworks like this in mind as you start legitimately using LLMs.
[blog] Generating zero-shot personalized portraits. Need a new headshot? This post from Google Research shows off a new AI model that can transform selfies into different artistic styles.
[blog] Adopting Bazel for Web at Scale. Lots of details here from Airbnb about how they moved from bespoke scripts for their builds to a single tool.
[blog] How I ship projects at big tech companies. Somewhat cynical, but entirely reasonable perspective about what it takes to get a product out the door, and have it considered “shipped.”
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Richard Seroter's Architecture Musings 