Category: AI/ML

The ability to use your own codebase to customize the suggestions from an AI coding assist is a big deal. This feature—available in products like Gemini Code Assist, GitHub Copilot, and Tabnine—gives developers coding standards, data objects, error messages, and method signatures that they recognize from previous projects. Data shows that the acceptance rate for AI coding assistants goes way up when devs get back trusted results that look familiar. But I don’t just want up-to-date and familiar code that *I* wrote. How can I make sure my AI coding assistant gives me the freshest and best code possible? I used code customization in Gemini Code Assist to reference Google Cloud’s official code sample repos and now I get AI suggestions that feature the latest Cloud service updates and best practices for my preferred programming languages. Let me show you how I did it.

Last month, I showed how to use local codebase awareness in Gemini Code Assist (along with its 128,000 input token window) to “train” the model on the fly using code samples or docs that an LLM hasn’t been trained on yet. It’s a cool pattern, but also requires upfront understanding of what problem you want to solve, and work to stash examples into your code repo. Can I skip both steps?

Yes, Gemini Code Assist Enterprise is now available and I can point to existing code repos in GitHub or GitLab. When I reference a code repo, Google Cloud automatically crawls it, chunks it up, and stores it (encrypted) in a vector database within a dedicated project in my Google Cloud environment. Then, the Gemini Code Assist plugin uses that data as part of a RAG pattern when I ask for coding suggestions. By pointing at Google Cloud’s code sample repos—any best practice repo would apply here—I supercharge my recommendations with data the base LLM doesn’t have (or prioritize).

Step #0 – Prerequisites and considerations

Code customization is an “enterprise” feature of Gemini Code Assist, so it requires a subscription to that tier of service. There’s a promotional $19-per-month price until March of 2025, so tell your boss to get moving.

Also, this is currently available in US, European, and Asian regions, you may need to request geature access via a form (depending on when you read this), and today it works with GitHub.com and GitLab.com repos, although on-premises indexing is forthcoming. Good? Good. Let’s keep going.

Step #1 – Create the source repo

One wrinkle here is that you need to own the repos you ask Gemini Code Assist to index. You can’t just point at any random repo to index. Deal breaker? Nope.

I can just fork an existing repo into my own account! For example, here’s the Go samples repo from Google Cloud, and the Java one. Each one is stuffed with hundreds of coding examples for interacting with most of Google Cloud’s services. These repos are updated multiple times per week to ensure they include support for all the latest Cloud service features.

I went ahead and forked each repo in GitHub. You can do it via the CLI or in the web console.

I didn’t overthink it and kept the repository name the same.

Gemini Code Assist can index up to 950 repos (and more if really needed), so you could liberally refer to best-practice repos that will help your developers write better code.

Any time I want to refresh my fork to grab the latest code sample updates, I can do so.

Step #2 – Add a reference to the source repo

Now I needed to reference these repos for later code customization. Google Cloud Developer Connect is a service that maintains connections to source code sitting outside Google Cloud.

I started by choosing GitHub.com as my source code environment.

Then I named my Developer Connect connection.

Then I installed a GitHub app into my GitHub account. This app is what enables the loading of source data into the customization service. From here, I chose the specific repos that I wanted available to Developer Connect.

When finished, I had one of my own repos, and two best practice repos all added to Developer Connect.

That’s it! Now to point these linked repos to Gemini Code Assist.

Step #3 – Add a Gemini Code Assist customization index

I had just two CLI commands to execute.

First, I created a code customization index. You’ve got one index per Cloud project (although you can request more) and you create it with one command.

Next, I created a repository group for the index. You use these to control access to repos, and could have different ones for different dev audiences. Here’s where you actually point to a given repo that has the Developer Connect app installed.

I ran this command a few times to ensure that each of my three repos was added to the repository group (and index).

Indexing can take up to 24 hours, so here’s where you wait. After a day, I saw that all my target repos were successfully indexed.

Whenever I sync the fork with the latest updates to code samples, Gemini Code Assist will index the updated code automatically. And my IDE with Gemini Code Assist will have the freshest suggestions from our samples repo!

Step #4 – Use updated coding suggestions

Let’s prove that this worked.

I looked for a recent commit to the Go samples repos that the base Gemini Code Assist LLM wouldn’t know about yet. Here’s one that has new topic-creation parameters for our Managed Kafka service. I gave the prompt below to Gemini Code Assist. First, I used a project and account that was NOT tied to the code customization index.

//function to create a topic in Google Cloud Managed Kafka and include parameters for setting replicationfactor and partitioncount

The coding suggestion was good, but incomplete as it was missing the extra configs the service can now accept.

When I went to my Code Assist environment that did have code customization turned on, you see that the same prompt gave me a result that mirrored the latest Go sample code.

I tried a handful of Java and Go prompts, and I regularly (admittedly, not always) got back exactly what I wanted. Good prompt engineering might have helped me reach 100%, but I still appreciated the big increase in quality results. It was amazing to have hundreds of up-to-date Google-tested code samples to enrich my AI-provided suggestions!

AI coding assistants that offer code customization from your own repos are a difference maker. But don’t stop at your own code. Index other great code repos that represent the coding standards and fresh content your developers need!

Trust. Without trust, AI coding assistants won’t become a default tool in a developer’s toolbox. Trust is the #1 concern of devs today, and it’s something I’ve struggled with in regards to getting the most relevant answers from an LLM. Specifically, am I getting back the latest information? Probably not, given that LLMs have a training cutoff date. Your AI coding assistant probably doesn’t (yet) know about Python 3.13, the most recent features of your favorite cloud service, or the newest architectural idea shared at a conference last week. What can you do about that?

To me, this challenge comes up in at least three circumstances. There are entirely new concepts or tools that the LLM training wouldn’t know about. Think something like pipe syntax as an alternative to SQL syntax. I wouldn’t expect a model trained last year to know about that. How about updated features to existing libraries or frameworks? I want suggestions that reflect the full feature set of the current technology and I don’t want to accidentally do something the hard (old) way. An example? Consider the new “enum type” structured output I can get from LangChain4J. I’d want to use that now! And finally, I think about improved or replicated framework libraries. If I’m upgrading from Java 8 to Java 23, or Deno 1 to Deno 2, I want to ensure I’m not using deprecated features. My AI tools probably don’t know about any of these.

I see four options for trusting the freshness of responses from your AI assistant. The final technique was brand new to me, and I think it’s excellent.

1. Fine-tune your model
2. Use retrieval augmented generation (RAG)
3. Ground the results with trusted sources
4. “Train” on the fly with input context

Let’s briefly look at the first three, and see some detailed examples of the fourth.

Fine-tune your model

Whether using commercial or open models, they all represent a point-in-time based on their training period. You could choose to repeatedly train your preferred model with fresh info about the programming languages, frameworks, services, and patterns you care about.

The upside? You can get a model with knowledge about whatever you need to trust it. The downside? It’s a lot of work—you’d need to craft a healthy number of examples and must regularly tune the model. That could be expensive, and the result wouldn’t naturally plug into most AI coding assistance tools. You’d have to jump out of your preferred coding tool to ask questions of a model elsewhere.

Use RAG

Instead of tuning a serving a custom model, you could choose to augment the input with pre-processed content. You’ll get back better, more contextual results when taking into account data that reflects the ideal state.

The upside? You’ll find this pattern increasingly supported in commercial AI assistants. This keeps you in your flow without having to jump out to another interface. GitHub Copilot offers this, and now our Gemini Code Assist provides code customization based on repos in GitHub or GitLab. With Code Assist, we handle the creation and management of the code index of your repos, and you don’t have to manually chunk and store your code. The downside? This only works well if you’ve got the most up-to-date data in an indexed source repo. If you’ve got old code or patterns in there, that won’t help your freshness problem. And while these solutions are good for extra code context, they may not support a wider range of possible context sources (e.g. text files).

Ground the results

This approach gives you more confidence that the results are accurate. For example, Google Cloud’s Vertex AI offers “ground with Google Search” so that responses are matched to real, live Google Search results.

If I ask a question about upgrading an old bit of Deno code, you can see that the results are now annotated with reference points. This gives me confidence to some extent, but doesn’t necessarily guarantee that I’m getting the freshest answers. Also, this is outside of my preferred tool, so it again takes me out of a flow state.

Train on the fly

Here’s the approach I just learned about from my boss’s boss, Keith Ballinger. I complained about freshness of results from AI assistance tools, and he said “why don’t you just train it on the fly?” Specifically, pass the latest and greatest reference data into a request within the AI assistance tool. Mind … blown.

How might it handle entirely new concepts or tools? Let’s use that pipe syntax example. In my code, I want to use this fresh syntax instead of classic SQL. But there’s no way my Gemini Code Assist environment knows about that (yet). Sure enough, I just get back a regular SQL statement.

But now, Gemini Code Assist supports local codebase awareness, up to 128,000 input tokens! I grabbed the docs for pipe query syntax, saved as a PDF, and then asked Google AI Studio to produce a Markdown file of the docs. Note that Gemini Code Assist isn’t (yet) multi-modal, so I need Markdown instead of passing in a PDF or image. I then put a copy of that Markdown file in a “training” folder within my app project. I used the new @ mention feature in our Gemini Code Assist chat to specifically reference the syntax file when asking my question again.

Wow! So by giving Gemini Code Assist a reference file of pipe syntax, it was able to give me an accurate, contextual, and fresh answer.

What about updated features to existing libraries or frameworks? I mentioned the new feature of LangChain4J for the Gemini model. There’s no way I’d expect my coding assistant to know about a feature added a few days ago. Once again, I grabbed some resources. This time, I snagged the Markdown doc for Google Vertex AI Gemini from the LangChain4J repo, and converted a blog post from Guillaume to Markdown using Google AI Studio.

My prompt to the Gemini Code Assist model was “Update the service function with a call to Gemini 1.5 Flash using LangChain4J. It takes in a question about a sport, and the response is mapped to an enum with values for baseball, football, cricket, or other.” As expected, the first response was a good attempt, but it wasn’t fully accurate. And it used a manual way to map the response to an enum.

What if I pass in both of those training files with my prompt? I get back exactly the syntax I wanted for my Cloud Run Function!

So great. This approach requires me to know what tech I’m interested in up front, but still, what an improvement!

Final example. How about improved or replicated framework libraries? Let’s say I’ve got a very old Deno app that I created when I first got excited about this excellent JavaScript runtime.

// from https://deno.com/blog/v1.35#denoserve-is-now-stable
async function handleHttp(conn: Deno.Conn) {
  // `await` is needed here to wait for the server to handle the request
  await (async () => {
    for await (const r of Deno.serveHttp(conn)) {
      r.respondWith(new Response("Hello World from Richard"));
    }
  })();
}

for await (const conn of Deno.listen({ port: 8000 })) {
  handleHttp(conn);
}

This code uses some libraries and practices that are now out of date. When I modernize this app, I want to trust that I’m doing it the best way. Nothing to fear! I grabbed the Deno 1.x to 2.x migration guide, a blog post about the new approach to web servers, and the launch blog for Deno 2. The result? Impressive, including a good description of why it generated the code this way.

I could imagine putting the latest reference apps into a repo and using Gemini Code Assist’s code customization feature to pull that automatically into my app. But this demonstrated technique gives me more trust in the output of tool when freshness is paramount. What do you think?

School is back in session, and I just met with a handful of teachers at a recent back-to-school night. They’re all figuring out how to account for generative AI tools that students have access to. I say, let’s give teachers the same tools to use. Specifically, what if a teacher wants a quick preliminary grade on book reports submitted by their students? To solve this, I used Gemini Flash 1.5 in Google Cloud Vertex AI in three different ways—one-off in the prompt editor, through code, and via declarative workflow.

Grade Homework in Vertex AI Studio

Maybe you just have one or two papers to grade. Something like Vertex AI Studio is a good choice. Even if you’re not a Google Cloud customer, you can use it for free through this link.

You don’t have any book reports handy to test this with? Me neither. In Vertex AI Studio, I prompted with something like “Write a 300 word book report for Pride and Prejudice from the perspective of an 8th grade student with a good vocabulary and strong writing skills.”

With that handy, let’s “grade” the paper. Here I’m using system instructions to direct the model. I used Gemini to generate the initial version and then I tweaked it a bit. Here’s the instructions:

Task: Evaluate 8th-grade book reports for an honors English class. You are a tough grader. Input: Book report text. Output: Initial letter grade (A, B, C, D, or F) based on: Structure: Clear introduction, body, and conclusion Grammar: Spelling, punctuation, sentence structure. Content: Understanding of the book, critical thinking. Consider: Age and language proficiency of the student.

This tells the model what to look for, the expected quality (“honors English”), and the desired output.

I added the system instructions to the marked section within Vertex AI Studio.

Then I simply pasted the book report into the “prompt” section and submitted the prompt.

I thought this was a pretty good book report, but Gemini 1.5 Flash had a high bar after obeying my system instructions.

This book report shows good understanding of the book and a strong grasp of the themes. However, there are some issues:

Structure:
The introduction is a little long and could be more concise.
The conclusion is also repetitive.
The report could benefit from more clear paragraph breaks to organize the information more effectively.

Grammar:
No significant grammar issues.

Content:
The student demonstrates good understanding of the novel's major themes and characters.
There's a strong focus on plot summary, which is okay, but the report could benefit from more critical analysis. The student needs to go beyond just describing the events to analyze how they impact the themes, characters, or messages of the book.
Overall: This report shows good effort, but it needs to be refined.

Grade: B-

If I’m a teacher, this is a fascinating way to get a preliminary assessment of a writing assignment!

Grade Homework Through Code

The above solution works fine for one-off experiences, but how might you scale this AI-assisted grader? Another option is code.

To try this scenario out, I used Cloud Firestore as my document database holding the book reports. I created a collection named “Papers” in the default database and added three documents. Each one holds a different book report.

I think used the Firestore API and Vertex AI API to write some simple Go code that iterates through each Firestore document, calls Vertex AI using the provided system instructions, and then logs out the grade for each report. Note that I could have used a meta framework like LangChain, LlamaIndex, or Firebase Genkit, but I didn’t see the need.

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"cloud.google.com/go/firestore"
	"cloud.google.com/go/vertexai/genai"
	"google.golang.org/api/iterator"
)

func main() {
	// get configuration from environment variables
	projectID := os.Getenv("PROJECT_ID") 
	collectionName := os.Getenv("COLLECTION_NAME") // "Papers"
	location := os.Getenv("LOCATION")              //"us-central1"
	modelName := os.Getenv("MODEL_NAME")           // "gemini-1.5-flash-001"

	ctx := context.Background()

	//initialize Vertex AI client
	vclient, err := genai.NewClient(ctx, projectID, location)
	if err != nil {
		log.Fatalf("error creating vertex client: %v\n", err)
	}
	gemini := vclient.GenerativeModel(modelName)

	//add system instructions
	gemini.SystemInstruction = &genai.Content{
		Parts: []genai.Part{genai.Text(`Task: Evaluate 8th-grade book reports for an honors English class. You are a tough grader. Input: Book report text. Output: Initial letter grade (A, B, C, D, or F) based on: Structure: Clear introduction, body, and conclusion Grammar: Spelling, punctuation, sentence structure. Content: Understanding of the book, critical thinking. Consider: Age and language proficiency of the student.
		`)},
	}

	// Initialize Firestore client
	client, err := firestore.NewClient(ctx, projectID)
	if err != nil {
		log.Fatalf("Failed to create client: %v", err)
	}
	defer client.Close()

	// Get documents from the collection
	iter := client.Collection(collectionName).Documents(ctx)
	for {
		doc, err := iter.Next()
		if err != nil {
			if err == iterator.Done {
				break
			}
			log.Fatalf("error iterating through documents: %v\n", err)
		}

		//create the prompt
		prompt := genai.Text(doc.Data()["Contents"].(string))

		//call the model and get back the result
		resp, err := gemini.GenerateContent(ctx, prompt)
		if err != nil {
			log.Fatalf("error generating context: %v\n", err)
		}

		//print out the top candidate part in the response
		log.Println(resp.Candidates[0].Content.Parts[0])
	}

	fmt.Println("Successfully iterated through documents!")
}

The code isn’t great, but the results were. I’m also getting more verbose responses from the model, which is cool. This is a much more scalable way to quickly grade all the homework.

Grade Homework in Cloud Workflows

I like the code solution, but maybe I want to run this preliminary grading on a scheduled basis? Every Tuesday night? I could do that with my above code, but how about using a no-code workflow engine? Our Google Cloud Workflows product recently got a Vertex AI connector. Can I make it work with the same system instructions as the above two examples? Yes, yes I can.

I might be the first person to stitch all this together, but it works great. I first retrieved the documents from Firestore, looped through them, and called Vertex AI with the provided system instructions. Here’s the workflow’s YAML definition:

main:
  params: [args]
  steps:
  - init:
      assign:
        - collection: ${args.collection_name}
        - project_id: ${args.project_id}
        - location: ${args.location}
        - model: ${args.model_name}
  - list_documents:
        call: googleapis.firestore.v1.projects.databases.documents.list
        args:
            collectionId: ${collection}
            parent: ${"projects/" + project_id + "/databases/(default)/documents"}
        result: documents_list
  - process_documents:
        for:
          value: document 
          in: ${documents_list.documents}
          steps:
            - ask_llm:
                call: googleapis.aiplatform.v1.projects.locations.endpoints.generateContent
                args: 
                    model: ${"projects/" + project_id + "/locations/" + location + "/publishers/google/models/" + model}
                    region: ${location}
                    body:
                        contents:
                            role: "USER"
                            parts:
                                text: ${document.fields.Contents.stringValue}
                        systemInstruction: 
                            role: "USER"
                            parts:
                                text: "Task: Evaluate 8th-grade book reports for an honors English class. You are a tough grader. Input: Book report text. Output: Initial letter grade (A, B, C, D, or F) based on: Structure: Clear introduction, body, and conclusion Grammar: Spelling, punctuation, sentence structure. Content: Understanding of the book, critical thinking. Consider: Age and language proficiency of the student."
                        generation_config:
                            temperature: 0.5
                            max_output_tokens: 2048
                            top_p: 0.8
                            top_k: 40
                result: llm_response
            - log_file_name:
                call: sys.log
                args:
                    text: ${llm_response}

No code! I executed the workflow, passing in all the runtime arguments.

In just a moment, I saw my workflow running, and “grades” being logged to the console. In real life, I’d probably update the Firestore document with this information. I’d also use Cloud Scheduler to run this on a regular basis.

While I made this post about rescuing educators from the toil of grading papers, you can apply these patterns to all sorts of scenarios. Use prompt editors like Vertex AI Studio for experimentation and finding the right prompt phrasing. Then jump into code to interact with models in a repeatable, programmatic way. And consider low-code tools when model interactions are scheduled, or part of long running processes.

I’m skeptical of anything that claims to make difficult things “easy.” Easy is relative. What’s simple for you might draw blood from me. And in my experience, when a product claims to make something “easy”, it’s talking about simplifying a subset of the broader, more complicated job-to-be-done.

So I won’t sit here and tell you that generative AI makes app modernization easy. Nothing does. It’s hard work and is as much about technology as it is psychology and archeology. But AI can make it easier. We’ll take any help we can get, right? I count at least five ways I’d use generative AI to make smarter progress on my modernization journey.

#1 Understand the codebase

Have you been handed a pile of code and scripts before? Told to make sense of it and introduce some sort of feature enhancement? You might spend hours, days, or weeks figuring out the relationships between components and side effects of any changes.

Generative AI is fairly helpful here. Especially now that things like Gemini 1.5 (with its 1 million token input) exist.

I might use something like Gemini (or ChatGPT, or whatever) to ask questions about the code base and get ideas for how something might be used. This is where the “generative” part is handy. When I use the Duet AI assistance in to explain SQL in BigQuery, I get back a creative answer about possible uses for the resulting data.

In your IDE, you might use Duet AI (or Copilot, Replit, Tabnine) to give detailed explanations of individual code files, shell scripts, YAML, or Dockerfiles. Even if you don’t decide to use any generative AI tools to write code, consider using them to explain it.

#2 Incorporate new language/framework features

Languages themselves modernize at a fairly rapid pace. Does your codebase rely on a pattern that was rad back in 2011? It happens. I’ve seen that generative AI is a handy way to modernize the code itself while teaching us how to apply the latest language features.

For instance, Go generics are fairly new. If your Go app is more than 2 years old, it wouldn’t be using them. I could go into my Go app and ask my generative AI chat tool for advice on how to introduce generics to my existing code.

Usefully, the Duet AI tooling also explains what it did, and why it matters.

I might use the same types of tools to convert an old ASP.NET MVC app to the newer Minimal APIs structure. Or replace deprecated features from Spring Boot 3.0 with more modern alternatives. Look at generative AI tools as a way to bring your codebase into the current era of language features.

#3 Improve code quality

Part of modernizing an app may involve adding real test coverage. You’ll never continuously deploy an app if you can’t get reliable builds. And you won’t get reliable builds without good tests and a CI system.

AI-assisted developer tools make it easier to add integration tests to your code. I can go into my Spring Boot app and get testing scaffolding for my existing functions.

Consider using generative AI tools to help with broader tasks like defining an app-wide test suite. You can use these AI interfaces to brainstorm ideas, get testing templates, or even generate test data.

In addition to test-related activities, you can use generative AI to check for security issues. These tools don’t care about your feelings; here, it’s calling out my terrible practices.

Fortunately, I can also ask the tool to “fix” the code. You might find a few ways to use generative AI to help you refactor and improve the resilience and quality of the codebase.

#4 Swap out old or unsupported components

A big part of modernization is ensuring that a system is running fully supported components. Maybe that database, plugin, library, or entire framework is now retired, or people don’t want to work with it. AI tools can help with this conversion.

For instance, maybe it’s time to swap out JavaScript frameworks. That app you built in 2014 with Backbone.js or jQuery is feeling creaky. You want to bring in React or Angular instead. I’ve had some luck coaxing generative AI tools into giving me working versions of just that. Even if you use AI chat tools to walk you through the steps (versus converting all the code), it’s a time-saver.

The same may apply to upgrades from Java 8 to Java 21, or going from classic .NET Framework to modern .NET. Heck, you can even have some luck switching from COBOL to Go. I wouldn’t blindly trust these tools to convert code; audit aggressively and ensure you understand the new codebase. But these tools may jump start your work and cut out some of the toil.

#5 Upgrade the architecture

Sometimes an app modernization requires some open-heart surgery. It’s not about light refactoring or swapping a frontend framework. No, there are times where you’re yanking out major pieces or making material changes.

I’ve had some positive experiences asking generative AI tools to help me upgrade a SOAP service to REST. Or REST to gRPC. You might use these tools to switch from a stored procedure-heavy system to one that puts the logic into code components instead. Speaking of databases, you could change from MySQL to Cloud Spanner, or even change a non-relational database dependency back to a relational one. Will generative AI do all the work? Probably not, but much of it’s pretty good.

This might be a time to make bigger changes like swapping from one cloud to another, or adding a major layer of infrastructure-as-code templates to your system. I’ve seen good results from generative AI tools here too. In some cases, a modernization project is your chance to introduce real, lasting changes to a architecture. Don’t waste the opportunity!

Wrap Up

Generative AI won’t eliminate the work of modernizing an app. There’s lots of work to do to understand, transform, document, and rollout code. AI tools can make a big difference, though, and you’re tying a hand behind your back if you ignore it! What other uses for app modernization come to mind?

How many chatbots do we really need? While chatbots are a terrific example app for generative AI use cases, I’ve been thinking about how developers may roll generative AI into existing “boring” apps and make them better.

As I finished all my Christmas shopping—much of it online—I thought about all the digital storefronts and how they provide recommended items based on my buying patterns, but serve up the same static item descriptions, regardless of who I am. We see the same situation with real estate listings, online restaurant menus, travel packages, or most any catalog of items! What if generative AI could create a personalized story for each item instead? Wouldn’t that create such a different shopping experience?

Maybe this is actually a terrible idea, but during the Christmas break, I wanted to code an app from scratch using nothing but Google Cloud’s Duet AI while trying out our terrific Gemini LLM, and this seemed like a fun use case.

The final app (and codebase)

The app shows three types of catalogs and offers two different personas with different interests. Everything here is written in Go and uses local files for “databases” so that it’s completely self-contained. And all the images are AI-generated from Google’s Imagen2 model.

When the user clicks on a particular catalog entry, the go to a “details” page where the generic product summary from the overview page is sent along with a description of the user’s preferences to the Google Gemini model to get a personalized, AI-powered product summary.

That’s all there is to it, but I think it demonstrates the idea.

How it works

Let’s look at what we’ve got here. Here’s the basic flow of the AI-augmented catalog request.

How did I build the app itself (GitHub repo here)? My goal was to only use LLM-based guidance either within the IDE using Duet AI in Google Cloud, or burst out to Bard where needed. No internet searches, no docs allowed.

I started at the very beginning with a basic prompt.

What are the CLI commands to create a new Go project locally?

The answer offered the correct steps for getting the project rolling.

The next commands are where AI assistance made a huge difference for me. With this series of natural language prompts in the Duet AI chat within VS Code, I got the foundation of this app set up in about five minutes. This would have easily taken me 5 or 10x longer if I did it manually.

Give me a main.go file that responds to a GET request by reading records from a local JSON file called property.json and passes the results to an existing html/template named home.html. The record should be defined in a struct with fields for ID, Name, Description, and ImageUrl.

Create an html/template for my Go app that uses Bootstrap for styling, and loops through records. For each loop, create a box with a thin border, an image at the top, and text below that. The first piece of text is "title" and is a header. Below that is a short description of the item. Ensure that there's room for four boxes in a single row.

Give me an example data.json that works with this struct

Add a second function to the class that responds to HTML requests for details for a given record. Accept a record id in the querystring and retrieve just that record from the array before sending to a different html/template

With these few prompts, I had 75% of my app completed. Wild! I took this baseline, and extended it. The final result has folders for data, personas, images, a couple HTML files, and a single main.go file.

Let’s look at the main.go file, and I’ll highlight a handful of noteworthy bits.

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"html/template"
	"log"
	"net/http"
	"os"
	"strconv"

	"github.com/google/generative-ai-go/genai"
	"google.golang.org/api/option"
)

// Define a struct to hold the data from your JSON file
type Record struct {
	ID          int
	Name        string
	Description string
	ImageURL    string
}

type UserPref struct {
	Name        string
	Preferences string
}

func main() {

	// Parse the HTML templates
	tmpl := template.Must(template.ParseFiles("home.html", "details.html"))

	//return the home page
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

		var recordType string
		var recordDataFile string
		var personId string

		//if a post-back from a change in record type or persona
		if r.Method == "POST" {
			// Handle POST request:
			err := r.ParseForm()
			if err != nil {
				http.Error(w, "Error parsing form data", http.StatusInternalServerError)
				return
			}

			// Extract values from POST data
			recordType = r.FormValue("recordtype")
			recordDataFile = "data/" + recordType + ".json"
			personId = r.FormValue("person")

		} else {
			// Handle GET request (or other methods):
			// Load default values
			recordType = "property"
			recordDataFile = "data/property.json"
			personId = "person1" // Or any other default person
		}

		// Parse the JSON file
		data, err := os.ReadFile(recordDataFile)
		if err != nil {
			fmt.Println("Error reading JSON file:", err)
			return
		}

		var records []Record
		err = json.Unmarshal(data, &records)
		if err != nil {
			fmt.Println("Error unmarshaling JSON:", err)
			return
		}

		// Execute the template and send the results to the browser
		err = tmpl.ExecuteTemplate(w, "home.html", struct {
			RecordType string
			Records    []Record
			Person     string
		}{
			RecordType: recordType,
			Records:    records,
			Person:     personId,
		})
		if err != nil {
			fmt.Println("Error executing template:", err)
		}
	})

	//returns the details page using AI assistance
	http.HandleFunc("/details", func(w http.ResponseWriter, r *http.Request) {

		id, err := strconv.Atoi(r.URL.Query().Get("id"))
		if err != nil {
			fmt.Println("Error parsing ID:", err)
			// Handle the error appropriately (e.g., redirect to error page)
			return
		}

		// Extract values from querystring data
		recordType := r.URL.Query().Get("recordtype")
		recordDataFile := "data/" + recordType + ".json"

		//declare recordtype map and extract selected entry
		typeMap := make(map[string]string)
		typeMap["property"] = "Create an improved home listing description that's seven sentences long and oriented towards a a person with these preferences:"
		typeMap["store"] = "Create an updated paragraph-long summary of this store item that's colored by these preferences:"
		typeMap["restaurant"] = "Create a two sentence summary for this menu item that factors in one or two of these preferences:"
		//get the preamble for the chosen record type
		aiPremble := typeMap[recordType]

		// Parse the JSON file
		data, err := os.ReadFile(recordDataFile)
		if err != nil {
			fmt.Println("Error reading JSON file:", err)
			return
		}

		var records []Record
		err = json.Unmarshal(data, &records)
		if err != nil {
			fmt.Println("Error unmarshaling JSON:", err)
			return
		}

		// Find the record with the matching ID
		var record Record
		for _, rec := range records {
			if rec.ID == id { // Assuming your struct has an "ID" field
				record = rec
				break
			}
		}

		if record.ID == 0 { // Record not found
			// Handle the error appropriately (e.g., redirect to error page)
			return
		}

		//get a reference to the persona
		person := "personas/" + (r.URL.Query().Get("person") + ".json")

		//retrieve preference data from file name matching person variable value
		preferenceData, err := os.ReadFile(person)
		if err != nil {
			fmt.Println("Error reading JSON file:", err)
			return
		}
		//unmarshal the preferenceData response into an UserPref struct
		var userpref UserPref
		err = json.Unmarshal(preferenceData, &userpref)
		if err != nil {
			fmt.Println("Error unmarshaling JSON:", err)
			return
		}

		//improve the message using Gemini
		ctx := context.Background()
		// Access your API key as an environment variable (see "Set up your API key" above)
		client, err := genai.NewClient(ctx, option.WithAPIKey(os.Getenv("GEMINI_API_KEY")))
		if err != nil {
			log.Fatal(err)
		}
		defer client.Close()

		// For text-only input, use the gemini-pro model
		model := client.GenerativeModel("gemini-pro")
		resp, err := model.GenerateContent(ctx, genai.Text(aiPremble+" "+userpref.Preferences+". "+record.Description))
		if err != nil {
			log.Fatal(err)
		}

		//parse the response from Gemini
		bs, _ := json.Marshal(resp.Candidates[0].Content.Parts[0])
		record.Description = string(bs)

		//execute the template, and pass in the record
		err = tmpl.ExecuteTemplate(w, "details.html", record)
		if err != nil {
			fmt.Println("Error executing template:", err)
		}
	})

	fmt.Println("Server listening on port 8080")
	fs := http.FileServer(http.Dir("./images"))
	http.Handle("/images/", http.StripPrefix("/images/", fs))
	http.ListenAndServe(":8080", nil)
}

I do not write great Go code, but it compiles, which is good enough for me!

On line 13, see that I refer to the Go package for interacting with the Gemini model. All you need is an API key, and we have a generous free tier.

On line 53, notice that I’m loading the data file based on the type of record picked on the HTML template.

On line 79, I’m executing the HTML template and sending the type of record (e.g. property, restaurant, store), the records themselves, and the persona.

On lines 108-113, I’m storing a map of prompt values to use for each type of record. These aren’t terrific, and could be written better to get smarter results, but it’ll do.

Notice on line 147 that I’m grabbing the user preferences we use for customization.

On line 163, I create a Gemini client so that I can interact with the LLM.

On line 171, see that I’m generating AI content based on the record-specific preamble, the record details, and the user preference data.

On line 177, notice that I’m extracting the payload from Gemini’s response.

Finally, on line 181 I’m executing the “details” template and passing in the AI-augmented record.

None of this is rocket science, and you can check out the whole project on GitHub.

What an “enterprise” version might look like

What I have here is a local example app. How would I make this more production grade?

• Store catalog images in an object storage service. All my product images shouldn’t be local, of course. They belong in something like Google Cloud Storage.
• Add catalog items and user preferences to a database. Likewise, JSON files aren’t a great database. The various items should all be in a relational database.
• Write better prompts for the LLM. My prompts into Gemini are meh. You can run this yourself and see that I get some silly responses, like personalizing the message for a pillow by mentioning sporting events. In reality, I’d write smarter prompts that ensured the responding personalized item summary was entirely relevant.
• Use Vertex AI APIs for accessing Gemini. Google AI Studio is terrific. For production scenarios, I’d use the Gemini models hosted in full-fledged MLOps platform like Vertex AI.
• Run app in a proper cloud service. If I were really building this app, I’d host it in something like Google Cloud Run, or maybe GKE if it was part of a more complex set of components.
• Explore whether pre-generating AI-augmented results and caching them would be more performant. It’s probably not realistic to call LLM endpoints on each “details” page. Maybe I’d pre-warm certain responses, or come up with other ways to not do everything on the fly.

This exercise helped me see the value of AI-assisted developer tooling firsthand. And, it feels like there’s something useful about LLM summarization being applied to a variety of “boring” app scenarios. What do you think?

The world is a better place now that no one pays me to write code. You’re welcome. But I still try to code on a regular basis, and there are a handful of activities I don’t particularly enjoy doing. Most of these relate to the fact that I’m not coding every day, and thus have to waste a lot of time looking up information I’ve forgotten. I’m not alone; the latest StackOverflow developer survey showed that most of us are spending 30 or more minutes a day searching online for answers.

Generative AI may solve those problems. Whether we’re talking AI-assisted development environments—think GitHub Copilot, Tabnine, Replit, or our upcoming Duet AI for Google Cloud—or chat-based solutions, we now have tools to save us from all the endless lookups for answers.

Google Bard has gotten good at coding-related activities, and I wanted to see if it could address some of my least-favorite developer activities. I won’t do anything fancy (e.g. multi-shot prompts, super sophisticated prompts) but will try and write smart prompts that give me back great results on the first try. Once I learn to write better prompts, the results will likely be even better!

#1 – Generate SQL commands

Virtually any app you build accesses a database. I’ve never been great at writing SQL commands, and end up pulling up a reference when I have to craft a JOIN or even a big INSERT statement. Stop judging me.

Here’s my prompt:

I’m a Go developer. Give me a SQL statement that inserts three records into a database named Users. There are columns for userid, username, age, and signupdate.

The result?

What if I want another SQL statement that joins the table with another and counts up the records in the related table? I was impressed that Bard could figure it out based on this subsequent prompt:

How about a join statement between that table and another table called LoginAttempts that has columns for userid, timestamp, and outcome where we want a count of loginattempts per user.

The result? A good SQL statement that seems correct to me. I love that it also gave me an example resultset to consider.

I definitely plan on using Bard for help me with SQL queries from now on.

#2 – Create entity definitions or DTOs

For anything but the simplest apps, I find myself creating classes to represent the objects used by my app. This isn’t hard work, but it’s tedious at times. I’d rather not plow through a series of getters and setters for private member variables, or setup one or more constructors. Let’s see how Bard does.

My prompt:

I’m a Java developer using Spring Boot. Give me a class that defines an object named Employees with fields for employeeid, name, title, location, and managerid.

The result is a complete object, with a “copy” button (so I can paste this right into my IDE), and even source attribution for where the model found the code.

What if I wanted a second constructor that only takes in one parameter? Because the chat is ongoing and supports back-and-forth engagement, I could simply ask “Give me the same class, but with a second constructor that only takes in the employeeid” and get:

This is a time-saver, and I can easily start with this and edit as needed.

#3 – Bootstrap my frontend pages

I like using Bootstrap for my frontend layout, but I don’t use it often enough to remember how to configure it the way I want. Bard to the rescue!

I asked Bard for an HTML page that has a basic Bootstrap style. This is where it’s useful that Bard is internet-connected. It knows the latest version of Bootstrap, whereas other generative chat tools don’t have access to the most recent info.

My prompt is:

Write an HTML page that uses the latest version of Bootstrap to center a heading that says “Feedback Form.” Then include a form with inputs for “subject” and “message” to go with a submit button.

I get back valid HTML and an explanation of what it generated.

It looked right to me, but I wanted to make sure it wasn’t just a hallucination. I took that code, pasted it into a new HTML file, and opened it up. Yup, looks right.

I may not use tools like Bard to generate an entirely complex app, but scaffolding the base of something like a web page is a big time-saver.

#4 – Create declarative definitions for things like Kubernetes deployment YAML

I have a mental block on remembering the exact structure of configuration files. Maybe I should see a doctor. I can read them fine, but I never remember how to write most of them myself. But in the meantime, I’m stoked that generative AI can make it easier to pump out Kubernetes deployment and service YAML, Dockerfiles, Terraform scripts, and most anything else.

Let’s say I want some Kubernetes YAML in my life. I provided this prompt:

Give me Kubernetes deployment and service YAML that deploys two replicas and exposes an HTTP endpoint

What I got back was a valid pile of YAML.

And I do like the results also explain a bit about what’s going. on here, including the actual command to apply these YAML files to a cluster.

Dockerfiles still intimidate me for some reason. I like the idea of describing what I need, and having Bard give me a starter Dockerfile to work with. Full disclosure, I tried getting Bard to generate a valid Dockerfile for a few of my sample GitHub repos, and I couldn’t get a good one.

But this prompt works well:

Show me a Dockerfile for a Go application that has the main Go file in a cmd folder, but also uses other folders named web and config.

The result is a Dockerfile, and it’s well explained in the accompanying text.

Bard also seems pretty good at giving me Terraform, Java property file configurations, and more. As always, verify the results!

#5 – Create sample data

I build a lot of sample apps, which means I need sample data. That might mean input into a REST endpoint, or it could be data for a starter database. Instead of struggling to produce a bunch of fake records, generative AI solutions can give me all the structured data I need.

In the SQL section above, I generated a SQL insert command that I could use for a database. But I can also generate some seed data for my app.

For instance, how about this prompt:

Generate a JavaScript variable holding a JSON array of five baseball players. Pull names from the baseball Hall of Fame. Each record should have an id, name, position, and age.

I get back valid JSON that I could drop in my Node.js app.

I can then ask Bard to convert that to XML, CSV, or any number of formats. I might want more records or fewer records. This also works if you ask for your favorite television characters, musicians from the 1990s, or types of potatoes. Generate data without taxing your brain.

#6 – Explain code I come across in GitHub, blog posts, or StackOverflow

Do you ever find some interesting code in a blog post or StackOverflow answer but can’t exactly figure out how it works? Happens to me all the time. Instead of going further down the rabbit hole to figure out what each line of code does, I can use generative AI to make some sense of it all.

For instance, let’s assume I’m terrible at generating random numbers and stumble upon this StackOverflow answer that’s succinct and without any description. I can ask Bard to describe this code in more depth:

What does this code do? Explain it to me as if I’m new to C#. public string RandomDigits(int length)
{
var random = new Random();
string s = string.Empty;
for (int i = 0; i < length; i++)
s = String.Concat(s, random.Next(10).ToString());
return s;
}

What I get back is a written description of each line. And then, helpfully, a restating of the code with comments added before each statement.

If you’re learning a new language, consider using generative AI to explain code snippets to you. I acknowledge that I’ve had mixed luck pointing Bard at a code file or repo and getting a perfect explanation. I sometimes get hallucinations (e.g. refer to files to methods that don’t exist), but I expect this functionality to get better quickly.

#7 – Convert code from one language (or version) to another

Do you ever come across code that shows a useful pattern, but isn’t in the language you’re coding in? Maybe I found that code above to generate a random number, but want the equivalent in Go. Not a big problem any more. A prompt following the above prompt:

Convert this C# code to Go code instead

I get back Go code, and a description of what’s different.

Consider the case where you find code for calling a particular HTTP endpoint, but it’s in Java and your app is written in Go. My colleague Guillaume just wrote up a great post about calling our new Google PaLM LLM API via a Java app. I can ask for all the code in the post to be converted to Go.

The prompt:

Can you give me the equivalent Go code for the Java code in this blog post?/ https://glaforge.dev/posts/2023/05/30/getting-started-with-the-palm-api-in-the-java-ecosystem/

That’s pretty cool. I wasn’t sure that was going to work.

With all of these examples, it’s still important to verify the results. Generative AI doesn’t absolve you of responsibility as a developer; but it can give you a tremendous head start and save you from wasting tons of time navigating from source to source for answers.

Any other development activities that you’d like generative AI to assist you with?