Author: Richard Seroter

  • Google Cloud Next ’24 is better than last year’s event in every way but one

    Conferences aren’t cheap to attend. Forget the financial commitment—although that’s far from trivial—it’s expensive with regards to your time. You’re likely traveling out of town and spend time commuting. Then there’s the event itself, which takes you away from work and life for at least a few days. All of this in the hope of getting equal or greater value than what you spent. Risky bet? No doubt. I’ve been attending and organizing conferences for many years, and I’ll honestly say that this year’s Google Cloud Next ’24 is one of the surer bets I’ve seen. Even if you’re not a Google Cloud user (yet), I’m confident that you’d get a lot out of attending.

    The last edition of the event was terrific, but this one is better; except for one aspect, which I’ll mention at the end. This might be your best 2024 investment to learn about AI, modern app architectures and development, best practices for data access and analysis, and operations at scale. But why do I think it’s better than last year?

    There’s much more technical content

    We had too much introductory material in our breakout sessions last year. Level 100 content is super valuable, but you can get that anywhere. Many of us attend events to hear stories and go deeper than we can someplace else. This year, well over half of the breakouts are Level 200 or 300 content, and there’s a proper mix of introductory and in-depth material.

    There are breakouts for everybody. If you want to learn about AI, this is maybe the best event of the year. Go deep on GPUs and TPUs, learn about AI and serverless, study ML and streaming, build LLM apps with a RAG architecture, building AI apps with Go, creating gen AI apps with LangChain, using Gemini through Vertex AI, understanding vector searching, and 175+ more sessions.

    Are you a database enthusiast? Learn about high availability for relational databases, picking the right cloud database, non-relational database design patterns, how Yahoo! uses Cloud Spanner, managing databases with AI, and more.

    This is a terrific event for data scientists with dozens of breakouts. Learn about natural language analytics queries, continuous queries, using LLMs in BigQuery, vector search and multimodal embeddings, and lots more.

    Ops folks get a ton of content this year. Whether you’re building an internal developer platform on GKE, managing edge retail experiences at scale, embracing observability, setting up continuous deployment of AI models, migrating legacy workloads, securing multi-tenant Kubernetes, building a global service mesh, or advancing your logging infrastructure, you’ll leave the event smarter.

    And don’t forget about developers! We didn’t. With over 100 breakouts, we amped up the deep developer content. Learn about Java on serverless platforms, deploying apps to cloud, testing apps with testcontainers, building apps from scratch using AI assistance, pushing JavaScript apps to cloud, app troubleshooting, and tons more.

    Notice a better focus on developers and onsite learning

    Historically, Cloud Next was focused heavily on cloud services, but we also wanted to expand our usefulness for folks who are actually coding!

    For breakouts, we’ve got content for Android developers, those building Firebase apps, devs using Flutter, game developers, devs building with Angular, builders extending Workspace through APIs, and even those running training for Llama2!

    Our Innovator’s Hive is where you have 10s of thousands of square feet worth of demo stations featuring creative and educational examples of technology. And our first-time Community Hub offers education on Google-sponsored open tech like Android, Flutter, and more.

    Also, come for the dedicated tech training and certification options. This is more of a developer-centric program than I’ve ever seen from us.

    See more “now” technology to accompany “next” technology

    Last year’s event had lots of exciting previews, but much the tech wasn’t ready yet. We showed off AI developer assistance, previewed some new AI models, and talked about many things that were coming up soon.

    That’s all good, but now we have a better mix of “now” and “next.” You’ll continue seeing cutting edge tech that’s coming in the future, but you also will see more products, services, and frameworks that you can use RIGHT NOW.

    Hear from more industry expert voices

    Our developer keynote last year was so much fun, and we heard from awesome Google Champion Innovators. I loved it.

    We thought we’d mix it up this year and invite folks to our main stage that aren’t directly associated with Google. Our developer keynote features Guillermo Rauch, the CEO of Vercel; Josh Long, Spring advocate at Broadcom; and Charity Majors co-founder at Honeycomb. I’m a fan of all three people, which is why I’m amped that they accepted my invitation to join us on stage.

    And the breakouts themselves feature an absolute ton of customers and independent experts. A quick scan through our program gave me a list of speakers from companies like AMD, ANZ Bank, ASML, Accenture, Alaska Airlines, American Express, Anthropic, Anyscale, Apple, BMW AG, Bayer, Bayer Corporation, Belk, Bombardier, Boston Consulting Group, Box, CME Group, Carrefour, Charles Schwab, Chicago CTA, Citadel, Commerzbank AG, Core Logic, Covered California, Cox Communication, DZ Bank, Databricks, Deloitte, Deutsche Telekom, Devoteam G Cloud, Dialpad, DoIT International, Docker, Fiserv, Ford Motor Company, GitLab, Glean, Globe Telecom, Goldman Sachs, GrowthLoop, HCA Healthcare, HCL, HSBC, Harness, Hashicorp, Illinois Department of Human Services, Intel, KPMG, Lloyds Banking Group, Logitech, L’Oreal, Macquarie Bank, Major League Baseball, Mayo Clinic, Mercado Libre, Moloco, MongoDB, NJ Cybersecurity and Communications Cell, Nuro, Onix, OpenText, Palo Alto Networks, Paramount, Paypal, Pfizer, PwC, Quantiphi, Red Hat, Reddit, Rent the Runway, Roche, Roku, SADA, Sabre, Salesforce, Scotia Bank, Shopify, Snap, Spotify, Stability AI, Stagwell, Stanford, Symphony, Synk, TD Securities, Telus corporation, TransUnion, Trendyol, Typeface, UC Riverside, UPS, US News and World Report, Uber, Ubie, Inc, Unilever, Unity Technologies, Verizon, Walmart, Wayfair, Weights & Biases, Wells Fargo, Yahoo, and apree health. So many folks to learn from!

    Enjoy a bigger overall event

    This version of Next is going to be significantly larger than the last one, and that’s a good thing. I don’t want the conference to ever be festival-sized like Dreamforce or re:Invent, but having tens of thousands of folks in one place means a bigger breakout program, more learning opportunities, more serendipitous meetups, and a unique energy for attendees.

    We don’t have any musical numbers 😦

    The one thing that’s not better than last year? We couldn’t top our last keynote intro, and I didn’t try. There’s no musical tune featuring a sousaphone. That said, I genuinely think our developer keynote itself is even better overall this time, and the whole event should be memorable.

    There’s still time to register, and I’d love to bump into you if you attend. Let me know if you’ll be there!

  • Daily Reading List – March 15, 2024 (#277)

    Back at work after two bonkers days with a hundred fourth grade kids and their parents. Work today was more chill than the school field trip, and that’s saying something. Some light reading, but still good ones!

    [article] Measuring Developer Productivity via Humans. This article will get more added to it, but even the start is good. It offers a useful perspective on qualitative metrics and using that data to make better decisions.

    [blog] Bring Back Fun. Seriously. “Fun” doesn’t have to mean silly or lazy. But there should be more joy in our work as we get to build or support amazing things in tech.

    [blog] How to fail at platform engineering. This feels like an important post to internalize, as I worry that many folks will go about over-engineering their platforms in 2024.

    [article] The Benefits of Innovation That Isn’t Disruptive. It can be exciting to target the core of an existing industry with a disruptive solution, but there are risks as called out here.

    [article] Cloud Migrations Pick up the Pace in 2024. People are moving to cloud faster, while still facing challenges teasing apart app dependencies.

    [article] Meet DBOS: A Database Alternative to Kubernetes. Interesting idea. Run operating system services atop a distributed database?

    [blog] More powerful Go execution traces. If you’re troubleshooting an app, having access to trace data is super helpful. But tracing can also be resource intensive and tough to scale. The Go team made some excellent updates to tracing in the standard library.

    [blog] Maximize your cloud potential at Next ‘24: Lead with learning at any skill level. You can learn from anywhere, but there’s something powerful about intentionally going somewhere with intent to learn. Try and give yourself that gift this year by finding a conference that helps you grow professionally.

    ##

    Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:

  • Daily Reading List – March 12, 2024 (#276)

    I’m taking off tomorrow and Thursday to accompany my daughter on an overnight school field trip to our state capitol. It’ll be … something. But I’m fairly sure this is one of those moments I’ll look back on in 20 years and be grateful I took the time off. The Reading List will be back on Friday!

    [article] Why most AI benchmarks tell us so little. All these “we’re best at measure X or Y” claims for LLMs are impressive, but what does it really convey about how people use these models?

    [article] How to Succeed When You’re Not the Boss’s Favorite. Useful advice. It’s naive to think that we don’t have people in our families or at work that we prefer to spend time with. But managers have to be careful to check for bias.

    [blog] Building Meta’s GenAI Infrastructure. Simply absurd scale, but there are a handful of companies that need this level of capacity to serve the world.

    [paper] Retrieval-Augmented Generation for AI-Generated Content: A Survey. If you’re going from “curious” about RAG to studying it deeply, this seems like a useful paper to review.

    [blog] Introducing Security Command Center Enterprise: The first multicloud risk management solution fusing AI-powered SecOps with cloud security. There are some impressive capabilities here which should help keep you safer, regardless of what cloud you use.

    [blog] Great Marketing Machines Are Like Costco. Just like there’s not one thing that makes Costco amazing, a complex system like a marketing engine doesn’t have a single lever.

    [article] JPL’s Voyager team ‘extremely hopeful’ after ailing, faraway craft shows signs of former self. One of mankind’s greatest machines carries on, and might be on the mend.

    [blog] Processing One Billion Rows in PHP! That one billion row challenges comes to PHP? Sure, why not. The author does a good job continuously refining the work.

    [article] Costs, not security, worry enterprise leaders most as cloud estates multiply. Not surprising, and it’s not a binary choice; you can be cost conscious while also diligent about security.

    [blog] Tips for Getting Un-Stuck on a Blogging Journey. It’s not hard to fall out of rhythm when blogging or writing! I liked this post about how to get back on track.

    [blog] Chess.com boosts performance, cuts response times by 71% with Cloud SQL Enterprise Plus. For live gaming companies, every millisecond of latency counts. Heres a story of database optimization.

    [blog] From ESB to Cloud Native: Building Modern Integrations. Is today’s event broker or even cloud-based messaging service wildly different from a classic ESB? In some respects, but it’s still possible to use modern shiny tech to craft a centralized service with lots of coupling.

    ##

    Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:

  • Daily Reading List – March 11, 2024 (#275)

    I enjoyed a lot of sunshine with the family this weekend, while also getting some productive work done. This is a short week for me given some time off, so I’m doing the “five days of work in three days” approach. Never a good idea. Don’t be like me.

    [article] The Shift from Models to Compound AI Systems. The one matters if you’re a developer, architect, or ops person. A compound AI system uses multiple components, and is becoming increasingly common. There are a lot of implications of a system like this.

    [article] How to Make Small Talk with Anyone from Anywhere. I’m not really using my degree from college, but the thing I learned most during those four years was how to small talk with strangers. This article is helpful if you find yourself stuck.

    [blog] The life of an Ollama prompt. This tool has gotten fairly popular as a means for trying out LLMs locally. It’s not a terrible idea to become more familiar with it!

    [blog] Intelligent Document Discovery with Vertex AI Search. Create a more personalized, Google-quality search experience for your website? Interesting stuff that you can use today.

    [article] Accenture to buy Udacity, pledges $1B more in AI skills push. Interesting! I can understand a professional services firm absorbing a training platform.

    [blog] Gen AI Grounding with Vertex AI LLM. “Grounding” an LLM to reduce hallucinations and increase trustworthiness sounds complicated. This walkthrough shows that it’s easier than you might think.

    [book] Eloquent JavaScript, 4th Edition. Do yourself a favor, and at least skim through this. It’s an excellent book, and it’s free online or as a download.

    [blog] Memorystore for Redis vector search and LangChain integrations for gen AI. You don’t always need to introduce specialty services to embrace a new trend. Need a vector database? Redis works. As does your existing Cloud Spanner database.

    [article] How generative AI will change low-code development. Can AI just produce the same code a low-code platform does? It’s probably not that simple, but classic low-code feels less desirable nowadays.

    ##

    Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:

  • Daily Reading List – March 8, 2024 (#274)

    Happy Friday! I’ve bringing some work home with me this weekend, but I try to only mess with it while the family sleeps in. Dig into some good architectural and leadership pieces below.

    [blog] Domain-specific AI apps: A three-step design pattern for specializing LLMs. Really good. This helps you compare three options (prompt engineering, RAG, and fine tuning) for customizing AI model output.

    [site] Google Cloud Documentation. These types of sites “change” at most every few years. But we just published a massive update to our docs portal, and it offers easier navigation and better discoverability. Poke around!

    [article] Kubernetes is (not) a cost optimization problem. There’s an angle here by someone offering a container alternative, but Matt knows Kubernetes and talks about where the costs come from.

    [article] 3 Key Metrics to Measure Developer Productivity. This looks at task, time, and team metrics for figuring out if devs are doing their best work.

    [blog] Transformation Theater. Maybe this post makes you feel unsettled because you see yourself or your company in a few of these. That’s ok. The first step is recognizing it. Then, you can fix it!

    [article] An Introvert’s Guide to Visibility in the Workplace. Believe it or not, I’m introverted. This piece resonated with me, and reflects some things I do to get out of my comfort zone.

    [blog] GKE provides fully managed Kubernetes support for Elastic Cloud. If you’re going to run packaged software atop Kubernetes, try and pick a fully managed k8s with no node maintenance. I like that Elastic and Hashicorp now support GKE Autopilot.

    [article] What if we rotate pairs every day? What an interesting experiment! The ThoughtWorks folks tried a rapid rotation for their developer pairs, and shared their findings.

    [blog] The RedMonk Programming Language Rankings: January 2024. We might see these sorts of rankings change over time with the introduction of AI-generated code. For now, the top five are stable, but there’s movement below that.

    [article] Event Interception. Good post on a technique for modernizing a legacy system by intercepting application events and redirecting them to the new system.

    [blog] DZ BANK unlocks 70% toil savings and 90% cost savings with a Cloud Run-first approach. Serverless computing at a 150 year-old German bank? This is a good story about using modern tech to modernize and save time/money.

    ##

    Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:

  • Daily Reading List – March 7, 2024 (#273)

    Fifteen or so meetings today, and I’m still standing. It’s good that I like these people. Below, you’ll find some thought-provoking content on leadership, OSS, and disruption.

    [blog] Populating documentation templates using AI. Tom’s on a content heater right now. Here’s another fascinating post about using generative AI to get an initial draft of documentation.

    [article] 3 Ways Humility Can Undermine Your Leadership. You often hear about the importance of servant leadership, humility, etc. I’m a big believer in that. But this post offers a constructive lens on when it can go wrong.

    [blog] Creating Spring-based gRPC microservices managed by Prometheus and Grafana. This is a good deep dive that does exactly what the title says.

    [article] Applying the DevEx framework. If you care about developer experience and productive software teams, you’ll like this exploration of a particular framework.

    [article] Open Source Software Use Driven by Cost Cutting, Survey Says. Some new survey data shows that “access to innovation” and “modernizing the technology stack” aren’t the most popular reasons people use open source nowadays. It’s about cost.

    [blog] Data Reliability at Chick-fil-A. The CFA folks apply principles from Site Reliability Engineering to something they call Data Reliability Engineering. The focus of the post is on a homegrown tool they use to conduct data quality checks.

    [blog] Croissant: a metadata format for ML-ready datasets. Data is represented all sorts of ways when feeding it into pipelines for model training. This new format looks to standardize a bit, and already has some industry commitment.

    [blog] Risk-Aware Product Decisions in A/B Tests with Multiple Metrics. Lots of details in this post from Spotify engineers. Learn about their decision rules for taking results of experiments and using in product decisions.

    [blog] AI startups require new strategies: This time it’s actually different. Startups can’t use the same playbook of expecting incumbents to be slow with new tech or poaching talent when it comes to AI.

    ##

    Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:

  • Daily Reading List – March 6, 2024 (#272)

    Do you plan on going to many in-person tech events this year? If so, are you looking at local events or actually getting on a plane to go somewhere? It seems like folks are traveling more, but training budgets continue be tight across the industry. Even if you can’t go anywhere right now, I’ll try to bring the learning to you with these reading lists!

    [blog] Shifting Security Down Early. I didn’t just read and share this because it references my “shift down” metaphor. Doesn’t hurt, though. Aron offers up a good take on security in the dev lifecycle.

    [article] When New Hires Get Paid More, Top Performers Resign First. Interesting findings. If you level up existing staff salaries when new (highly paid) folks join, people stay. When you don’t, the high performers disproportionally leave.

    [blog] Running AI on fully managed GKE, now with new compute options, pricing and resource reservations. It seems like nothing in the world is getting cheaper, especially anything associated with AI. So hey, let’s rejoice that GPU workloads in Kubernetes are cheaper than before.

    [article] Legacy tech is still popping up as a cost-control barrier. Short piece, but a reminder that you can’t put off modernization for long and expect to keep successfully shoving new tech into your architecture.

    [blog] Reverse engineering prompts. This post brought me joy. I’ve never thought of asking an LLM for a prompt that would generate the output I want. I tried with Duet AI and got it to give me a really good prompt for an existing block of Go code. Love that idea as a way to learn good prompting.

    [blog] Secure by Design: Google’s Perspective on Memory Safety. You may be like us and moving towards memory-safe languages, but also in possession of piles of C or C++ code. Here’s what we’re considering.

    [blog] Improving Shopify App’s Performance. What’s the worst feedback you can hear about a distributed or complex app? “Feels slow.” Where do you even START? Shopify wanted to improve the performance of their app, and tackled three different performance bottlenecks.

    [article] Exploring the API of Google’s Gemini Language Model. Learn about the types of API parameters you find in LLMs like Gemini.

    [blog] “Bad” Strategy. Why? Have you said to yourself “my company has no real strategy”? COME ON SUITS UPSTAIRS, FIGURE IT OUT. But we don’t always see the whole picture, as John talks about here.

    ##

    Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:

  • Daily Reading List – March 5, 2024 (#271)

    Good day, and I’m heading out to meet a friend for tacos by the beach. No complaints! I really enjoyed the items I read today, and I think you will too.

    [blog] All you need is Wide Events, not “Metrics, Logs and Traces.” So good. Talk about a post that makes you rethink things! Ivan explains a key aspect of real observability.

    [blog] Cloud CISO Perspectives: Building better cyber defenses with AI. Digital security is ripe for positive impact from AI. The preamble in this post is an important look at what’s possible.

    [blog] Building a flexible platform for optimal use of LLMs. I think we’re going to see a LOT of this in 2024. Most every company will be building generative AI platforms for their teams to use. Here is what Intuit is doing.

    [blog] Go slices, deleting items, and memory usage. Did you think that so much went into creating a “delete” operation on an array type? I was fascinated by Val’s explanation of what he added to the Go standard library.

    [blog] Using Duet AI to generate a Starter App from an OpenAPI Specification. I like this. Maybe one of the issues in years past with a spec-first approach to APIs was that it wasn’t easy to implement that spec in code. With generative AI dev tooling, it is.

    [article] The open source problem. Matt says the big problems with OSS isn’t corporations using it without contributing. Rather, it’s that users/devs really only have one choice when licensing their creations.

    [article] Using AI to Help Developers Work with Regular Expressions. Are you good at writing regular expressions? My friends, I am not. This seems like a good scenario for AI assistance. This article looks at a few options at your disposal.

    [blog] Free data transfer out to internet when moving out of AWS. I’m glad we started this trend! The user wins. Love the cloud you’re in, but you shouldn’t be penalized on the way out.

    [article] This Is The Easiest Way To Make Your Life More Awesome. Eric’s point is that you can improve your life through subtraction. There may be stages of life where you ignore this for good reason, but you likely settle into this reality over the long term.

    [blog] Regional vs. zonal GKE clusters: making the right choice for your workloads. High availability is a choice in the public cloud. Sometimes it’s cheaper to be down then pay for extra resilience. Other times, downtime isn’t tolerable so you pay for better scale. This post explains some of the considerations.

    [article] NIST 800-207A: Implementing Zero Trust Architecture. Read, watch, or ask AI to summarize this content. It’s a good overview of what it takes to establish a zero trust architecture.

    [blog] Revolutionize Your CI/CD Pipeline: Integrating Testcontainers and Bazel. Here’s a deep walkthrough of two technologies that can help you produce self-contained build processes.

    ##

    Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:

  • Daily Reading List – March 4, 2024 (#270)

    Last Friday we had the day off, so I walked around our local zoo by myself. It was great! But I’m back in the saddle today, and this promises to be a very busy week. There’s always time for reading though, so please enjoy what I consumed today.

    [blog] Introducing the next generation of Claude. Anthropic has raised a ton of money, and they’re putting it to good use. Their new models have excellent performance. And we’ve already got one of those available in Vertex AI.

    [article] Add More Rigor to Your Reference Calls With These 25 Questions. This is a good list of questions you might pick from when checking in on external or internal job candidates.

    [docs] Configure networks for FedRAMP and DoD in Google Cloud. You might not need or want DoD-level of security in your network architecture, but it’s useful to see what it looks like for highly regulated workloads.

    [blog] The One Billion Row Challenge in Go: from 1m45s to 4s in nine solutions. Folks are still experimenting with how fast their language of choice can process a billion rows of data. This is an excellent review of multiple options using Go.

    [article] Apple Open Sources Pkl, a Configuration as Code Programming Language. Do we need another DSL for configurations? I dunno, but Apple seems to like this one.

    [article] How to Get Started with Google’s Gemini Large Language Model. Janakiram always does good tech deep dives, and this look at Gemini is easy enough to follow along with.

    [blog] 4 Reasons to Use Angular for Your Next Web App. Love it. There are good reasons, but I also appreciate the quiet confidence of those using something good, who aren’t personally crushed if you don’t make the same choice.

    [blog] The four tenets of SOA revisited. yes, tech is changing faster now than likely at any time in history. But there are plenty of durable ideas that stick around. Mark is starting a look at service oriented architecture principles.

    [article] “Good enough” code is just not good enough. There are times when “good enough” is ok, but continuously adopting that approach doesn’t yield excellence in the long run.

    ##

    Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:

  • No cloud account, no problem. Try out change streams in Cloud Spanner locally with a dozen-ish shell commands.

    No cloud account, no problem. Try out change streams in Cloud Spanner locally with a dozen-ish shell commands.

    If you have a choice, you should test software against the real thing. The second best option is to use a “fake” that implements the target service’s API. In the cloud, it’s straightforward to spin up a real instance of a service for testing. But there are reasons (e.g. cost or speed) or times (e.g. within a CI pipeline, or rapid testing on your local machine) when an emulator is a better bet.

    Let’s say that you wanted to try out Google Cloud Spanner, and it’s useful change streams functionality. Consider creating a real instance and experimenting, but you have an alternative option. The local emulator just added support for change streams, and you can test the whole thing out from the comfort of your own machine. Or, to make life even easier, test it out from a free cloud machine.

    With just a Google account (which most everyone has?), you can use a free cloud-based shell and code editor. Just go to shell.cloud.google.com. We’ve loaded this environment up with language CLIs for Java, .NET, Go, and others. It’s got the Docker daemon running. And it’s got our gcloud CLI pre-loaded and ready to go. It’s pretty cool. From here, we can install the Spanner emulator, and run just a few shell commands to see the entire thing in action.

    Let’s begin by installing the emulator for Cloud Spanner. It takes just one command.

    sudo apt-get install google-cloud-sdk-spanner-emulator
    

    Then we start up the emulator itself with this command:

    gcloud emulators spanner start 
    

    After a couple of seconds, I see the emulator running, and listening on two ports.

    Great. I want to leave that running while having the freedom to run more commands. It’s easy to spin up new tabs in the Cloud Shell Editor, so I created a new one.

    In this new tab, I ran a set of commands that configured the gcloud CLI to work locally with the emulator. The CLI supports the concept of multiple configurations, so we create one that is emulator friendly. Also note that Google Cloud has the idea of “projects.” But if you don’t have a Google Cloud account, you’re ok here. For the emulators, you can use a non-existent value for “project” as I have here.

    gcloud config configurations create emulator
    gcloud config set auth/disable_credentials true
    gcloud config set project local-project
    gcloud config set api_endpoint_overrides/spanner http://localhost:9020/
    

    It’s time to create a (local) Spanner instance. I ran this one command to do so. It’s super fast, which makes it great for CI pipeline scenarios. That second command sets the default instance name so that we don’t have to provide an instance value in subsequent commands.

    gcloud spanner instances create test-instance \
       --config=emulator-config --description="Test Instance" --nodes=1
    gcloud config set spanner/instance test-instance
    

    Now, we need a database in this instance. Spanner supports multiple “dialects”, including PostgreSQL. Here’s how I create a new database.

    gcloud spanner databases create example-db --database-dialect=POSTGRESQL
    

    Let’s throw a couple of tables into this database. We’ve got one for Singers, and one for Albums.

    gcloud spanner databases ddl update example-db \
    --ddl='CREATE TABLE Singers ( SingerId bigint NOT NULL, FirstName varchar(1024), LastName varchar(1024), SingerInfo bytea, PRIMARY KEY (SingerId) )'
    gcloud spanner databases ddl update example-db \
    --ddl='CREATE TABLE Albums ( SingerId bigint NOT NULL, AlbumId bigint NOT NULL, AlbumTitle varchar, PRIMARY KEY (SingerId, AlbumId) ) INTERLEAVE IN PARENT Singers ON DELETE CASCADE'
    

    Now we’ll insert a handful of rows into each table.

    gcloud spanner databases execute-sql example-db \
      --sql="INSERT INTO Singers (SingerId, FirstName, LastName) VALUES (1, 'Marc', 'Richards')"
    gcloud spanner databases execute-sql example-db \
      --sql="INSERT INTO Singers (SingerId, FirstName, LastName) VALUES (2, 'Catalina', 'Smith')"
    gcloud spanner databases execute-sql example-db   \
      --sql="INSERT INTO Singers (SingerId, FirstName, LastName) VALUES (3, 'Alice', 'Trentor')"
    gcloud spanner databases execute-sql example-db   \
      --sql="INSERT INTO Albums (SingerId, AlbumId, AlbumTitle) VALUES (1, 1, 'Total Junk')"
    gcloud spanner databases execute-sql example-db   \
      --sql="INSERT INTO Albums (SingerId, AlbumId, AlbumTitle) VALUES (2, 1, 'Green')"
    

    If you want to prove this works (thus far), you can execute regular queries against the new tables. Here’s an example of retrieving the albums.

    gcloud spanner databases execute-sql example-db \
        --sql='SELECT SingerId, AlbumId, AlbumTitle FROM Albums'
    

    It’s time to turn on change streams, and this takes an extra step. It doesn’t look like I can smuggle utility commands through the “execute-sql” operation, so we need to run a DDL statement instead. Note that you can create change streams that listen to specific tables or columns. This one listens to anything changing in any table.

    gcloud spanner databases ddl update example-db \
    --ddl='CREATE CHANGE STREAM EverythingStream FOR ALL'
    

    If you want to prove everything is in place, you can run this command to see all the database objects.

    gcloud spanner databases ddl describe example-db --instance=test-instance
    

    I’m now going to open a third tab in the Cloud Shell Editor. This is so that we can continuously tail the change stream results. We’ve created this nice little sample project that lets you tail the change stream. Install the app by running this command in the third tab.

    go install github.com/cloudspannerecosystem/spanner-change-streams-tail@latest
    

    Then, in this same tab, we want the Go SDK (which this app uses) to look at the local emulator’s gRPC port instead of the public cloud. Set the environment variable that overrides the default behavior.

    export SPANNER_EMULATOR_HOST=localhost:9010
    

    Awesome. Now we start up the change stream app with a single command. You should see it start up and hold waiting for data.

    spanner-change-streams-tail -p local-project -i test-instance -d example-db -s everythingstream
    

    Back in the second tab (the first should still be running the emulator, the third is running the change stream tail), let’s add a new record to the Spanner database table. What SHOULD happen is that we see a change record pop up in the third tab.

    gcloud spanner databases execute-sql example-db   \
      --sql="INSERT INTO Albums (SingerId, AlbumId, AlbumTitle) VALUES (2, 2, 'Go, Go, Go')"
    

    Sure enough, I see a record pop into the third tab showing the before and after values of the row.

    You can mess around with updating records, deleting records, and so on. A change stream is powerful for event sourcing scenarios, or simply feeding data changes to downstream systems.

    In this short walkthrough, we tried out the Cloud Shell Editor, spun up the Spanner emulator, and experimented with database change streams. All without needing a Google Cloud account, or installing a lick of software on our own device. Not bad!