Had a good Wednesday? I was able to write up a demo of cloud migration, do some real work, and read a few interesting items. Take a look at a handful of pieces below, many of which have a focus on technology tools.
[blog] Making Cloudflare the best place for your web applications. Cloudflare keep shipping interesting things. They sit in this fascinating in-between space nowadays with JAMstack-type vendors to one side, and hyperscale cloud platforms on the other.
[blog] AI-powered coding, free of charge with Colab. Millions of devs use Colab to program in Python for free. Now you’re getting some of the awesome AI assistance we announced last week.
[article] How to reduce your devops tool sprawl. Related to the above, tool sprawl is a real thing, and you should be aiming to consolidate in a few areas.
[blog] Deprecating an Open Source Project, Part 2. When is it time to put that product or open source project in the icebox? This post takes a look at how to handle an open source project that’s nearing its end.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Moving isn’t fun. At least not for me. Even if you can move from one place to another, there are plenty of things that add friction. In the public cloud, you might want to switch from your first cloud to your next one, but it just feels like a lot of work. And while we cloud vendors like to talk about flashy serverless/container compute options, let’s be honest, most companies have their important workloads running in virtual machines. So how do you move those VMs from one place to another without a ton of effort? I’m going to look at four of the options, including one we just shipped at Google Cloud.
Option #1 – Move the workload, not the VM
In this case, you take what was on the original VM, and install it onto a fresh instance in the next cloud. The VM doesn’t move, the workload does. Maybe you do move the software manually, or re-point your build system to a VM instance in the new cloud.
Why do this? It’s a clean start and might give you the opportunity to do that OS upgrade (or swap) you’ve been putting off. Or you could use this time to split up the websites on a stuff server into multiple servers. This is also the one option that’s mostly guaranteed to work regardless of where you’re coming from, and where you’re going to.
The downside? It’s the most work of any of these options. You’ve got to install software, move state around, reconfigure things. Even if you do automated deployments, there’s likely new work here to bake golden images or deploy to a new cloud.
Option #2 – Export the VM images from one cloud and import into the next one
All the major clouds (and software vendors) support exporting and importing a VM image. These images come in all sorts of formats (e.g. VMDK, VHDX).
Why do this? It gives you a portable artifact that you can bring to another cloud and deploy. It’s a standard approach, and gives you a manageable asset to catalog, secure, backup, and use wherever you want. AWS offers guidance, so does Azure, as does Google Cloud. This usually carries no explicit cost, but brings with it costs for storage of the assets.
The downsides? This too is manual, although can be automated with APIs. It also moves the entire VM image without an opportunity to shrink or modernize any aspect of it. Additionally, it usually requires extra configuration of storage buckets and permissions to store the temporary artifacts.
Option #3 – Convert the VM to a container and move that artifact to the new cloud
Another way to move a VM to another cloud is to extract the VM-based application to a container image. The workload moves, but in a different format. All the major public clouds have something here. Azure Migrate helps with this, AWS provides an App2Container CLI tool, and Google Cloud offers Migrate to Containers as a CLI and UI-based experience.
So, what do you do with those existing .NET Framework apps on Windows? Most advice is "upgrade to #dotnet Core." But I hear that many folks want to freeze on .NET 4.8 for now.
Why do this? This offers a means of “shrinking” the workload by reducing it to its own components, without bringing along the OS with it. This can bring higher workload density in the target cloud (if you throw a bunch of app containers onto consolidated hardware) and reduce cost. Also, this gives you flexibility on where you run the workload next. For instance, the container image you generate from the Google Cloud tool can run on a Kubernetes cluster or serverless Cloud Run environment.
Downsides? This doesn’t work for all workload types. Don’t shove SharePoint into a container, for example. And not all tools work with all the various clouds, so you might have to move the VM manually and then run the containerization tool. Also, doing this may give the impression you’re modernizing the app, but in reality, you’re only modernizing the underlying platform. That is valuable, but doesn’t remove the need for other modernization activities.
Option #4 – Use a managed service that moves the VM and turns down the old instance
Can migration be easier? Can you move VMs around with fewer steps and moving parts? There are definitely solutions for this from a variety of vendors. Among cloud providers, what Google Cloud has is unique. We just added a new experience, and figured we could walk through it together.
First, I built an Amazon EC2 instance and installed a web server onto it. I added a custom tag with the key “type” and value “web-server” so that I could easily find this VM later. I also added two total volumes in order to see if they successfully move alongside the VM itself.
After a few moments, I had my EC2 instance up and running.
Let’s fast forward for a period of time, and maybe it’s time to evolve and pick my next cloud. I chose Google Cloud, WHICH MUST SHOCK YOU. This workload needs a happier home.
The new Migrate to Virtual Machines experience in the Google Cloud console is pretty sweet. From here, I can add migration sources, target projects, create groups of VMs for migration, and monitor the progress.
First, I needed to create a source. We recently added AWS as a built-in option. We’ve supported VMware-based migrations for a while now.
I created the “AWS source” by giving it a name, choosing the source AWS region, the target Google Cloud region, and providing credentials to access my account. Also note that I added an (optional) tag to search for when retrieving instances, and an (optional) tag for the migrated VMs.
My connection was in a “pending” state for a couple of minutes, and after that, showed me a list of VMs that met the criteria (AWS region, tag). Pretty cool.
From here, I chose that VM and picked the option to “add migration.” This added this particular VM into a migration set. Now I could set the “target” details of the VM in Google Cloud Compute Engine that this AWS image loads into. That means the desired machine name, machine type, network, subnet, and such.
I started the migration. Note that I did not have to stop the VM on AWS for this migration to commence.
When it’s done replicating, I don’t yet have a running VM. My last major step is choosing to do a test-clone phase where I test my app before making it “live”, or, jump right to cut-over. In cut-over, the services takes a final data replica, stops the original VM, and makes a Compute Engine instance using the replicated data.
After a few more minutes, I saw a running Google Cloud Compute Engine VM, and a stopped EC2 instance.
I “finalized” the migration to clean up all the temporary data replicas and the like. After not being sure if this migration experience grabbed the secondary disks from my EC2 instance, I confirmed that yes, we brought them all over. Very nice!
Why do this? The Migrate to Virtual Machines experience offers a clean way to move one or multiple VMs from AWS, vSphere, or Azure (preview) to Google Cloud. There’s very little that you have to do yourself. And I like that it handles the shut down of the initial VM, and offers ways to pause and resume the migration.
The downsides? It’s specific to Google Cloud as a target. You’re not using this to move workloads out of Google Cloud. It’s also not yet available in every single Google Cloud region, but will be soon.
What did I miss? How do you prefer to move your VMs or VM-based workloads around?
I can across some good thinking and analysis today. Check out how Spotify and Chick-fil-A manage their fleets of infrastructure, and what Gartner sees shaping the future of cloud.
[blog] Stumbling Into GitOps at the Edge. Good story from the Chick-fil-A team that was an early adopter of a GitOps approach for distributed deployment of apps.
[blog] This Week in Spring – May 16th 2023. I still use Spring Boot when I get the chance, and I like some of the integrations coming in Spring Boot 3.1.
[article] When Your Employee Tells You They’re Burned Out. Don’t tell your team to “tough it out” or “this is what you’re paid to do” when they tell you they’re wiped. Here’s a good article on acknowledging and addressing burnout.
[article] DevEx, a New Metrics Framework From the Authors of SPACE. If you’ve struggled to measure (dev) productivity, you’re not alone. It’s hard and fraught with fake metrics. here’s one look at a new way to assess dev experience.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Today’s reading list included some good content on “boring” platforms like RabbitMQ and Kubernetes, along with product announcements and a hot take on multicloud.
[article] RabbitMQ Is Boring, and I Love It. Your messaging middleware just needs to work. No drama. RabbitMQ is a great choice, as are the first-party cloud message brokers. Boring is good.
[article] Nobl9 Adds Tools to Make SLO Queries More Efficient. Does your team create Service Level Objectives and manage to those? Nobl9 does good work here, and they’re making some powerful improvements to their stack.
[article] Sigstore: Roots of trust for software artifacts. This project may be under-the-radar for you, but it’s something you should probably be aware of. Trust in your software artifacts is key. Sigstore now plays a big role.
[blog] Choosing the right MongoDB type on Google Cloud. Run open source software yourself, or use a managed provider? Both are valid choices, although I tend towards managed products versus DIY.
It’s the end of a wild week. I read a number of pieces today, many around AI. There’s also some good stuff around app modernization, remote work, and GitOps. Enjoy your weekend!
[blog] AI isn’t the app, it’s the UI. Agree with the macro point here that AI can serve as a powerful bridge between the user and the machine. That’s what we did with Duet AI for Google Cloud.
[article] 4 Core Principles of GitOps. Are you following a GitOps-style approach to define a desired state for your infrastructure and keeping that infrastructure consistently configured? This post calls out four core principles.
[article] Is Your Remote Job Making You Lonely? Working remotely can be awesome. It’s definitely something I enjoy at times. But the isolation can be tough. Here’s some quick guidance on addressing it.
[blog] 100 things we announced at I/O 2023. Nice recap here. There aren’t many (any?) events quite like I/O given the breadth of topics we cover.
Back from I/O, and still recovering. What a fun show. I read some good items today, including content on frontend development, perf testing serverless, and what the media thought of our generative AI announcements.
[article] How to Disagree Productively. People at work disagree all the time; it’s just whether they’re vocal about it or not. Here’s a good article on how to do it productively
[blog] Trapped by Technology Fallacies. Good post by my colleague Michele. She tackles 3 fallacies that many folks tend to believe on the vendor and enterprise side.
[blog] Go Developer Survey 2023 Q1 Results. Almost 6,000 Go devs replied to this survey and showed what tools they use, what apps they build, and what challenges they face.
[article] Google launches a GitHub Copilot competitor. An interview with me in TechCrunch, focused on Duet AI for Google Cloud and delivering the right type of experience for a modern cloud.
[blog] Multi-Cluster Services on GKE. Our GKE team has done good work simplifying the experience of connecting Kubernetes services together. Check out this walkthrough.
Light content day, as I spent most of today in Mountain View attending Google I/O. Four hours of keynotes later, and I’m still processing everything I heard. I delivered a keynote (which you can watch below) and will have more I/O content to read (and share) in the coming days.
85 issues in, let’s change the name a bit. Since this little bulletin is a recap of what I read in a given day—not a general “wrap up” of daily news—the name “daily reading list” seems more accurate.
Today, I read through some useful survey data, announcements about generative AI, and good material on continuous delivery.
[article] CDF Report Reveals Lack of DevOps Maturity. I freely admit that I live in a tech bubble, but these numbers still surprised me. We have a long ways to go on fundamentals like CI/CD, let alone failure recovery.
[blog] Untangling the many aspects of EDA. The modern architect and developer should be aware of “event driven architecture” and what it entails. This post describes some of the main ideas.
[blog] Connecting GitHub Actions and Google Cloud Deploy. Just because you CAN use a particular service for many things, doesn’t mean you should. If you want to tack on a proper continuous deployment service to your GitHub Actions, consider Cloud Deploy.
I didn’t work this past weekend, and that was unusual. And awesome. Today, back in the saddle and read lots of great material. Check out some survey results, guidance for Go developers, and a handful of pieces on the Amazon Prime microservices/monolith dust-up.
[article] Survey: Much Greater Focus on Reining in IT Costs. Database costs at the top of that list. Which is interesting, given you only have SO many levers to pull short of deleting data, or making a major shift in human costs.
[blog] The revenge of the monolith. Great take from Brian on the above topic. Are microservices done? Are monoliths cool again? Read this for reasonable guidance.
[article] Banking on Thousands of Microservices. When microservices get used in the right situation, they add value. This deep dive into financial firm Monzo’s cloud architecture shows why they did it, and the value they get from it.
[blog] Introducing rules_oci. If you use Bazel for building containers, or are looking for a scalable tool to do so, you’ll like this new ability to make secure builds easier.
[blog] The Illusion of Choice : A Review. Does guidance for sales and marketing apply to your security and risk management program? This book review does a good job doing just that.
I hope you week ended on a high note. If not, maybe this little bundle of interesting reads will get your weekend off to a good start. Learn some things about AI, “private” serverless apps, zero-trust, and more.
[blog] Are We Making Lambda Too Hard? Are you possibly over-thinking your serverless architecture and some classic bloat to the structure? This post encourages a focus on the basics.
[blog] Using Cloud Spanner to handle high throughput writes. One reason that cloud (managed) databases are powerful is their flexibility to handle bursts. This posts explores how to support heavy-write situations.
Richard Seroter's Architecture Musings 