Did you have a good weekend? Summer is still in full swing here in San Diego and I was able to enjoy some nice weather. At work, we’re running full-speed ahead towards Google Cloud Next, which should be a terrific show. Sign up at least for the online portion!
[blog] Building a Successful SRE Team. Good advice to make sure your SRE team doesn’t become some ivory-tower group that nobody listens to.
[article] Mastering the Art of the Request. Are you good at asking for what you want? Does it work? This article has a few helpful tips to have more impact.
[report] August 2023 Threat Horizons Report. The Google Cybersecurity Action Team (great name) shipped their latest report which shared their latest intelligence on threats. Related to the story above.
[site] TIOBE Index for August 2023. Most popular programming languages? Some big changes from last August with C++, JavaScript, and Go up. Java, VB, and Swift down. And Julia in the top 20.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Quiet reading day as I churned through some writing activities and cleared off my to-do list. Have a great weekend!
[article] Exploring Generative AI. This post has a short but useful look at when in-line coding assistance gets in the way of your work.
[article] The Value of Digital Transformation. The Harvard Biz Review article has some data and perspectives about digital leaders outperforming laggards in their business goals.
I like working from the close-by Google office a few days a week. Today was a work from home day, and it was nice to take short breaks to mess with the kids enjoying Summer break. Helping a kiddo with a puzzle for five minutes is a good reset from work madness!
[blog] Why Open Source Matters. Does it matter if Meta misuses the term open source? Or if companies invent new types of licenses? To most of us, it’s white noise. But the definitions do matter, as Stephen expertly explains here.
[blog] Technical Feasibility in Software Development. When deciding on whether to do a project or not, are you consider all the dimensions? Technical feasibility? Schedule feasibility? Operational feasibility?
[blog] Your failures are yours. We don’t like to admit this, but we’re better off if we own our failings and learn from them.
[page] SRE Checklist. Whether you’re doing SRE or not, you might find usefulness in this checklist of processes, production requirements, and more.
[blog] How To Measure the Value of Internal Tools. I’d bet that most of you have homegrown tools that help you do your job better. What’s it worth? What does it cost? Square Engineering shares their thinking on it.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Ending the day at Inbox Zero, so that’s a plus. At least until I drive home from work and notice a dozen new emails. But we take the victories where we can get them, right? I read a handful of items today, and even had a chance to try out the new Go templates called out below.
[blog] The Transformer architecture of GPT models. This is more like a paper than a blog post. It explains the Google-created Transformer architecture in depth. Skim through for a perspective on encoders, embeddings, and more.
[blog] Cloud CISO Perspectives: Late July 2023. Good set of links related to security happenings, along with an intro that covers how cloud can help security and resilience in healthcare settings.
[blog] Accelerating the mission with generative AI. We’re doing some wonderful work with AI in the consumer space, as well as redefining AI in the enterprise as well. Here’s a look at public sector scenarios.
[blog] How to create meeting notes in Google Docs. I didn’t know about this. It’s these sorts of things that come out of an online, collaboration-first productivity suite.
[blog] In command. You can’t be in control of everything. Or frankly much at all. But you can be in command of yourself and how you approach things.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Sheesh, August 1st! I know folks tend to say “this year is flying by” but I’ve found ways to slow it down. Specifically, last year I started a routine where at the beginning of every day, I write a quick log of what I did the day before. That four minutes of daily reflection really helped me keep the days from all blurring together.
[blog] How TDD Seems To Go Against Agility. Are you “agile” if you’re planning a bunch of upfront tests for your code? This post looks at that concern, and addresses it from a different perspective.
[docs] Aggregate infrastructure availability. The availability of your application depends on the aggregate availability of each component in your stack. These updated Google Cloud docs explain this well.
[blog] Saying No In Open Source. This likely applies to plenty of scenarios beyond open source software, but there are unique challenges in managing something with so many random contributors.
After a work-week (and weekend) in New York City, it’s good to be back to my normal routines. Block off a few minutes to read some good content below, and give a listen to the short podcast episode I linked to.
[blog] Creating a developer portal for Cloud Run with sidecar containers and Apigee. I like the sidecar pattern when you don’t want to stuff a bunch of infrastructure code or cross-cutting services into your application. Now that you can apply this pattern to our serverless stack, you get some fresh architectural options.
[podcast] The Best Leaders Are Also Technical Experts. VERY interesting perspective, that deeply resonates with me. I want to be led by those who have domain expertise, and want to be a leader my team can trust to know their work area.
[blog] How to grow your multicloud resume as a cloud architect. You won’t be an expert at multiple clouds. Honestly, I don’t trust people who say they’re an expert at even one cloud! But there are areas to develop your skills if you want to be more successful across cloud providers.
[blog] Backpressure controls in Google Cloud PubSub. Sometimes messaging solutions can process more messages than the downstream system can handle. Look for services that have built-in back-pressure controls.
[article] AWS’s search for AI momentum. It’s so fascinating to watch AWS stumble around a bit here. They’ll end up fine, but are clearly disadvantaged at the moment.
Wrapping up an excellent work-week in New York City, and taking tomorrow off to be a tourist. I’ll be back here on Monday with fresh links, and enjoy this batch in the meantime!
[blog] All you need to know about timeouts. Why should you set service timeouts, and what’s the right value? The tech team at retailer Zalando digs in via this post.
[blog] Machine Learning Communities: Q2 ‘23 highlights and achievements. The Google developer community does remarkable work, and their output in Q2 of 2023 is worthy of your attention. You’ll find video links, GitHub repos, and written content with creative and informative AI/ML perspectives.
[blog] Always be prepared. Brief, but useful perspective from the McDonalds team that explains their disaster recovery practice.
[blog] Is full stack product management a good idea? I want to see a blog post that outlines all the tech roles, and what “full stack” looks like for each. What’s a full stack architect? Full stack business analyst? Full stack project manager? Here, a look at full stack PMs.
[blog] Kubeflow joins the CNCF family. If you’re looking to do ML processing on Kubernetes, you might be considering Kubeflow. It offers a solid end to end stack, and now is more broadly supported in the Cloud Native Computing Foundation ecosystem.
[blog] Announcing OverflowAI. I’ve seen some vigorous debate on what generative AI means to Q&A sites and those creating source material for these big AI models. StackOverflow is working to reinvent itself a bit.
[blog] TensorFlow on GKE Autopilot with GPU acceleration. Can you get GPUs when your app needs them, without having to plan for it during Kubernetes cluster creation? Yes, with Autopilot, we provision and scale clusters on demand with you worrying about it. Good demo here.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
It was another enjoyable, busy day with meetings in NYC. I’m very grateful to work with such smart folks who are apparently as competitive and eager to win as I am. The content I read today spanned topics, and I’d bet you find one or two things you like.
[blog] Common pitfalls in Go benchmarking. Eli points out where your benchmarking test can go wrong, and these pitfalls should apply to most any compiled language.
[blog] PaLM API Firebase Extensions Tutorial. I’m not sure how you would add “text summarization” to a site or app in years past, but generative AI makes it fairly simple now.
[blog] Should you job hop every ~2 years? Staying at a company for nearly 20 years isn’t super common in the tech space. Brian’s done it, and has thoughts on when to stay and when to go.
[blog] Blog Writing for Developers. Struggling to share your written thoughts? This post targets those writing for developers, but can apply to most anyone.
[blog] 5 Ways to Detect Breaking Changes In Your APIs. Ideally, you detect a breaking change early on in your development process. That’s not always the case. Here are other places you can run tests to see if you’ve broken someone’s experience.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
At this time, exactly twenty three years ago, I was in a downtown Seattle skyscraper learning how to use a rough version of a new integration product from Microsoft called BizTalk Server. Along with 100+ others at this new consulting startup called Avanade (now, 50,000+ people), we were helping companies use Microsoft server products. From that point on at Avanade, through a stint at Microsoft and then a biotech company, I lived in the messaging/integration/ESB space. Even after I switched my primary attention to cloud a decade or so ago, I kept an eye on this market. It’s a very mature space, full of software products and as-a-service offerings. That’s why I was intrigued to see my colleagues at Google Cloud (note: not in my product area) ship a brand new service named Application Integration. I spent some time reading all the docs and building some samples, and formed some impressions. Here are the many things I like, and a few things I don’t like.
I LIKE the extremely obvious product naming. We don’t have a lot of whimsy or mystery in our Google Cloud product names. You can mostly infer what the service does from the name. You won’t find many Snowballs, Lightsails, or Fargates in this portfolio. The new service is for those doing application integration, as the name says.
I LIKE the rich set of triggers that kick off an integration. The Application Integration service is what Gartner calls an “integration platform as a service” and it takes advantage of other Google Cloud services instead of reinventing existing components. That means it doesn’t need to create its own messaging or operational layers. This gives us a couple of “free” triggers. Out of the box, Application Integration offers triggers for Pub/Sub, web requests (API), scheduler, Salesforce, and even some preview triggers for Jira, ZenDesk, and ServiceNow.
I LIKE the reasonable approach to data management. Any decent integration product needs to give you good options for defining, mapping, and transporting data. With Application Integrations, we work with “variables” that hold data. Variables can be global or local. See here how I explore the different data types including strings, arrays, and JSON.
The service also generates variables for you. If you connect to a database or a service like Google Cloud Storage, the service gives you typed objects that represent the input and output. Once you have variables, you can create data mappings. Here, I took an input variable and mapped the values to the values in the Cloud Storage variable.
There are some built-in functions to convert data types, extract values, and such. It’s a fairly basic interface, but functional for mappings that aren’t super complex.
I LIKE the strong list of tasks and control flow options. How do you actually do stuff in an integration? The answer: tasks. These are pre-canned activities that stitch together to build your process. The first type are “integration tasks” like data mapping (shown above), looping (for-each, do-while), sending-and-receiving user approval, sending emails, calling connectors, and more. This is on top of native support for forks and joins, along true/false conditions. You can do a lot with all these.
As you might expect (hope for? dream?), the service includes a handful of Google Cloud service tasks. Pull data from Firestore, invoke Translation AI services, list files in Google Drive, add data to a Google Sheet, call a Cloud Function, and more.
I LIKE the solid list of pre-built connectors. An iPaaS is really only as good as its connector library. Otherwise, it’s just a workflow tool. A good “connector” (or adapter) offers a standard interface for authentication, protocol translation, data handling, and more. Application Integration offers a good—not great—list of initial Google Cloud services, and an impressive set of third-party connectors. The Google Cloud ones are primarily databases (which makes sense), and the third party ones include popular systems like Active Directory, Cassandra, MongoDB, SAP HANA, Splunk, Stripe, Twilio, Workday and more. And through support for Private Service Connect, connectors can reach into private—even on-premises—endpoints.
I LIKE the extensibility options baked into the service. One of the complaints I’ve had with other iPaaS products is they initially offered constrained experiences. All you could use were the pre-built components which limited you to a fixed set of use cases. With Application Integration, I see a few smart ways the service lets me do my own thing:
JavaScript task. This “catch-all” tasks lets you run arbitrary JavaScript that might mess with variables, do more complex data transformations, or whatever else. It’s pretty slick that the code editor offers code completion and syntax highlighting.
Generic REST and HTTP call support. The service offers a task that invokes a generic REST endpoint—with surprisingly sophisticated configuration options—as well as a connector for a generic HTTP endpoint. This ensures that you can reach into a variety of systems that don’t have pre-built connectors.
Cloud Functions integration. We can debate whether you should ever embed business logic or code into maps or workflows. Ideally, all of that sits outside in components that you can independently version and manage. With Cloud Functions integration, that’s possible.
Build custom mappings using Jsonnet templates. The default mapping may not be the right choice for complex or big mappings. Fortunately you can define your own maps using a fairly standard approach.
I LIKE the post-development tools. I’ve occasionally seen “day 2” concerns left behind on the first release of an iPaaS. The focus is on the dev experience, with limited help for managing deployed resources. Not here. It’s coming out of the gate with good logging transparency:
It also offers a versioning concept so that you can not fear making changes and immediately having those changes in “production.”
The pre-defined monitoring dashboard is good and because it’s built on our Cloud Monitoring service, offers straightforward customization and powerful chart features.
I DON’T LIKE the explicit infrastructure feel of the connectors. With Application Integration, you explicit provision and become aware of connections. When creating a connection, you pick node pool sizes and wait for infrastructure to come online. This is likely good for predictable performance and security, but I’d rather this be hidden from the user!
I DON’T LIKE the lack of CI/CD options. Admittedly, this isn’t common for every mature iPaaS, but I’d like to see more turnkey ways to author, test, version, and deploy an integration using automation tools. I’m sure it’s coming, but not here yet.
All-in-all this, this is an impressive new service. The pricing is pay-as-you-go with a free tier, and seems reasonable overall. Would I recommend that you use this if you use NOTHING else from Google Cloud? I don’t think so. There are other very good, general purpose iPaaS products. But if you’re in our Cloud ecosystem, want easy access to our data and AI services from your integration workflows, then you should absolutely give this a look.
Richard Seroter's Architecture Musings 