[blog] Blog Writing for Developers. Struggling to share your written thoughts? This post targets those writing for developers, but can apply to most anyone.
[blog] 5 Ways to Detect Breaking Changes In Your APIs. Ideally, you detect a breaking change early on in your development process. That’s not always the case. Here are other places you can run tests to see if you’ve broken someone’s experience.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
At this time, exactly twenty three years ago, I was in a downtown Seattle skyscraper learning how to use a rough version of a new integration product from Microsoft called BizTalk Server. Along with 100+ others at this new consulting startup called Avanade (now, 50,000+ people), we were helping companies use Microsoft server products. From that point on at Avanade, through a stint at Microsoft and then a biotech company, I lived in the messaging/integration/ESB space. Even after I switched my primary attention to cloud a decade or so ago, I kept an eye on this market. It’s a very mature space, full of software products and as-a-service offerings. That’s why I was intrigued to see my colleagues at Google Cloud (note: not in my product area) ship a brand new service named Application Integration. I spent some time reading all the docs and building some samples, and formed some impressions. Here are the many things I like, and a few things I don’t like.
I LIKE the extremely obvious product naming. We don’t have a lot of whimsy or mystery in our Google Cloud product names. You can mostly infer what the service does from the name. You won’t find many Snowballs, Lightsails, or Fargates in this portfolio. The new service is for those doing application integration, as the name says.
I LIKE the rich set of triggers that kick off an integration. The Application Integration service is what Gartner calls an “integration platform as a service” and it takes advantage of other Google Cloud services instead of reinventing existing components. That means it doesn’t need to create its own messaging or operational layers. This gives us a couple of “free” triggers. Out of the box, Application Integration offers triggers for Pub/Sub, web requests (API), scheduler, Salesforce, and even some preview triggers for Jira, ZenDesk, and ServiceNow.
I LIKE the reasonable approach to data management. Any decent integration product needs to give you good options for defining, mapping, and transporting data. With Application Integrations, we work with “variables” that hold data. Variables can be global or local. See here how I explore the different data types including strings, arrays, and JSON.
The service also generates variables for you. If you connect to a database or a service like Google Cloud Storage, the service gives you typed objects that represent the input and output. Once you have variables, you can create data mappings. Here, I took an input variable and mapped the values to the values in the Cloud Storage variable.
There are some built-in functions to convert data types, extract values, and such. It’s a fairly basic interface, but functional for mappings that aren’t super complex.
I LIKE the strong list of tasks and control flow options. How do you actually do stuff in an integration? The answer: tasks. These are pre-canned activities that stitch together to build your process. The first type are “integration tasks” like data mapping (shown above), looping (for-each, do-while), sending-and-receiving user approval, sending emails, calling connectors, and more. This is on top of native support for forks and joins, along true/false conditions. You can do a lot with all these.
As you might expect (hope for? dream?), the service includes a handful of Google Cloud service tasks. Pull data from Firestore, invoke Translation AI services, list files in Google Drive, add data to a Google Sheet, call a Cloud Function, and more.
I LIKE the solid list of pre-built connectors. An iPaaS is really only as good as its connector library. Otherwise, it’s just a workflow tool. A good “connector” (or adapter) offers a standard interface for authentication, protocol translation, data handling, and more. Application Integration offers a good—not great—list of initial Google Cloud services, and an impressive set of third-party connectors. The Google Cloud ones are primarily databases (which makes sense), and the third party ones include popular systems like Active Directory, Cassandra, MongoDB, SAP HANA, Splunk, Stripe, Twilio, Workday and more. And through support for Private Service Connect, connectors can reach into private—even on-premises—endpoints.
I LIKE the extensibility options baked into the service. One of the complaints I’ve had with other iPaaS products is they initially offered constrained experiences. All you could use were the pre-built components which limited you to a fixed set of use cases. With Application Integration, I see a few smart ways the service lets me do my own thing:
JavaScript task. This “catch-all” tasks lets you run arbitrary JavaScript that might mess with variables, do more complex data transformations, or whatever else. It’s pretty slick that the code editor offers code completion and syntax highlighting.
Generic REST and HTTP call support. The service offers a task that invokes a generic REST endpoint—with surprisingly sophisticated configuration options—as well as a connector for a generic HTTP endpoint. This ensures that you can reach into a variety of systems that don’t have pre-built connectors.
Cloud Functions integration. We can debate whether you should ever embed business logic or code into maps or workflows. Ideally, all of that sits outside in components that you can independently version and manage. With Cloud Functions integration, that’s possible.
Build custom mappings using Jsonnet templates. The default mapping may not be the right choice for complex or big mappings. Fortunately you can define your own maps using a fairly standard approach.
I LIKE the post-development tools. I’ve occasionally seen “day 2” concerns left behind on the first release of an iPaaS. The focus is on the dev experience, with limited help for managing deployed resources. Not here. It’s coming out of the gate with good logging transparency:
It also offers a versioning concept so that you can not fear making changes and immediately having those changes in “production.”
The pre-defined monitoring dashboard is good and because it’s built on our Cloud Monitoring service, offers straightforward customization and powerful chart features.
I DON’T LIKE the explicit infrastructure feel of the connectors. With Application Integration, you explicit provision and become aware of connections. When creating a connection, you pick node pool sizes and wait for infrastructure to come online. This is likely good for predictable performance and security, but I’d rather this be hidden from the user!
I DON’T LIKE the lack of CI/CD options. Admittedly, this isn’t common for every mature iPaaS, but I’d like to see more turnkey ways to author, test, version, and deploy an integration using automation tools. I’m sure it’s coming, but not here yet.
All-in-all this, this is an impressive new service. The pricing is pay-as-you-go with a free tier, and seems reasonable overall. Would I recommend that you use this if you use NOTHING else from Google Cloud? I don’t think so. There are other very good, general purpose iPaaS products. But if you’re in our Cloud ecosystem, want easy access to our data and AI services from your integration workflows, then you should absolutely give this a look.
[blog] us-west-1: The Flagship AWS Region That Isn’t. With every cloud, all regions aren’t the same. AWS seems to have some large disparities on this one, but I do feel their pain. But, customers don’t care; consistency matters!
[blog] Gossip Protocol Explained. Detailed look at how distributed systems stay in sync. You might never directly implement one of these, but the products you depend on likely use one them.
[blog] 3 Reasons Why Teams Move Away from AWS Lambda. Fluidity among runtimes is important. I like the Google Cloud bet on containers because it helps you move between compute services more easily.
[blog] Bash basics. Here are some simple shell commands that are good to know. I’m always a fan of introductory content that helps folks get a strong foundation.
It was a light meeting day, so I had opportunities to get some writing done while also recording internal demos of soon-to-be-released products. Oddly, with all this “open” time today, I didn’t read as much as usual. But I think you’ll find an item or two below that piques your interest.
[article] Y’all Against My Lingo? Why Everyone Hates on YAML. I don’t have strong affection for any particular declarative format. After years of XML, JSON came along, now lots of folks rely on YAML. Is it the best? No. The worst? Nah.
[docs] Payload unwrapping for Pub/Sub push subscriptions. I really like this new feature of our messaging engine. Because we support push subscriptions, you can send messages to HTTP endpoints. Now, you have the option to strip the envelope and send the raw payload.
[blog] Cloud CISO Perspectives: Early July 2023. What does AI mean to the security field, and what’s new and exciting for security folks? Good edition of this twice-monthly brief.
[blog] Google Cloud Platform vs. AWS: A Cost Comparison Showdown. Nobody moves to an entirely new cloud purely for 5-10% cost savings, but if you have multiple in use, these price differences may have you bias towards one or the other.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
Yesterday, I said I was playing “offense” and tackling all sorts of activities. Today was a “defense” day where the goal was to keep my head above water until 5pm. Mission accomplished. Did you play offense or defense today? Even if you were stuck being reactive, be proactive now and read some good pieces below!
[blog] Selecting the right product metrics. Thought-provoking stuff from Jason that I need to chew on. It seems like a good way to consider the broader role of your product and who it impacts.
I start each day deciding whether I’m playing “offense” or “defense.” On defense days, my calendar is booked solid and I’m just aiming to survive meetings and keep my inbox from imploding. On offense days, I have time blocked off to create things, tackle complex tasks, and feel a sense of accomplishment. Today was offense. And digesting some good reading material, that you’ll find below.
[blog] Google’s AI Red Team: the ethical hackers making AI safer. Wow, this is pretty awesome. Google’s Red Team consists of hackers who help us better prepare for attacks. This new paper (no registration needed) helps you do red teaming to prepare for what’s coming against you AI-based systems.
[article] How platform teams get stuff done. How do people work together to deliver a shared platform? This article looks at a few patterns for platform teams.
[blog] Lines of Code. We say that “lines of code” is a lousy measure of developer output, but why do some vendors tout the “lines of code” generated by AI tools? Rachel has a quick, relevant post on the topic.
[blog] From MySQL to NoSQL: Bitly’s big move to Bigtable. Moving 40 billion rows is no joke. But the Bitly folks did it when moving from MySQL to Bigtable, and now they’re set up for limitless future growth.
[article] How Taking a Vacation Improves Your Well-Being. You hear people say (sometimes bosses) that “there’s never a good time for vacation.” Nonsense. It’s always a good time. Plan ahead and get away.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
So, so much AI stuff going on right now. Its not what matters most (see an article below on leadership development), or in its final consumption form (see an article below on abstractions), but it’s something to absolutely pay attention to.
[page] Introducing Llama 2. Meta is doing some cool work here. Check this out for an quasi-open, powerful LLM. Also take a look at the detailed paper about the model and its training process.
[blog] Why I’m Not Writing About Generative AI. This Forrester analyst mentions that it’s more about leadership development than any particular tech trend. Good point.
[blog] Abstractions. Brian offers up a strong take on the idea of software abstractions, and choosing the right ones for your architecture.
[blog] Teaching Programming in the Age of ChatGPT. What a wild time to be a computing instructor. This study shows that teachers are concerned about using AI-assisted tools for cheating, and that students won’t learn fundamentals. But they also see promise.
[blog] Adding HTTP around Wasm with Wagi. Here you’ll find more exploration of the Web Assembly ecosystem and how to serve up HTTP-based microservices.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
This past weekend I was reminded that the only control I have in life relates to how I react to what happens. I control virtually nothing else. Once we accept that, it changes our outlook. Check out some great content from today, including a heavy dose of AI material.
[blog] What makes developers productive? Terrific post that reminds us that “productivity” isn’t a checkbox in a product somewhere, but the result of a well-designed system that developers work within.
[article] Keeping Your Spirits Up When You Really Need a Win at Work. You’ve had those days, months, or years where it feels like you’ve felt beat up? It’s normal. This article looks at how to stay motivated and approach life when you need that win.
##
Want to get this update sent to you every day? Subscribe to my RSS feed or subscribe via email below:
I had a good time today talking to a prospective customer who already bets big on another hyperscaler. By the end of the chat, they seemed psyched about the handful of ways that Google Cloud is different and better. But hey, just use a cloud, any cloud. It’s better up here. Have a good weekend!
[blog] Business Advice Plagued by Survivor Bias. Are we just listening to the people who “made it” versus learning more lessons from those that failed along the way? Maybe so, and it should give us pause.
[blog] Traffic Routing in Cloud Run. This is a good look at how to route traffic to different revisions of your app. This matters if you’re doing gradual rollouts, or even rollbacks.
[blog] Govulncheck v1.0.0 is released! Even if your cloud isn’t super-secure, at least use a programming language that’s doubled-down on vulnerability management. Great work by the Go team.
[repo] Create a trace through Cloud Pub/Sub. If you’re tracing requests through your system, you don’t want to lose the part of the path that goes through systems like message brokers. This code sample shows how to keep it all together.
Richard Seroter's Architecture Musings 