Category: DevOps

Take a fresh look at Cloud Foundry? In 20 minutes we’ll get Tanzu Application Service for Kubernetes running on your machine.
It’s been nine years since I first tried out Cloud Foundry, and it remains my favorite app platform. It runs all kinds of apps, has a nice dev UX for deploying and managing software, and doesn’t force me to muck with infrastructure. The VMware team keeps shipping releases (another today) of the most popular packaging of Cloud Foundry, Tanzu Application Service (TAS). One knock against Cloud Foundry has been its weight—in typically runs on dozens of VMs. Others have commented on its use of open-source, but not widely-used, components like BOSH, the Diego scheduler, and more. I think there are good justifications for its size and choice of plumbing components, but I’m not here to debate that. Rather, I want to look at what’s next. The new Tanzu Application Service (TAS) for Kubernetes (now in beta) eliminates those prior concerns with Cloud Foundry, and just maybe, leapfrogs other platforms by delivering the dev UX you like, with the underlying components—things like Kubernetes, Cluster API, Istio, Envoy, fluentd, and kpack—you want. Let me show you.

TAS runs on any Kubernetes cluster: on-premises or in the cloud, VM-based or a managed service, VMware-provided or delivered by others. It’s based on the OSS Cloud Foundry for Kubernetes project, and available for beta download with a free (no strings attached) Tanzu Network account. You can follow along with me in this post, and in just a few minutes, have a fully working app platform that accepts containers or source code and wires it all up for you.

Step 1 – Download and Start Stuff (5 minutes)

Let’s get started. Some of these initial steps will go away post-beta as the install process gets polished up. But we’re brave explorers, and like trying things in their gritty, early stages, right?

First, we need a Kubernetes. That’s the first big change for Cloud Foundry and TAS. Instead of pointing it at any empty IaaS and using BOSH to create VMs, Cloud Foundry now supports bring-your-own-Kubernetes. I’m going to use Minikube for this example. You can use KinD, or any other number of options.

Install kubectl (to interact with the Kubernetes cluster), and then install Minikube. Ensure you have a recent version of Minikube, as we’re using the Docker driver for better performance. With Minikube installed, execute the following command to build out our single-node cluster. TAS for Kubernetes is happiest running on a generously-sized cluster.
```
minikube start --cpus=4 --memory=8g --kubernetes-version=1.15.7 --driver=docker
```
After a minute or two, you’ll have a hungry Kubernetes cluster running, just waiting for workloads.

We also need a few command line tools to get TAS installed. These tools, all open source, do things like YAML templating, image building, and deploying things like Cloud Foundry as an “app” to Kubernetes. Install the lightweight kapp, klbd, and ytt tools using these simple instructions.

You also need the Cloud Foundry command line tool. This is for interacting with the environment, deploying apps, etc. This same CLI works against a VM-based Cloud Foundry, or Kubernetes-based one. You can download the latest version via your favorite package manager or directly.

Finally, you’ll want to install the BOSH CLI. Wait a second, you say, didn’t you say BOSH wasn’t part of this? Am I just a filthy liar? First off, no name calling, you bastards. Secondly, no, you don’t need to use BOSH, but the CLI itself helps generate some configuration values we’ll use in a moment. You can download the BOSH CLI via your favorite package manager, or grab it from the Tanzu Network. Install via the instructions here.

With that, we’re done the environmental setup.

Step 2 – Generate Stuff (2 minute)

This is quick and easy. Download the 844KB TAS for Kubernetes bundle from the Tanzu Network.

I downloaded the archive to my desktop, unpacked it, and renamed the folder “tanzu-application-service.” Create a sibling folder named “configuration-values.”

Now we’re going to create the configuration file. Run the following command in your console, which should be pointed at the tanzu-application-service directory. The first quoted value is the domain. For my local instance, this value is vcap.me. When running this in a “real” environment, this value is the DNS name associated with your cluster and ingress point. The output of this command is a new file in the configuration-values folder.
```
./bin/generate-values.sh -d "vcap.me" > ../configuration-values/deployment-values.yml
```
After a couple of seconds, we have an impressive-looking YAML file with passwords, certificates, and all sorts of delightful things.

We’re nearly done. Our TAS environment won’t just run containers; it will also use kpack and Cloud Native Buildpacks to generate secure container images from source code. That means we need a registry for stashing generated images. You can use most any one you want. I’m going to use Docker Hub. Thus, the final configuration values we need are appended to the above file. First, we need the credentials to the Tanzu Network for retrieving platform images, and secondly, credentials for container registry.

With our credentials in hand, add them to the very bottom of the file. Indentation matters, this is YAML after all, so ensure you’ve got it lined up right.

The last thing? There’s a file that instructs the installation to create a cluster IP ingress point versus a Kubernetes load balancer resource. For Minikube (and in public cloud Kubernetes-as-a-Service environments) I want the load balancer. So, within the tanzu-application-service folder, move the replace-loadbalancer-with-clusterip.yaml file from the custom-overlays folder to the config-optional folder.

Finally, to be safe, I created a copy of this remove-resource-requirements.yml file and put it in the custom-overlays folder. It relaxes some of the resource expectations for the cluster. You may not need it, but I saw CPU exhaustion issues pop up when I didn’t use it.

All finished. Let’s deploy this rascal.

Step 3 – Deploy Stuff (10 minutes)

Deploying TAS to Kubernetes takes 5-9 minutes. With your console pointed at the tanzu-application-service directory, run this command:
```
./bin/install-tas.sh ../configuration-values
```
There’s a live read-out of progress, and you can also keep checking the Kubernetes environment to see the pods inflate. Tools like k9s make it easy to keep an eye on what’s happening. Notice the Istio components, and some familiar Cloud Foundry pieces. Observe that the entire Cloud Foundry control plane is containerized here—no VMs anywhere to be seen.

While this is still installing, let’s open up the Minikube tunnel to expose the LoadBalancer service our ingress gateway needs. Do this in a separate console window, as its a blocking call. Note that the installation can’t complete until you do it!
```
minikube tunnel
```
After a few minutes, we’re ready to deploy workloads.

Step 4 – Test Stuff (3 minutes)

We now have a full-featured Tanzu Application Service up and running. Neat. Let’s try a few things. First, we need to point the Cloud Foundry CLI at our environment.
```
cf api --skip-ssl-validation https://api.vcap.me
```
Great. Next, we log in, using generated cf_admin_password from the deployment-values.yaml file.
```
cf auth admin <password>
```
After that, we’ll enable containers in the environment.
```
cf enable-feature-flag diego_docker
```
Finally, we set up a tenant. Cloud Foundry natively supports isolation between tenants. Here, I set up an organization, and within that organization, a “space.” Finally, I tell the Cloud Foundry CLI that we’re working with apps in that particular org and space.
```
cf create-org seroter-org
cf create-space -o seroter-org dev-space
cf target -o seroter-org -s dev-space
```
Let’s do something easy, first. Push a previously-containerized app. Here’s one from my Docker Hub, but it can be anything you want.
```
cf push demo-app -o rseroter/simple-k8s-app-kpack
```
After you enter that command, 15 seconds later you have a hosted, routable app. The URL is presented in the Cloud Foundry CLI.

How about something more interesting? TAS for Kubernetes supports a variety of buildpacks. These buildpacks detect the language of your app, and then assemble a container image for you. Right now, the platform builds Java, .NET Core, Go, and Node.js apps. To make life simple, clone this sample Node app to your machine. Navigate your console to that folder, and simple enter cf push.

After a minute or so, you end up with a container image in whatever registry you specified (for me, Docker Hub), and a running app.

This beta release of TAS for Kubernetes also supports commands around log streaming (e.g. cf logs cf-nodejs), connecting to backing services like databases, and more. And yes, even the simple, yet powerful, cf scale command works to expand and contract pod instances.

It’s simple to uninstall the entire TAS environment from your Kubernetes cluster with a single command:
```
kapp delete -a cf
```
Thanks for trying this out with me! If you only read along, and want to try it yourself later, read the docs, download the bits, and let me know how it goes.
April 14, 2020

I’ve noticed three types of serverless compute platforms. Let’s deploy something to each.

Are all serverless compute platforms—typically labeled Function-as-a-Service—the same? Sort of. They all offer scale-to-zero compute triggered by events and billed based on consumed resources. But I haven’t appreciated the nuances of these offerings, until now. Last week, Laurence Hecht did great work analyzing the latest CNCF survey data. It revealed which serverless (compute) offerings have the most usage. To be clear, this is about compute, not databases, API gateways, workflow services, queueing, or any other managed services.

Serverless usage rose from 37% in 2018 to 46% in 2019. Waiting for the 2020 study, but IMHO don't significant breadth of growth (adoption %). Instead, growth will be in terms of depth — # of workloads, # of app components. https://t.co/gjgF30YqNp pic.twitter.com/Izcmcdkb7L
— Lawrence Hecht (@LawrenceHecht) March 27, 2020

To me, the software in that list falls into one of three categories: connective compute, platform expanding, and full stack apps. Depending on what you want to accomplish, one may be better than the others. Let’s look at those three categories, see which platforms fall into each one, and see an example in action.

Category 1: Connective Compute

Trigger / Destination	Signature	Packaging	Deployment
Database, storage, message queue, API Gateway, CDN, Monitoring service	Handlers with specific parameters	ZIP archive, containers	Web portal, CLI, CI/CD pipelines

The best functions are small functions that fill the gaps between managed services. This category is filled with products like AWS Lambda, Microsoft Azure Functions, Google Cloud Functions, Alibaba Cloud Functions, and more. These functions are triggered when something happens in another managed service—think of database table changes, messages reaching a queue, specific log messages hitting the monitoring system, and files uploaded to storage. With this category of serveless compute, you stitch together managed services into apps, writing as little code as possible. Little-to-none of your existing codebase transfers over, as this caters to greenfield solutions based on a cloud-first approach.

AWS Lambda is the grandaddy of them all, so let’s take a look at it.

In my example, I want to read messages from a queue. Specifically, have an AWS Lambda function read from Amazon SQS. Sounds simple enough!

You can write AWS Lambda functions in many ways. You can also deploy them in many ways. There are many frameworks that try to simplify the latter, as you would rarely deploy a single function as your “app.” Rather, a function is part of a broader collection of resources that make up your system. Those resources might be described via the AWS Serverless Application Model (SAM), where you can lay out all the functions, databases, APIs and more that should get deployed together. And you could use the AWS Serverless Application Repository to browse and deploy SAM templates created by you, or others. However you define it, you’ll deploy your function-based system via the AWS CLI, AWS console, AWS-provided CI/CD tooling, or 3rd party tools like CircleCI.

For this simple demo, I’m going to build a C#-based function and deploy it via the AWS console.

First up, I went to the AWS console and defined a new queue in SQS. I chose the “standard queue” type.

Next up, creating a new AWS Lambda function. I gave it a name, chose .NET Core 3.1 as my runtime, and created a role with basic permissions.

After clicking “create function”, I get a overview screen that shows the “design” of my function and provides many configuration settings.

I clicked “add trigger” to specify what event kicks off my function. I’ve got lots of options to choose from, which is the hallmark of a “connective compute” function platform. I chose SQS, selected my previously-created queue from the dropdown list, and clicked “Add.”

Now all I have to do is the write the code that handles the queue message. I chose VS Code as my tool. At first, I tried using the AWS Toolkit for Visual Studio Code to generate a SAM-based project, but the only template was an API-based “hello world” one that forced me to retrofit a bunch of stuff after code generation. So, I decided to skip SAM for now, and code the AWS Lambda function directly, by itself.

The .NET team at AWS has done below-the-radar great work for years now, and their Lambda tooling is no exception. They offer a handful of handy templates you can use with the .NET CLI. One basic command installs them for you: dotnet new -i Amazon.Lambda.Templates

I chose to create a new project by entering dotnet new lambda.sqs. This produced a pair of projects, one with the function source code, and one that has unit tests. The primary project also has a aws-lambda-tools-default.json file that includes command line options for deploying your function. I’m not sure if I need it given I’m deploying via CLI, but I updated references to .NET Core 3.1 anyway. Note that the “function-handler” value *is* important, as we’ll need that shortly. This tells Lambda which operation (in which class) to invoke.

I kept the generated function code, which simply prints out the contents of the message pulled from Amazon SQS.

I successfully built the project, and then had to “publish” it to get the right assets for packaging. This publish command ensures that configuration files get bundled up as well:

dotnet publish /p:GenerateRuntimeConfigurationFiles=true

Now, all I have to do is zip up the resulting files in the “publish” directory. With those DLLs and *.json files zipped up, I return to the AWS console to upload my code. In most cases, you’re going to stash the archive file in Amazon S3 (either manually, or as the result of a CI process). Here, I uploaded my ZIP file directly, AND, set the function handler value equal to the “function-handler” value from my configuration file.

After I click “save”, I get a notice that my function was updated. I went back to Amazon SQS, and sent a few messages to the queue, using the “send a message” option.

After a moment, I saw entries in the “monitoring” view of the AWS Lambda console, and drilled into the CloudWatch logs and saw that my function wrote out the SQS payloads.

I’m impressed at how far the AWS Lambda experience has come since I first tried it out. You’ll find similarly solid experiences from Microsoft, Google and others as you use their FaaS platforms as glue code to connect managed services.

Category 2: Platform Expanding

Trigger / Destination	Signature	Packaging	Deployment
HTTP	Handlers with specific parameters	code packages	Web portal, CLI

There’s a category of FaaS that, to me, isn’t about connecting services together, as much as it’s about expanding or enriching the capabilities of a host platform. From the list above, I’d put offerings like Cloudflare Workers, Twilio Functions, and Zeit Serverless Functions into that bucket.

Most, if not all, of these start with an HTTP request and only support specific programming languages. For Twilio, you can use their integrated FaaS to serve up tokens, call outbound APIs after receiving an SMS, or even change voice calls. Zeit is an impressive host for static sites, and their functions platform supports backend operations like authentication, form submissions, and more. And Cloudflare Workers is about adding cool functionality whenever someone sends a request to a Cloudfare-managed domain. Let’s actually mess around with Cloudflare Workers.

I go to my (free) Cloudflare account to get started. You can create these running-at-the-edge functions entirely in the browser, or via the Wrangler CLI. Notice here that Workers support JavaScript, Rust, C, and C++.

After I click “create a Worker”, I’m immediately dropped into a web console where I can author, deploy, and test my function. And, I get some sample code that represents a fully-working Worker. All workers start by responding to a “fetch” event.

I don’t think you’d use this to create generic APIs or standalone apps. No, you’d use this to make the Cloudflare experience better. They handily have a whole catalog of templates to inspire you, or do your work for you. Most of these show examples of legit Cloudflare use cases: inspect and purge sensitive data from responses, deny requests missing an authorization header, do A/B testing based on cookies, and more. I copied the code from the “redirect” template which redirects requests to a different URL. I changed a couple things, clicked “save and deploy” and called my function.

On the left is my code. In the middle is the testing console, where I submitted a GET request, and got back a “301 Moved Permanently” HTTP response. I also see a log entry from my code. If you call my function in your browser, you’ll get redirected to cloudflare.com.

That was super simple. The serverless compute products in this category have a constrained set of functionality, but I think that’s on purpose. They’re meant to expand the set of problems you can solve with their platform, versus creating standalone apps or services.

Category 3: Full Stack Apps

Trigger / Destination	Signature	Packaging	Deployment
HTTP, queue, time	None	Containers	Web portal, CLI, CI/CD pipelines

This category—which I can’t quite figure out the right label for—is about serverless computing for complete web apps. These aren’t functions, per-se, but run on a serverless stack that scales to zero and is billed based on usage. The unit of deployment is a container, which means you are providing more than code to the platform—you are also supplying a web server. This can make serverless purists squeamish since a key value prop of FaaS is the outsourcing of the server to the platform, and only focusing on your code. I get that. The downside of that pure FaaS model is that it’s an unforgiving host for any existing apps.

What fits in this category? The only obvious one to me is Google Cloud Run, but AWS Fargate kinda fits here too. Google Cloud Run is based on the popular open source Knative project, and runs as a managed service in Google Cloud. Let’s try it out.

First, install the Google Cloud SDK to get the gcloud command line tool. Once the CLI gets installed, you do a gcloud init in order to link up your Google Cloud credentials, and set some base properties.

Now, to build the app. What’s interesting here, is this is just an app. There’s no special format or method signature. The app just has to accept HTTP requests. You can write the app in any language, use any base image, and end up with a container of any size. The app should still follow some basic cloud-native patterns around fast startup and attached storage. This means—and Google promotes this—that you can migrate existing apps fairly easily. For my example, I’ll use Visual Studio for Mac to build a new ASP.NET Web API project with a couple RESTful endpoints.

The default project generates a weather-related controller, so let’s stick with that. To show that Google Cloud Run handles more than one endpoint, I’m adding a second method. This one returns a forecast for Seattle, which has been wet and cold for months.

namespace seroter_api_gcr.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class WeatherForecastController : ControllerBase
    {
        private static readonly string[] Summaries = new[]
        {
            "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
        };

        private readonly ILogger<WeatherForecastController> _logger;

        public WeatherForecastController(ILogger<WeatherForecastController> logger)
        {
            _logger = logger;
        }

        [HttpGet]
        public IEnumerable<WeatherForecast> Get()
        {
            var rng = new Random();
            return Enumerable.Range(1, 5).Select(index => new WeatherForecast
            {
                Date = DateTime.Now.AddDays(index),
                TemperatureC = rng.Next(-20, 55),
                Summary = Summaries[rng.Next(Summaries.Length)]
            })
            .ToArray();
        }

        [HttpGet("seattle")]
        public WeatherForecast GetSeattleWeather()
        {
            return new WeatherForecast { Date = DateTime.Now, Summary = "Chilly", TemperatureC = 6 };
        }
    }
}

If I were doing this the right way, I’d also change my Program.cs file and read the port from a provided environment variable, as Google suggests. I’m NOT going to do that, and instead will act like I’m just shoveling an existing, unchanged API into the service.

The app is complete and works fine when running locally. To work with Google Cloud Run, my app must be containerized. You can do this a variety of ways, including the most reasonable, which involves Google Cloud Build and continuous delivery. I don’t roll like that. WE’RE DOING IT BY HAND.

I will cheat and have Visual Studio give me a valid Dockerfile. Right-click the project, and add Docker support. This creates a Docker Compose project, and throws a Dockerfile into my original project.

Let’s make one small tweak. In the Dockerfile, I’m exposing port 5000 from my container, and setting an environment variable to tell my app to listen on that port.

I opened my CLI, and navigated to the folder directly above this project. From there, I executed a Docker build command that pointed to the generated Dockerfile, and tagged the image for Google Container Registry (where Google Cloud Run looks for images).

docker build --file ./seroter-api-gcr/Dockerfile . --tag gcr.io/seroter/seroter-api-gcr

That finished, and I had a container image in my local registry. I need to get it up to Google Container Registry, so I ran a Docker push command.

docker push gcr.io/seroter/seroter-api-gcr

After a moment, I see that container in the Google Container Registry.

Neat. All that’s left is to spin up Google Cloud Run. From the Google Cloud portal, I choose to create a new Google Cloud Run service. I choose a region and name for my service.

Next up, I chose the container image to use, and set the container port to 5000. There are lots of other settings here too. I can create a connection to managed services like Cloud SQL, choose max requests per container, set the request timeout, specify the max number of container instances, and more.

After creating the service, I only need to wait a few seconds before my app is reachable.

As expected, I can ping both API endpoints and get back a result. After a short duration, the service spins compute down to zero.

Wrap up

The landscape of serverless computing is broader than you may think. Depending on what you’re trying to do, it’s possible to make a sub-optimal choice. If you’re working with many different managed services and writing code to connect them, use the first category. If you’re enriching existing platforms with bits of compute functionality, use the second category. And if you’re migrating or modernizing existing apps, or have workloads that demand more platform flexibility, choose the third. Comments? Violent disagreement? Tell me below.

April 2, 2020

My new Pluralsight course—DevOps in Hard Places—is now available

Design user-centric products and continuously deliver your software to production while collecting and incorporating feedback the whole time? Easy peasy. Well, if you’re a software startup. What about everyone else in the real world? What gets in your way and sucks the life from your eager soul? You name it, siloed organizations, outsourcing arrangements, overworked teams, regulatory constraints, annual budgeting processes, and legacy apps all add friction. I created a new Pluralsight course that looks at these challenges, and offers techniques for finding success.

Home page for the course on Pluralsight

DevOps in Hard Places is my 21st Pluralsight course, and hopefully one of the most useful ones. It clocks in at about 90 minutes, and is based on my own experience, the experience of people at other large companies, and feedback from some smart folks.

You’ll find three modules in this course, looking at the people, process, and technology challenges you face making a DevOps model successful in complex organizations. For each focus area, I review the status quo, how that impacts your chance of success, and 2-3 techniques that you can leverage.

The first looks at the people-related issues, and various ways to overcome them. In my experience, few people WANT to be blockers, but change is hard, and you have to lead the way.

The status quo facing orgs with siloed structures

The second module looks at processes that make a continuous delivery mindset difficult. I don’t know of too many processes that are SUPPOSED to be awful—except expense reporting which is designed to make you retire early—but over time, many processes make it difficult to get things done quickly.

How annual budgeting processes make adopting DevOps harder

Finally, we go over the hard technology scenarios that keep you from realizing your potential. If you have these problems, congratulations, it means you’ve been in business for a while and have technology that your company depends on. Now is the time to address some of those things holding you back.

One technique for doing DevOps with databases and middleware

Let me know what you think, and I hope this course helps you get un-stuck or recharged in your effort to get better at software.

February 7, 2020
2019 in Review: Watching, Reading, and Writing Highlights

Be still and wait. This was the best advice I heard in 2019, and it took until the end of the year for me to realize it. Usually, when I itch for a change, I go all in, right away. I’m prone to thinking that “patience” is really just “indecision.” It’s not. The best things that happened this year were the things that didn’t happen when I wanted! I’m grateful for an eventful, productive, and joyful year where every situation worked out for the best.

2019 was something else. My family grew, we upgraded homes, my team was amazing, my company was acquired by VMware, I spoke at a few events around the world, chaired a tech conference, kept up a podcast, created a couple new Pluralsight classes, continued writing for InfoQ.com, and was awarded a Microsoft MVP for the 12th straight time.

For the last decade+, I’ve started each year by recapping the last one. I usually look back at things I wrote, and books I read. This year, I’ll also add “things I watched.”

Things I Watched

I don’t want a ton of “regular” TV—although I am addicted to Bob’s Burgers and really like the new FBI—and found myself streaming or downloading more things while traveling this year. These shows/seasons stood out to me:

Crashing – Season 3 [HBO] Pete Holmes is one of my favorite stand-up comedians, and this show has some legit funny moments, but it’s also complex, dark, and real. This was a good season with a great ending.

BoJack Horseman – Season 5 [Netflix] Again, a show with absurdist humor, but also a dark, sobering streak. I’m got to catch up on the latest season, but this one was solid.

Orange is the New Black – Season 7 [Netflix] This show has had some ups and downs, but I’ve stuck with it because I really like the cast, and there are enough surprises to keep me hooked. This final season of the show was intense and satisfying.

Bosch – Season 4 [Amazon Prime] Probably the best thing I watched this year? I love this show. I’ve read all the books the show is based on, but the actors and writers have given this its own tone. This was a super tense season, and I couldn’t stop watching.

Schitt’s Creek – Seasons 1-4 [Netflix] Tremendous cast and my favorite overall show from 2019. Great writing, and some of the best characters on TV. Highly recommended.

Jack Ryan – Season 1 [Amazon Prime] Wow, what a show. Throughly enjoyed the story and cast. Plenty of twists and turns that led me to binge watch this on one of my trips this year.

Things I Wrote

I kept up a reasonable writing rhythm on my own blog, as well as publication to the Pivotal blog and InfoQ.com site. Here were a few pieces I enjoyed writing the most:

[Pivotal blog] Five part series on digital transformation. You know what you should never do? Write a blog post and in it, promise that you’ll write four more. SO MUCH PRESSURE. After the overview post, I looked at the paradox of choice, design thinking, data processing, and automated delivery. I’m proud of how it all ended up.

[blog] Which of the 295,680 platform combinations will you create on Microsoft Azure? The point of this post wasn’t that Microsoft, or any cloud provider for that matter, has a lot of unique services. They do, but the point was that we are prone to thinking that we’re getting a complete solution from someone, but really getting some really cool components to stitch together.

[Pivotal blog] Kubernetes is a platform for building platforms. Here’s what that means. This is probably my favorite piece I wrote this year. It required a healthy amount of research and peer review, and dug into something I see very few people talking about.

[blog] Go “multi-cloud” while *still* using unique cloud services? I did it using Spring Boot and MongoDB APIs. There’s so many strawman arguments on Twitter when it comes to multi-cloud that it’s like a scarecrow convention. Most people I see using multiple clouds aren’t dumb or lazy. They have real reasons, including a well-founded lack of trust in putting all their tech in one vendor’s basket. This blog post looked at how to get the best of all worlds.

[blog] Looking to continuously test and patch container images? I’ll show you one way. I’m not sure when I give up on being a hands on technology person. Maybe never? This was a demo I put together for my VMworld Barcelona talk, and like the final result.

[blog] Building an Azure-powered Concourse pipeline for Kubernetes – Part 3: Deploying containers to Kubernetes. I waved the white flag and learned Kubernetes this year. One way I forced myself to do so was sign up to teach an all-day class with my friend Rocky. While leading up to that, I wrote up this 3-part series of posts on continuous delivery of containers.

[blog] Want to yank configuration values from your .NET Core apps? Here’s how to store and access them in Azure and AWS. It’s fun to play with brand new tech, curse at it, and document your journey for others so they curse less. Here I tried out Microsoft’s new configuration storage service, and compared it to other options.

[blog] First Look: Building Java microservices with the new Azure Spring Cloud. Sometimes it’s fun to be first. Pivotal worked with Microsoft on this offering, so on the day it was announced, I had a blog post ready to go. Keep an eye on this service in 2020; I think it’ll be big.

[InfoQ] Swim Open Sources Platform That Challenges Conventional Wisdom in Distributed Computing. One reason I keep writing for InfoQ is that it helps me discover exciting new things. I don’t know if SWIM will be a thing long term, but their integrated story is unconventional in today’s “I’ll build it all myself” world.

[InfoQ] Weaveworks Releases Ignite, AWS Firecracker-Powered Software for Running Containers as VMs. The other reason I keep writing for InfoQ is that I get to talk to interesting people and learn from them. Here, I engaged in an informative Q&A with Alexis and pulled out some useful tidbits about GitOps.

[InfoQ] Cloudflare Releases Workers KV, a Serverless Key-Value Store at the Edge. Feels like edge computing has the potential to disrupt our current thinking about what a “cloud” is. I kept an eye on Cloudflare this year, and this edge database warranted a closer look.

Things I Read

I like to try and read a few books a month, but my pace was tested this year. Mainly because I chose to read a handful of enormous biographies that took a while to get through. I REGRET NOTHING. Among the 32 books I ended up finishing in 2019, these were my favorites:

Churchill: Walking with Destiny by Andrew Roberts (@aroberts_andrew). This was the most “highlighted” book on my Kindle this year. I knew the caricature, but not the man himself. This was a remarkably detailed and insightful look into one of the giants of the 20th century, and maybe all of history. He made plenty of mistakes, and plenty of brilliant decisions. His prolific writing and painting were news to me. He’s a lesson in productivity.

At Home: A Short History of Private Life by Bill Bryson. This could be my favorite read of 2019. Bryson walks around his old home, and tells the story of how each room played a part in the evolution of private life. It’s a fun, fascinating look at the history of kitchens, studies, bedrooms, living rooms, and more. I promise that after you read this book, you’ll be more interesting at parties.

Messengers: Who We Listen To, Who We Don’t, and Why by Stephen Martin (@scienceofyes) and Joseph Marks (@Joemarks13). Why is it that good ideas get ignored and bad ideas embraced? Sometimes it depends on who the messenger is. I enjoyed this book that looked at eight traits that reliably predict if you’ll listen to the messenger: status, competence, attractiveness, dominance, warm, vulnerability, trustworthiness, and charisma.

Six Days of War: June 1967 and the Making of the Modern Middle East by Michael Oren (@DrMichaelOren). What a story. I had only a fuzzy understanding of what led us to the Middle East we know today. This was a well-written, engaging book about one of the most consequential events of the 20th century.

The Unicorn Project: A Novel about Developers, Digital Disruption, and Thriving in the Age of Data by Gene Kim (@RealGeneKim). The Phoenix Project is a must-read for anyone trying to modernize IT. Gene wrote that book from a top-down leadership perspective. In The Unicorn Project, he looks at the same situation, but from the bottom-up perspective. While written in novel form, the book is full of actionable advice on how to chip away at the decades of bureaucratic cruft that demoralizes IT and prevents forward progress.

Talk Triggers: The Complete Guide to Creating Customers with Word of Mouth by Jay Baer (@jaybaer) and Daniel Lemin (@daniellemin). Does your business have a “talk trigger” that leads customers to voluntarily tell your story to others? I liked the ideas put forth by the authors, and the challenge to break out from the pack with an approach (NOT a marketing gimmick) that really resonates with customers.

I Heart Logs: Event Data, Stream Processing, and Data Integration by Jay Kreps (@jaykreps). It can seem like Apache Kafka is the answer to everything nowadays. But go back to the beginning and read Jay’s great book on the value of the humble log. And how it facilitates continuous data processing in ways that preceding technologies struggled with.

Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale by Neha Narkhede (@nehanarkhede), Gwen Shapira (@gwenshap), and Todd Palino (@bonkoif). Apache Kafka is probably one of the five most impactful OSS projects of the last ten years, and you’d benefit from reading this book by the people who know it. Check it out for a great deep dive into how it works, how to use it, and how to operate it.

The Players Ball: A Genius, a Con Man, and the Secret History of the Internet’s Rise by David Kushner (@davidkushner). Terrific story that you’ve probably never heard before, but have felt its impact. It’s a wild tale of the early days of the Web where the owner of sex.com—who also created match.com—had it stolen, and fought to get it back. It’s hard to believe this is a true story.

Mortal Prey by John Sanford. I’ve read a dozen+ of the books in this series, and keep coming back for more. I’m a sucker for a crime story, and this is a great one. Good characters, well-paced plots.

Your God is Too Safe: Rediscovering the Wonder of a God You Can’t Control by Mark Buchanan (@markaldham). A powerful challenge that I needed to hear last year. You can extrapolate the main point to many domains—is something you embrace (spirituality, social cause, etc) a hobby, or a belief? Is it something convenient to have when you want it, or something powerful you do without regard for the consequences? We should push ourselves to get off the fence!

Escaping the Build Trap: How Effective Product Management Creates Real Value by Melissa Perri (@lissijean). I’m not a product manager any longer, but I still care deeply about building the right things. Melissa’s book is a must-read for people in any role, as the “build trap” (success measured by output instead of outcomes) infects an entire organization, not just those directly developing products. It’s not an easy change to make, but this book offers tangible guidance to making the transition.

Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework by Mik Kersten (@mik_kersten). This is such a valuable book for anyone trying to unleash their “stuck” I.T. organization. Mik does a terrific job explaining what’s not working given today’s realities, and how to unify an organization around the value streams that matter. The “flow framework” that he pioneered, and explains here, is a brilliant way of visualizing and tracking meaningful work.

Range: Why Generalists Triumph in a Specialized World by David Epstein (@DavidEpstein). I felt “seen” when I read this. Admittedly, I’ve always felt like an oddball who wasn’t exceptional at one thing, but pretty good at a number of things. This book makes the case that breadth is great, and most of today’s challenges demand knowledge transfer between disciplines and big-picture perspective. If you’re a parent, read this to avoid over-specializing your child at the cost of their broader development. And if you’re starting or midway through a career, read this for inspiration on what to do next.

John Newton: From Disgrace to Amazing Grace by Jonathan Aitken. Sure, everyone knows the song, but do you know the man? He had a remarkable life. He was the captain of a slave ship, later a pastor and prolific writer, and directly influenced the end of the slave trade.

Blue Ocean Shift: Beyond Competing – Proven Steps to Inspire Confidence and Seize New Growth by W. Chan Kim and Renee Mauborgne. This is a book about surviving disruption, and thriving. It’s about breaking out of the red, bloody ocean of competition and finding a clear, blue ocean to dominate. I liked the guidance and techniques presented here. Great read.

Leonardo da Vinci by Walter Isaacson (@WalterIsaacson). Huge biography, well worth the time commitment. Leonardo had range. Mostly self-taught, da Vinci studying a variety of topics, and preferred working through ideas to actually executing on them. That’s why he had so many unfinished projects! It’s amazing to think of his lasting impact on art, science, and engineering, and I was inspired by his insatiable curiosity.

AI Superpowers: China, Silicon Valley, and the New World Order by Kai-Fu Lee (@kaifulee). Get past some of the hype on artificial intelligence, and read this grounded book on what’s happening RIGHT NOW. This book will make you much smarter on the history of AI research, and what AI even means. It also explains how China has a leg up on the rest of the world, and gives you practical scenarios where AI will have a big impact on our lives.

Never Split the Difference: Negotiating As If Your Life Depended On It by Chris Voss (@VossNegotiation) and Tahl Raz (@tahlraz). I’m fascinated by the psychology of persuasion. Who better to learn negotiation from than an FBI’s international kidnapping negotiator? He promotes empathy over arguments, and while the book is full of tactics, it’s not about insincere manipulation. It’s about getting to a mutually beneficial state.

Amazing Grace: William Wilberforce and the Heroic Campaign to End Slavery by Eric Metaxas (@ericmetaxas). It’s tragic that this generation doesn’t know or appreciate Wilberforce. The author says that Wilberforce could be the “greatest social reformer in the history of the world.” Why? His decades-long campaign to abolish slavery from Europe took bravery, conviction, and effort you rarely see today. Terrific story, well written.

Unlearn: : Let Go of Past Success to Achieve Extraordinary Results by Barry O’Reilly (@barryoreilly). Barry says that “unlearning” is a system of letting go and adapting to the present state. He gives good examples, and offers actionable guidance for leaders and team members. This strikes me as a good book for a team to read together.

The Soul of a New Machine by Tracy Kidder. Our computer industry is younger than we tend to realize. This is such a great book on the early days, featuring Data General’s quest to design and build a new minicomputer. You can feel the pressure and tension this team was under. Many of the topics in the book—disruption, software compatibility, experimentation, software testing, hiring and retention—are still crazy relevant today.

Billion Dollar Whale: The Man Who Fooled Wall Street, Hollywood, and the World by Tom Wright (@TomWrightAsia) and Bradley Hope (@bradleyhope). Jho Low is a con man, but that sells him short. It’s hard not to admire his brazenness. He set up shell companies, siphoned money from government funds, and had access to more cash than almost any human alive. And he spent it. Low befriended celebrities and fooled auditors, until it all came crashing down just a few years ago.

Multipliers: How the Best Leaders Make Everyone Smarter by Liz Wiseman (@LizWiseman). It’s taken me very long (too long?) to appreciate that good managers don’t just get out of the way, they make me better. Wiseman challenges us to release the untapped potential of our organizations, and people. She contrasts the behavior of leaders that diminish their teams, and those that multiply their impact. Lots of food for thought here, and it made a direct impact on me this year.

Darwin’s Doubt: The Explosive Origin of Animal Life and the Case for Intelligent Design by Stephen Meyer (@StephenCMeyer). The vast majority of this fascinating, well-researched book is an exploration of the fossil record and a deep dive into Darwin’s theory, and how it holds up to the scientific research since then. Whether or not you agree with the conclusion that random mutation and natural selection alone can’t explain the diverse life that emerged on Earth over millions of years, it will give you a humbling appreciation for the biological fundamentals of life.

Napoleon: A Life by Adam Zamoyski. This was another monster biography that took me months to finish. Worth it. I had superficial knowledge of Napoleon. From humble beginnings, his ambition and talent took him to military celebrity, and eventually, the Emperorship. This meticulously researched book was an engaging read, and educational on the time period itself, not just Bonaparte’s rise and fall.

The Paradox of Choice: Why More Is Less by Barry Schwartz. I know I’ve used this term for year’s since it was part of other book’s I’ve read. But I wanted to go to the source. We hate having no choices, but are often paralyzed by having too many. This book explores the effects of choice on us, and why more is often less. It’s a valuable read, regardless of what job you have.

I say it every year, but thank you for having me as part of your universe in 2019. You do have a lot of choices of what to read or watch, and I truly appreciate when you take time to turn that attention to something of mine. Here’s to a great 2020!

January 6, 2020
Looking to continuously test and patch container images? I’ll show you one way.
A lot of you are packaging code into container images before shipping it off to production. That’s cool. For many, this isn’t a one-time exercise at the end of a project; it’s an ongoing exercise throughout the lifespan of your product. Last week in Barcelona, I did a presentation at VMworld Europe where I took a custom app, ran tests in a pipeline, containerized it, and pushed to a cloud runtime. I did all of this with fresh open-source technologies like Kubernetes, Concourse, and kpack. For this blog post, I’ll show you my setup, and for fun, take the resulting container image and deploy it, unchanged, to one Microsoft Azure service, and one Pivotal service.

First off, containers. Let’s talk about them. The image that turns into running container is made up of a series of layers. This union of read-only layers gets mounted to present itself as a single filesystem. Many commands in your Dockerfile, generate a layer. When I pull the latest Redis image, and run a docker history command, I see all the layers:

Ok, Richard, we get it. Like onions and ogres, images have layers. I bring it up, because responsibly maintaining a container image means continually monitoring and updating those layers. For a custom app, that means updating layers that store app code, the web server, and the root file system. All the time. Ideally, I want a solution that automatically builds and patches all this stuff so that I don’t have to. Whatever pipeline to production you build should have that factored in!

Let’s get to it. Here’s what I built. After coding a Spring Boot app, I checked the code into a GitHub master branch. That triggered a Concourse pipeline (running in Kubernetes) that ran unit tests, and promoted the code to a “stable” branch if the tests passed. The container build service (using the kpack OSS project) monitored the stable branch, and built a container image which got stored in the Docker Hub. From there, I deployed the Docker image to a container-friendly application runtime. Easy!

Step #1 – Build the app

The app is simple, and relatively inconsequential. Build a .NET app, Go app, Node.js app, whatever. I built a Spring Boot app using Spring Initializr. Click here to download the same scaffolding. This app will simply serve up a web endpoint, and also offer a health endpoint.

In my code, I have a single RESTful endpoint that responds to GET requests at the root. It reads an environment variable (so that I can change it per runtime), and returns that in the response.
```
@RestController
public class GreetingController {
	
  @Value("${appruntime:Spring Boot}")
  private String appruntime;
	
  @GetMapping("/")
  public String SayHi() {
    return "Hello VMworld Europe! Greetings from " + appruntime;
  }
}
```
I also created a single JUnit test to check the response value from my RESTful service. I write great unit tests; don’t be jealous.
```
@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class BootKpackDemoApplicationTests {

  @LocalServerPort
  private int port;
	
  @Autowired
  private TestRestTemplate restTemplate;
	
  @Test
  public void testEndpoint() {
    assertThat(this.restTemplate.getForObject("http://localhost:" + port + "/",
    String.class)).contains("Hello");
  }
}
```
After crafting this masterpiece, I committed it to a GitHub repo. Ideally, this is all a developer ever has to do in their job. Write code, test it, check it in, repeat. I don’t want to figure out the right Dockerfile format, configure infrastructure, or any other stuff. Just let me write code, and trigger a pipeline that gets my code securely to production, over and over again.

Step #2 – Set up the CI pipeline

For this example, I’m using minikube on my laptop to host the continuous integration software and container build service. I got my Kubernetes 1.15 cluster up (since Concourse currently works up to v 1.15) with this command:
```
minikube start --memory=4096 --cpus=4 --vm-driver=hyperkit --kubernetes-version v1.15.0
```
Since I wanted to install Concourse in Kubernetes via Helm, I needed Helm and tiller set up. I used a package manager to install Helm on my laptop. Then I ran three commands to generate a service account, bind a cluster role to that service account, and initialize Helm in the cluster.
```
kubectl create serviceaccount -n kube-system tiller 
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller 
helm init --service-account tiller 
```
With that business behind me, I could install Concourse. I talk a lot about Concourse, taught a Pluralsight course about it, and use it regularly. It’s such a powerful tool for continuous processing of code. To install into Kubernetes, it’s just a single reference to a Helm chart.
```
helm install --name vmworld-concourse stable/concourse
```
After a few moments, I saw that I had pods created and services configured.

The chart also printed out commands for how to do port forwarding to access the Concourse web console.
```
export POD_NAME=$(kubectl get pods --namespace default -l "app=vmworld-concourse-web" -o jsonpath="{.items[0].metadata.name}")
 echo "Visit http://127.0.0.1:8080 to use Concourse"
 kubectl port-forward --namespace default $POD_NAME 8080:8080
```
After running those commands, I pinged the localhost URL and saw the dashboard.

All that was left was the actual pipeline. Concourse pipelines are defined in YAML. My GitHub repo has two branches (master and stable), so I declared “resources” for both. Since I have to write to the stable branch, I also included credentials to GitHub in the “stable” resource definition. My pipeline has two jobs: one that runs the JUnit tests, and another puts the master branch code into the stable branch if the unit tests pass.
```
---
# declare resources
resources:
- name: source-master
  type: git
  icon: github-circle
  source:
    uri: https://github.com/rseroter/boot-kpack-demo
    branch: master
- name: source-stable
  type: git
  icon: github-circle
  source:
    uri: git@github.com:rseroter/boot-kpack-demo.git
    branch: stable
    private_key: ((github-private-key))

jobs:
- name: run-tests
  plan:
  - get: source-master
    trigger: true
  - task: first-task
    config: 
      platform: linux
      image_resource:
        type: docker-image
        source: {repository: maven, tag: latest}
      inputs:
      - name: source-master
      run:
          path: sh
          args:
          - -exec
          - |
            cd source-master
            mvn package
- name: promote-to-stable
  plan:
  - get: source-master
    trigger: true
    passed: [run-tests]
  - get: source-stable
  - put: source-stable
    params:
      repository: source-master
```
Deploying this pipeline is easy. From the fly CLI tool, it’s one command. Note that my GitHub creds are stored in another file, which is the one I reference in the command.
```
fly -t vmworld set-pipeline --pipeline vmworld-pipeline --config vmworld-pipeline.yaml --load-vars-from params.yaml
```
After unpausing the pipeline, it ran. Once it executed the unit tests, and promoted the master code to the stable branch, the pipeline was green.

Step #3 – Set up kpack for container builds

Now to take that tested, high-quality code and containerize it. Cloud Native Buildpacks turn code into Docker images. Buildpacks are something initially created by Heroku, and then used by Cloud Foundry to algorithmically determine how to build a container image based on the language/framework of the code. Instead of developers figuring out how to layer up an image, buildpacks can compile and package up code in a repeatable way by bringing in all the necessary language runtimes and servers. What’s cool is that operators can also extend buildpacks to add org-specific certs, monitoring agents, or whatever else should be standard in your builds.

kpack is an open-source project from Pivotal that uses Cloud Native Buildpacks, also adds the ability to watch for changes to anything impacting the image, and initiating an update. kpack, which is commercialized as the Pivotal Build Service, watches for changes in source code, buildpacks, or base image and then puts the new or patched image into the registry. Thanks to some smarts, it only updates the impacted layers, thus saving you on data transfer costs and build times.

The installation instructions are fairly straightforward. You can put this into your Kubernetes cluster in a couple minutes. Once installed, I saw the single kpack controller pod running.

The only thing left to do was define an image configuration. This declarative config tells kpack where to find the code, and what to do with it. I had already set up a secret to hold my Docker Hub creds, and that corresponding Kubernetes service account is referenced in the image configuration.
```
apiVersion: build.pivotal.io/v1alpha1
kind: Image
metadata:
  name: vmworld-image
spec:
  tag: rseroter/vmworld-demo
  serviceAccount: vmworld-service-account
  builder:
    name: default-builder
    kind: ClusterBuilder
  source:
    git:
      url: https://github.com/rseroter/boot-kpack-demo.git
      revision: stable
```
That’s it. Within moments, kpack detected my code repo, compiled my app, built a container image, cached some layers for later, and updated the Docker Hub image.

I made a bunch of code changes to generate lots of builds, and all the builds showed up my Kubernetes cluster as well.

Now when I updated my code, my pipeline automatically kicks off and updates the stable branch. Thus, whenever my tested code changes, or the buildpack gets updated (every week or so) with framework updates and patches, my container automatically gets rebuilt. That’s crazy powerful stuff, especially as we create more and more containers, that deploy to more and more places.

Step #4 – Deploy the container image

And that’s the final step. I had to deploy this sucker and see it run.

First, I pushed it to Pivotal Application Service (PAS) because I make good choices. I can push code or containers here. This single command takes that Docker image, deploys it, and gives me a routable endpoint in 20 seconds.
```
cf push vmworld-demo --docker-image rseroter/vmworld-demo -i 2
```
That worked great, and my endpoint returned the expected values after I added an environment variable to the app.

Can I deploy the same container to Azure Web Apps? Sure. That takes code or containers too. I walked through the wizard experience in the Azure Portal and chose the Docker Hub image created by kpack.

After a few minutes, the service was up. Then I set the environment variable that the Spring Boot app was looking for (appruntime to “Azure App Service”) and another to expose the right port (WEBSITES_PORT to 8080), and pinged the RESTful endpoint.

Whatever tech you land on, just promise me that you’ll invest in a container patching strategy. Automation is non-negotiable, and there are good solutions out there that can improve your security posture, while speeding up software delivery.
November 13, 2019
Fronting web sites, a classic .NET app, and a serverless function with Spring Cloud Gateway
Automating deployment of custom code and infrastructure? Not always easy, but feels like a solved problem. It gets trickier when you want to use automation to instantiate and continuously update databases and middleware. Why? This type of software stores state which makes upgrades more sensitive. You also may be purchasing this type of software from vendors who haven’t provided a full set of automation-friendly APIs. Let’s zero in on one type of middleware: API gateways.

API gateways do lots of things. They selectively expose private services to wider audiences. With routing rules, they make it possible to move clients between versions of a service without them noticing. They protect downstream services by offering capabilities like rate limiting and caching. And they offer a viable way for those with a microservices architecture to secure services without requiring each service to do their own authentication. Historically, your API gateway was a monolith of its own. But a new crop of automation-friendly OSS (and cloud-hosted) options are available, and this gives you new ways to deploy many API gateway instances that get continuously updated.

I’ve been playing around with Spring Cloud Gateway, which despite its name, can proxy traffic to a lot more than just Spring Boot applications. In fact, I wanted to try and create a configuration-only-no-code API Gateway that could do three things:
1. Weighted routing between “regular’ web pages on the internet.
2. Add headers to a JavaScript function running in Microsoft Azure.
3. Performing rate-limiting on a classic ASP.NET Web Service running on the Pivotal Platform.
Before starting, let me back up and briefly explain what Spring Cloud Gateway is. Basically, it’s a project that turns a Spring Boot app into an API gateway that routes requests while applying cross-cutting functionality for things like security. Requests come in, and if the request matches a declared route, the request is passed through a series of filters, sent to the target endpoint, and “post” filters get applied on the way back to the client. Spring Cloud Gateway built on a Reactive base, which means it’s non-blocking and efficiently handles many simultaneous requests.

The biggest takeaway? This is just an app. You can write tests and do continuous integration. You can put it on a pipeline and continuously deliver your API gateway. That’s awesome.

Note that you can easily follow along with the steps below without ANY Java knowledge! Everything I’m doing using configuration you can also do with the Java DSL, but I wanted to prove how straightforward the configuration-only option is.

Creating the Spring Cloud Gateway project

This is the first, and easiest, part of this demonstration. I went to start.spring.io, and generated a new Spring Boot project. This project has dependencies on Gateway (to turn this into an API gateway), Spring Data Reactive Redis (for storing rate limiting info), and Spring Boot Actuator (so we get “free” metrics and insight into the gateway). Click this link to generate an identical project.

Doing weighed routing between web pages

For the first demonstration, I wanted to send traffic to either spring.io or pivotal.io/spring-app-framework. You might use weighted routing to do A/B testing with different versions of your site, or even to send a subset of traffic to a new API.

I added an application.yml file (to replace the default application.properties file) to hold all my configuration settings. Here’s the configuration, and we’ll go through it bit by bit.
```
spring:
  cloud:
    gateway:
      routes:
      # doing weighted routing between two sites
      - id: test1
        uri: https://www.pivotal.io
        predicates:
        - Path=/spring
        - Weight=group1, 3
        filters:
        - SetPath=/spring-app-framework
      - id: test2
        uri: https://www.spring.io
        predicates:
        - Path=/spring
        - Weight=group1, 7
        filters:
        - SetPath=/
```
Each “route” is represented by a section in the YAML configuration. A route has a URI (which represents the downstream host), and a route predicate that indicates the path on the gateway you’re invoking. For example, in this case, my path is “/spring” which means that sending a request to “localhost:8080/spring” would map to this route configuration.

Now, you’ll see I have two routes with the same path. These are part of the same weighted routing group, which means that traffic to /spring will go to one of the two downstream endpoints. The second endpoint is heavily weighted (7 vs 3), so most traffic goes there. Also see that I applied one filter to clear out the path. If I didn’t do this, then requests to localhost:8080/spring would result in a call to spring.io/spring, as the path (and querystring) is forwarded. Instead, I stripped that off for requests to spring.io, and added the secondary path into the pivotal.io endpoint.

I’ve got Java and Maven installed locally, so a simple command (mvn spring-boot:run) starts up my Spring Cloud Gateway. Note that so far, I’ve written exactly zero code. Thanks to Spring Boot autoconfiguration and dependency management, all the right packages exist and runtime objects get inflated. Score!

Once, the Spring Cloud Gateway was up and running, I pinged the Gateway’s endpoint in the browser. Note that some browser’s try to be helpful by caching things, which screws up a weighted routing demo! I opened the Chrome DevTools and disabled request caching before running a test.

That worked great. Our gateway serves up a single endpoint, but through basic configuration, I can direct a subset of traffic somewhere else.

Adding headers to serverless function calls

Next, I wanted to stick my gateway in front of some serverless functions running in Azure Functions. You could imagine having a legacy system that you were slowly strangling and replacing with managed services, and leveraging Spring Cloud Gateway to intercept calls and redirect to the new destination.

For this example, I built a dead-simple JavaScript function that’s triggered via HTTP call. I added a line of code that prints out all the request headers before sending a response to the caller.

The Spring Cloud Gateway configuration is fairly simple. Let’s walk through it.
```
spring:
  cloud:
    gateway:
      routes:
      # doing weighted routing between two sites
      - id: test1
        ...
      # adding a header to an Azure Function request
      - id: test3
        uri: https://seroter-function-app.azurewebsites.net
        predicates:
        - Path=/function
        filters:
        - SetPath=/api/HttpTrigger1
        - SetRequestHeader=X-Request-Seroter, Pivotal
```
Like before, I set the URI to the target host, and set a gateway path. On the pre-filters, I reset the path (removing the /function and replacing with the “real” path to the Azure Function) and added a new request header.

I started up the Spring Cloud Gateway project and sent in a request via Postman. My function expects a “name” value, which I provided as a query parameter.

I jumped back to the Azure Portal and checked the logs associated with my Azure Function. Sure enough, I see all the HTTP request headers, including the random one that I added via the gateway. You could imagine this type of functionality helping if you have modern endpoints and legacy clients and need to translate between them!

Applying rate limiting to an ASP.NET Web Service

You know what types of apps can benefit from an API Gateway? Legacy apps that weren’t designed for high load or modern clients. One example is rate limiting. Your legacy service may not be able to handle internet-scale requests, or have a dependency on a downstream system that isn’t mean to get pummeled with traffic. You can apply request caching and rate limiting to prevent clients from burying the legacy app.

First off, I built a classic ASP.NET Web Service. I hoped to never use SOAP again, but I’m dedicated to my craft.

I did a “cf push” to my Pivotal Application Service environment and deployed two instances of the app to a Windows environment. In a few seconds, I had a publicly-accessible endpoint.

Then it was back to my Gateway configuration. To do rate limiting, you need a way to identify callers. You know, some way to say that client X has exceeded their limit. Out of the box, there’s a rate limiter that uses Redis to store information about clients. That means I need a Redis instance. The simplest answer is “Docker”, so I ran a simple command to get Redis running locally (docker run --name my-redis -d -p 6379:6379 redis).

I also needed a way to identify the caller. Here, I finally had to write some code. Specifically, this rate limiter filter expects a “key resolver.” I don’t see a way to declare one via configuration, so I opened the .java file in my project and added a Bean declaration that pulls a query parameter named “user.” That’s not enterprise ready (as you’d probably pull source IP, or something from a header), but this’ll do.
```
@SpringBootApplication
public class CloudGatewayDemo1Application {

  public static void main(String[] args) {	 
   SpringApplication.run(CloudGatewayDemo1Application.class, args);
  }
	
  @Bean
  KeyResolver userKeyResolver() {
    return exchange -> 
   Mono.just(exchange.getRequest().getQueryParams().getFirst("user"));
  }
}
```
All that was left was my configuration. Besides adding rate limiting, I also wanted to to shield the caller from setting all those gnarly SOAP-related headings, so I added filters for that too.
```
spring:
  cloud:
    gateway:
      routes:
      # doing weighted routing between two sites
      - id: test1
        ...
        
      # adding a header to an Azure Function request
      - id: test3
        ...
        
      # introducing rate limiting for ASP.NET Web Service
      - id: test4
        uri: https://aspnet-web-service.apps.pcfone.io
        predicates:
        - Path=/dotnet
        filters:
        - name: RequestRateLimiter
          args:
            key-resolver: "#{@userKeyResolver}"
            redis-rate-limiter.replenishRate: 1
            redis-rate-limiter.burstCapacity: 1
        - SetPath=/MyService.asmx
        - SetRequestHeader=SOAPAction, http://pivotal.io/SayHi
        - SetRequestHeader=Content-Type, text/xml
        - SetRequestHeader=Accept, text/xml
```
Here, I set the replenish rate, which is how many request per second per user, and burst capacity, which is the max number of requests in a single second. And I set the key resolver to that custom bean that reads the “user” querystring parameter. Finally, notice the three request headers.

I once again started up the Spring Cloud Gateway, and send a SOAP payload (no extra headers) to the localhost:8080/dotnet endpoint.

A single call returned the expected response. If I rapidly submitted requests in, I saw an HTTP 429 response.

So almost zero code to do some fairly sophisticated things with my gateway. None of those things involved a Java microservice, although obviously, Spring Cloud Gateway does some very nice things for Spring Boot apps.

I like this trend of microservices-machinery-as-code where I can test and deploy middleware the same way I do custom apps. The more things we can reliably deliver via automation, the more bottlenecks we can remove.
October 16, 2019
First Look: Building Java microservices with the new Azure Spring Cloud
One of the defining characteristics of the public cloud is choice. Need to host an app? On-premises, your choices were a virtual machine or a virtual machine. Now? A public cloud like Microsoft Azure offers nearly a dozen options. You’ve got spectrum of choices ranging from complete control over the stack (e.g. Azure Virtual Machines) to opinionated runtimes (e.g. Azure Functions). While some of these options, like Azure App Service, cater to custom software, no platforms cater to a specific language or framework. Until today.

At Pivotal’s SpringOne Platform conference, Microsoft took the wraps off Azure Spring Cloud. This first-party managed service sits atop Azure Kubernetes Service and helps Java developers run cloud-native apps. How? It runs managed instances of Spring Cloud components like a Config Server and Service Registry. It builds secure container images using Cloud Native Buildpacks and kpack (both, Pivotal-sponsored OSS). It offers secure binding to other Azure services like Cosmos DB. It has integrated load balancing, log streaming, monitoring, and distributed tracing. And it delivers rolling upgrades and blue-green deployments so that it’s easy to continuously deliver changes. At the moment, Azure Spring Cloud is available as a private preview, so I thought I’d give you a quick tour.

First off, since it’s a first-party service, I can provision instances via the az command line tool, or the Azure Portal. From the Azure Portal, it’s quite simple. You just provide an Azure subscription, target region, and resource name.

Once my instance was created, I accessed the unique dashboard for Azure Spring Cloud. I saw some of the standard things that are part of every service (e.g. activity log, access control), as well as the service-specific things like Apps, Config Server, Deployments, and more.

A Spring Cloud Config Server pulls and aggregates name-value pairs from various sources and serves them up to your application via a single interface. In a typical architecture, you have to host that somewhere. For Azure Spring Cloud, it’s a managed service. So, all I need to do is point this instance to a repository holding my configuration files, and I’m set.

There’s no “special” way to build Spring Boot apps that run here. They’re just … apps. So I went to start.spring.io to generate my project scaffolding. Here, I chose dependencies for web, eureka (service registry), config client, actuator (health monitoring), and zipkin + sleuth (distributed tracing). Click here to generate an identical project.

My sample code is basic here. I just expose a REST endpoint, and pull a property value from the attached configuration store.
```
@RestController
@SpringBootApplication
public class AzureBasicAppApplication {

	public static void main(String[] args) {
	  SpringApplication.run(AzureBasicAppApplication.class, args);
	}
	
	@Value("${company:Not configured by a Spring Cloud Server}")
        private String company;
	
	@GetMapping("/hello")
	public String Hello() {
	  return "hello, from " + company;
	}
}
```
To deploy, first I create an application from the CLI or Portal. “Creating” the application doesn’t deploy it, as that’s a separate step.

With that created, I packaged my Spring Boot app into a JAR file, and deployed it via the az CLI.
```
az spring-cloud app deploy -n azure-app --jar-path azure-basic-app-0.0.1-SNAPSHOT.jar
```
What happened next? Azure Spring Cloud created a container image, stored it in an Azure Container Registry, and deployed it to AKS. And you don’t need to care about any of that, as you can’t access the registry or AKS! It’s plumbing, that forms a platform. After a few moments, I saw my running instance, and the service registry shows that my instance is UP.

We’re dealing with containers here, so scaling is fast and easy. The “scale” section lets me scale up with more RAM and CPU, or out with more instances.

Cloud native, 12-factor apps should treat backing services like attached resources. Azure Spring Cloud embodies this by letting me set up service bindings. Here, I set up a linkage to another Azure service, and at runtime, its credentials and connection string are injected into the app’s configuration. All of this is handled auto-magically by the Spring Boot starters from Azure.

Logging data goes into Azure Monitor. You can set up Log Analytics for analysis, or pump out to a third party Application Performance Monitoring tool.

So you have logging, you have built-in monitoring, and you ALSO get distributed tracing. For microservices, this helps you inspect the call graph and see where your performance bottlenecks are. The pic below is from an example app built by Julien at Microsoft.

Finally, I can do blue/green deploy. This means that I deploy a second instance of the app (via the az CLI, it’s another “deployment”), can independently test it, and then choose to swap traffic over to that instance when I’m ready. If something goes wrong, I can switch back.

So far, it’s pretty impressive. This is one of the first industry examples of turning Kubernetes into an actual application platform. There’s more functionality planned as the service moves to public preview, beta, and general availability. I’m happy to see Microsoft make such a big bet on Spring, and even happier that developers have a premium option for running Java apps in the public cloud.
October 8, 2019
My Pluralsight course—Getting Started with Concourse—is now available!

Design software that solves someone’s “job to be done“, build it, package it, ship it, collect feedback, learn, and repeat. That’s the dream, right? For many, shipping software is not fun. It’s downright awful. Too many tickets, too many handoffs, and too many hours waiting. Continuous integration and delivery offer some relief, as you keep producing tested, production-ready artifacts. Survey data shows that we’re not all adopting this paradigm as fast as we should. I figured I’d do my part by preparing and delivering a new video training course about Concourse.

I’ve been playing a lot with Concourse recently, and published a 3-part blog series on using it to push .NET Core apps to Kubernetes. It’s an easy-to-use CI system with declarative pipelines and stateless servers. Concourse runs jobs on Windows or Linux, and works with any programming language you use.

My new hands-on Pluralsight course is ~90 minutes long, and gives you everything you need to get comfortable with the platform. It’s made up of three modules. The first module looks at key concepts, the Concourse architecture, and user roles, and we set up our local environment for development.

The second module digs deep into the primitives of Concourse: tasks, jobs and resources. I explain how to configure each, and then we go hands on with each. There are aspects that took me a while to understand, so I worked hard to explain these well!

The third and final module looks at pipeline lifecycle management and building manageable pipelines. We explore troubleshooting and more.

Believe it or not, this is my 20th course with Pluralsight. Over these past 8 years, I’ve switched job roles many times, but I’ve always enjoyed learning new things and sharing that information with others. Pluralsight makes that possible for me. I hope you enjoy this new course, and most importantly, start doing CI/CD for more of your workloads!

August 29, 2019
Building an Azure-powered Concourse pipeline for Kubernetes – Part 3: Deploying containers to Kubernetes
So far in this blog series, we’ve set up our local machine and cloud environment, and built the initial portion of a continuous delivery pipeline. That pipeline, built using the popular OSS tool Concourse, pulls source code from GitHub, generates a Docker image that’s stored in Azure Container Registry, and produces a tarball that’s stashed in Azure Blob Storage. What’s left? Deploying our container image to Azure Kubernetes Service (AKS). Let’s go.

Generating AKS credentials

Back in blog post one, we set up a basic AKS cluster. For Concourse to talk to AKS, we need credentials!

From within the Azure Portal, I started up an instance of the Cloud Shell. This is a hosted Bash environment with lots of pre-loaded tools. From here, I used the AKS CLI to get the administrator credentials for my cluster.
```
az aks get-credentials --name seroter-k8s-cluster --resource-group demos --admin
```
This command generated a configuration file with URLs, users, certificates, and tokens.

I copied this file locally for use later in my pipeline.

Creating a role-binding for permission to deploy

The administrative user doesn’t automatically have rights to do much in the default cluster namespace. Without explicitly allowing permissions, you’ll get some gnarly “does not have access” errors when doing most anything. Enter role-based access controls. I created a new rolebinding named “admin” with admin rights in the cluster, and mapped to the existing clusterAdmin user.
```
kubectl create rolebinding admin --clusterrole=admin --user=clusterAdmin --namespace=default
```
Now I knew that Concourse could effectively interact with my Kubernetes cluster.

Giving AKS access to Azure Container Registry

Right now, Azure Container Registry (ACR) doesn’t support an anonymous access strategy. Everything happens via authenticated users. The Kubernetes cluster needs access to its container registry, so I followed these instructions to connect ACR to AKS. Pretty easy!

Creating Kubernetes deployment and service definitions

Concourse is going to apply a Kubernetes deployment to create pods of containers in the cluster. Then, Concourse will apply a Kubernetes service to expose my pod with a routable endpoint.

I created a pair of configurations and added them to the ci folder of my source code.

The deployment looks like:
```
apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
   name: demo-app
   namespace: default
   labels:
     app: demo-app
 spec:
   replicas: 1
   template:
     metadata:
       labels:
         app: demo-app
     spec:
       containers:
       - name: demo-app
         image: myrepository.azurecr.io/seroter-api-k8s:latest
         imagePullPolicy: Always
         ports:
         - containerPort: 8080
       restartPolicy: Always 
```
This is a pretty basic deployment definition. It points to the latest image in the ACR and deploys a single instance (replicas: 1).

My service is also fairly simple, and AKS will provision the necessary Azure Load Balancer and public IP addresses.
```
 apiVersion: v1
 kind: Service
 metadata:
   name: demo-app
   namespace: default
   labels:
     app: demo-app
 spec:
   selector:
     app: demo-app
   type: LoadBalancer
   ports:
     - name: web
       protocol: TCP
       port: 80
       targetPort: 80 
```
I now had all the artifacts necessary to finish up the Concourse pipeline.

Adding Kubernetes resource definitions to the Concourse pipeline

First, I added a new resource type to the Concourse pipeline. Because Kubernetes isn’t a baked-in resource type, we need to pull in a community definition. No problem. This one’s pretty popular. It’s important than the Kubernetes client and server are expecting the same Kubernetes version, so I set the tag to match my AKS version.
```
resource_types:
- name: kubernetes
  type: docker-image
  source:
    repository: zlabjp/kubernetes-resource
    tag: "1.13"
```
Next, I had to declare my resource itself. It has references to the credentials we generated earlier.
```
resources:
- name: azure-kubernetes-service
  type: kubernetes
  icon: azure
  source:
    server: ((k8s-server))
    namespace: default
    token: ((k8s-token))
    certificate_authority: |
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
```
There are a few key things to note here. First, the “server” refers to the cluster DNS server name in the credentials file. The “token” refers to the token associated with the clusterAdmin user. For me, it’s the last “user” called out in the credentials file. Finally, let’s talk about the certificate authority. This value comes from the “certificate-authority-data” entry associated with the cluster DNS server. HOWEVER, this value is base64 encoded, and I needed a decoded value. So, I decoded it, and embedded it as you see above.

The last part of the pipeline? The job!
```
jobs:
- name: run-unit-tests
  [...]
- name: containerize-app
  [...]
- name: package-app
  [...]
- name: deploy-app
  plan:
  - get: azure-container-registry
    trigger: true
    passed:
    - containerize-app
  - get: source-code
  - get: version
  - put: azure-kubernetes-service
    params:
      kubectl: apply -f ./source-code/seroter-api-k8s/ci/deployment.yaml -f ./source-code/seroter-api-k8s/ci/service.yaml
  - put: azure-kubernetes-service
    params:
      kubectl: |
        patch deployment demo-app -p '{"spec":{"template":{"spec":{"containers":[{"name":"demo-app","image":"myrepository.azurecr.io/seroter-api-k8s:'$(cat version/version)'"}]}}}}' 
```
Let’s unpack this. First, I “get” the Azure Container Registry resource. When it changes (because it gets a new version of the container), it triggers this job. It only fires if the “containerize app” job passes first. Then I get the source code (so that I can grab the deployment.yaml and service.yaml files I put in the ci folder), and I get the semantic version.

Next I “put” to the AKS resource, twice. In essence, this resource executes kubectl commands. The first command does a kubectl apply for both the deployment and service. On the first run, it provisions the pod and exposes it via a service. However, because the container image tag in the deployment file is to “latest”, Kubernetes actually won’t retrieve new images with that tag after I apply a deployment. So, I “patched” the deployment in a second “put” step and set the deployment’s image tag to the semantic version. This triggers a pod refresh!

Deploy and run the Concourse pipeline

I deployed the pipeline as a new revision with this command:
```
fly -t rs set-pipeline -c azure-k8s-final.yml -p azure-k8s-final
```
I unpaused the pipeline and watched it start up. It quickly reached and completed the “deploy to AKS” stage.

But did it actually work? I jumped back into the Azure Cloud Shell to check it out. First, I ran a kubectl get pods command. Then, a kubectl get services command. The first showed our running pod, and the second showed the external IP assigned to my pod.

I also issued a request to that URL in the browser, and got back my ASP.NET Core API results.

Also to prove that my “patch” command worked, I ran the kubectl get deployment demo-app –output=yaml command to see which container image my deployment referenced. As you can see below, it no longer references “latest” but rather, a semantic version number.

With all of these settings, I now have a pipeline that “just works” whenever I updated my ASP.NET Core source code. It tests the code, packages it up, and deploys it to AKS in seconds. I’ve added all the pipelines we created here to GitHub so that you can easily try this all out.

Whatever CI/CD tool you use, invest in automating your path to production.
Concourse series links

Part 1 – Setup
Part 2 – Packaging and containerization
Part 3 – Deploying containers to Kubernetes
August 21, 2019
Building an Azure-powered Concourse pipeline for Kubernetes – Part 2: Packaging and containerizing code
Let’s continuously deliver an ASP.NET Core app to Kubernetes using Concourse. In part one of this blog series, I showed you how to set up your environment to follow along with me. It’s easy; just set up Azure Container Registry, Azure Storage, Azure Kubernetes Service, and Concourse. In this post, we’ll start our pipeline by pulling source code, running unit tests, generating a container image that’s stored in Azure Container Registry, and generating a tarball for Azure Blob Storage.

We’re building this pipeline with Concourse. Concourse has three core primitives: tasks, jobs, and resources. Tasks form jobs, jobs form pipelines, and state is stored in resources. Concourse is essentially stateless, meaning there are no artifacts on the server after a build. You also don’t register any plugins or extensions. Rather, the pipeline is executed in containers that go away after the pipeline finishes. Any state — be it source code or Docker images — resides in durable resources, not Concourse itself.

Let’s start building a pipeline.

Pulling source code

A Concourse pipeline is defined in YAML. Concourse ships with a handful of “known” resource types including Amazon S3, git, and Cloud Foundry. There are dozens and dozens of community ones, and it’s not hard to build your own. Because my source code is stored in GitHub, I can use the out-of-the-box resource type for git.

At the top of my pipeline, I declared that resource.
```
---
resources:
- name: source-code
  type: git
  icon: github-circle
  source:
    uri: https://github.com/rseroter/seroter-api-k8s
    branch: master
```
I’ve gave the resource a name (“source-code”) and identified where the code lives. That’s it! Note that when you deploy a pipeline, Concourse produces containers that “check” resources on a schedule for any changes that should trigger a pipeline.

Running unit tests

Next up? Build a working version of a pipeline that does something. Specifically, it should execute unit tests. That means we need to define a job.

A job has a build plan. That build plan contains any of three things: get steps (to retrieve a resource), put steps (to push something to a resource), and task steps (to run a script). Our job below has one get step (to retrieve source code), and one task (to execute the xUnit tests).
```
jobs:
- name: run-unit-tests
  plan:
  - get: source-code
    trigger: true
  - task: first-task
    config: 
      platform: linux
      image_resource:
        type: docker-image
        source: {repository: mcr.microsoft.com/dotnet/core/sdk}
      inputs:
      - name: source-code
      run:
          path: sh
          args:
          - -exec
          - |
            dotnet test ./source-code/seroter-api-k8s/seroter-api-k8s.csproj 
```
Let’s break it down. First, my “plan” gets the source-code resource. And because I set “trigger: true” Concourse will kick off this job whenever it detects a change in the source code.

Next, my build plan has a “task” step. Tasks run in containers, so you need to choose a base image that runs the user-defined script. I chose the Microsoft-provided .NET Core image so that I’d be confident it had all the necessary .NET tooling installed. Note that my task has an “input.” Since tasks are like functions, they have inputs and outputs. Anything I input into the task is mounted into the container and is available to any scripts. So, by making the source-code an input, my shell script can party on the source code retrieved by Concourse.

Finally, I embedded a short script that invokes the “dotnet test” command. If I were being responsible, I’d refactor this embedded script into an external file and reference that file. But hey, this is easier to read.

This is now a valid pipeline. In the previous post, I had you install the fly CLI to interact with Concourse. From the fly CLI, I deploy pipelines with the following command:
```
fly -t rs set-pipeline -c azure-k8s-rev1.yml -p azure-k8s-rev1
```
That command says to use the “rs” target (which points to a given Concourse instance), use the YAML file holding the pipeline, and name this pipeline azure-k8s-rev1. It deployed instantly, and looked like this in the Concourse web dashboard.

After unpausing the pipeline so that it came alive, I saw the “run unit tests” job start running. It’s easy to view what a job is doing, and I saw that it loaded the container image from Microsoft, mounted the source code, ran my script and turned “green” because all my tests passed.

Nice! I had a working pipeline. Now to generate a container image.

Producing and publishing a container image

A pipeline that just run tests is kinda weird. I need to do something when tests pass. In my case, I wanted to generate a Docker image. Another of the built-in Concourse resource types is “docker-image” which generates a container image and puts it into a registry. Here’s the resource definition that worked with Azure Container Registry:
```
resources:
- name: source-code
  [...]
- name: azure-container-registry
  type: docker-image
  icon: docker
  source:
    repository: myrepository.azurecr.io/seroter-api-k8s
    tag: latest
    username: ((azure-registry-username))
    password: ((azure-registry-password))
```
Where do you get those Azure Container Registry values? From the Azure Portal, they’re visible under “Access keys.” I grabbed the Username and one of the passwords.

Next, I added a new job to the pipeline.
```
jobs:
- name: run-unit-tests
  [...]
- name: containerize-app
  plan:
  - get: source-code
    trigger: true
    passed:
    - run-unit-tests
  - put: azure-container-registry
    params:
      build: ./source-code
      tag_as_latest: true
```
What’s this job doing? Notice that I “get” the source code again. I also set a “passed” attribute meaning this will only run if the unit test step completes successfully. This is how you start chaining jobs together into a pipeline! Then I “put” into the registry. Recall from the first blog post that I generated a Dockerfile from within Visual Studio for Mac, and here, I point to it. The resource does a “docker build” with that Dockerfile, tags the resulting image as the “latest” one, and pushes to the registry.

I pushed this as a new pipeline to Concourse:
```
fly -t rs set-pipeline -c azure-k8s-rev2.yml -p azure-k8s-rev2
```
I now had something that looked like a pipeline.

I manually triggered the “run unit tests” job, and after it completed, the “containerize app” job ran. When that was finished, I checked Azure Container Registry and saw a new repository one with image in it.

Generating and storing a tarball

Not every platform wants to run containers. BLASPHEMY! BURN THE HERETIC! Calm down. Some platforms happily take your source code and run it. So our pipeline should also generate a single artifact with all the published ASP.NET Core files.

I wanted to store this blob in Azure Storage. Since Azure Storage isn’t a built-in Concourse resource type, I needed to reference a community one. No problem finding one. For non-core resources, you have to declare the resource type in the pipeline YAML.
```
resource_types:
- name: azure-blobstore
  type: docker-image
  source:
    repository: pcfabr/azure-blobstore-resource
```
A resource type declaration is fairly simple; it’s just a type (often docker-image) and then the repo to get it from.

Next, I needed the standard resource definition. Here’s the one I created for Azure Storage:
```
name: azure-blobstore
  type: azure-blobstore
  icon: azure
  source:
    storage_account_name: ((azure-storage-account-name))
    storage_account_key: ((azure-storage-account-key))
    container: coreapp
    versioned_file: app.tar.gz
```
Here the “type” matches the resource type name I set earlier. Then I set the credentials (retrieved from the “Access keys” section in the Azure Portal), container name (pre-created in the first blog post), and the name of the file to upload. Regex is supported here too.

Finally, I added a new job that takes source code, runs a “publish” command, and creates a tarball from the result.
```
jobs:
- name: run-unit-tests
  [...]
- name: containerize-app
  [...]
- name: package-app
  plan:
  - get: source-code
    trigger: true
    passed:
    - run-unit-tests
  - task: first-task
    config:
      platform: linux
      image_resource:
        type: docker-image
        source: {repository: mcr.microsoft.com/dotnet/core/sdk}
      inputs:
      - name: source-code
      outputs:
      - name: compiled-app
      - name: artifact-repo
      run:
          path: sh
          args:
          - -exec
          - |
            dotnet publish ./source-code/seroter-api-k8s/seroter-api-k8s.csproj -o .././compiled-app
            tar -czvf ./artifact-repo/app.tar.gz ./compiled-app
            ls
  - put: azure-blobstore
    params:
      file: artifact-repo/app.tar.gz
```
Note that this job is also triggered when unit tests succeed. But it’s not connected to the containerization job, so it runs in parallel. Also note that in addition to an input, I also have outputs defined on the task. This generates folders that are visible to subsequent steps in the job. I dropped the tarball into the “artifact-repo” folder, and then “put” that file into Azure Blob Storage.

I deployed this pipeline as yet another revision:
```
fly -t rs set-pipeline -c azure-k8s-rev3.yml -p azure-k8s-rev3
```
Now this pipeline’s looking pretty hot. Notice that I have parallel jobs that fire after I run unit tests.

I once again triggered the unit test job, and watched the subsequent jobs fire. After the pipeline finished, I had another updated container image in Azure Container Registry and a file sitting in Azure Storage.

Adding semantic version to the container image

I could stop there and push to Kubernetes (next post!), but I wanted to do one more thing. I don’t like publishing Docker images with the “latest” tag. I want a real version number. It makes sense for many reasons, not the least of which is that Kubernetes won’t pick up changes to a container if the tag doesn’t change! Fortunately, Concourse has a default resource type for semantic versioning.

There are a few backing stores for the version number. Since Concourse is stateless, we need to keep the version value outside of Concourse itself. I chose a git backend. Specifically, I added a branch named “version” to my GitHub repo, and added a single file (no extension) named “version”. I started the version at 0.1.0.

Then, I ensured that my GitHub account had an SSH key associated with it. I needed this so that Concourse could write changes to this version file sitting in GitHub.

I added a new resource to my pipeline definition, referencing the built-in semver resource type.
```
- name: version  
  type: semver
  source:
    driver: git
    uri: git@github.com:rseroter/seroter-api-k8s.git
    branch: version
    file: version
    private_key: |
        -----BEGIN OPENSSH PRIVATE KEY-----
        [...]
        -----END OPENSSH PRIVATE KEY-----
```
In that resource definition, I pointed at the repo URI, branch, file name, and embedded the private key for my account.

Next, I updated the existing “containerization” job to get the version resource, use it, and then update it.
```
jobs:
- name: run-unit-tests
  [...] 
- name: containerize-app
  plan:
  - get: source-code
    trigger: true
    passed:
    - run-unit-tests
  - get: version
    params: {bump: minor}
  - put: azure-container-registry
    params:
      build: ./source-code
      tag_file: version/version
      tag_as_latest: true
  - put: version
    params: {file: version/version}
- name: package-app
  [...]
```
First, I added another ‘get” for version. Notice that its parameter increments the number by one minor version. Then, see that the “put” for the container registry uses “version/version” as the tag file. This ensures our Docker image is tagged with the semantic version number. Finally, notice I “put” the incremented version file back into GitHub after using it successfully.

I deployed a fourth revision of this pipeline using this command:
```
fly -t rs set-pipeline -c azure-k8s-rev4.yml -p azure-k8s-rev4
```
You see the pipeline, post-execution, below. The “version” resource comes into and out of the “containerize app” job.

With the pipeline done, I saw that the “version” value in GitHub was incremented by the pipeline, and most importantly, our Docker image has a version tag.

In this blog post, we saw how to gradually build up a pipeline that retrieves source and prepares it for downstream deployment. Concourse is fun and easy to use, and its extensibility made it straightforward to deal with managed Azure services. In the final blog post of this series, we’ll take pipeline-generated Docker image and deploy it to Azure Kubernetes Service.
Concourse series links

Part 1 – Setup
Part 2 – Packaging and containerization
Part 3 – Deploying containers to Kubernetes
August 20, 2019