Building a long-running, serverless, event-driven system with as little code as possible

Is code a liability or an asset? What it does should be an asset, of course. But there’s a cost to running and maintaining code. Ideally, we take advantage of (managed) services that minimize how much code we have to write to accomplish something.

What if I want to accept document from a partner or legacy business system, send out a request for internal review of that document, and then continue processing? In ye olden days, I’d build file watchers, maybe a database to hold state of in-progress reviews, a poller that notified reviewers, and a web service endpoint to handle responses and update state in the database. That’s potentially a lot of code. Can we get rid of most that?

Google Cloud Workflows recently added a “callback” functionality which makes it easier to create long-running processes with humans in the middle. Let’s build out an event-driven example with minimal code, featuring Cloud Storage, Eventarc, Cloud Workflows, and Cloud Run.

Step 1 – Configure Cloud Storage

Our system depends on new documents getting added to a storage location. That should initiate the processing. Google Cloud Storage is a good choice for an object store.

I created a new bucket named “loan-application-submissions’ in our us-east4 region. At the moment, the bucket is empty.

Step 2 – Create Cloud Run app

The only code in our system is the application that’s used to review the document and acknowledge it. The app accepts a querystring parameter that includes the “callback URL” that points to the specific Workflow instance waiting for the response.

I built a basic Go app with a simple HTML page, and a couple of server-side handlers. Let’s go through the heart of it. Note that the full code sample is on GitHub.

func main() {

	fmt.Println("Started up ...")

	e := echo.New()
	e.Use(middleware.Logger())
	e.Use(middleware.Recover())

	t := &Template{
		Templates: template.Must(template.ParseGlob("web/home.html")),
	}

	e.Renderer = t
	e.GET("/", func(c echo.Context) error {
		//load up object with querystring parameters
		wf := workflowdata{LoanId: c.QueryParam("loanid"), CallbackUrl: c.QueryParam("callbackurl")}

		//passing in the template name (not file name)
		return c.Render(http.StatusOK, "home", wf)
	})

	//respond to POST requests and send message to callback URL
	e.POST("/ack", func(c echo.Context) error {
		loanid := c.FormValue("loanid")
		fmt.Println(loanid)
		callbackurl := c.FormValue("callbackurl")

		fmt.Println("Sending workflow callback to " + callbackurl)

		wf := workflowdata{LoanId: loanid, CallbackUrl: callbackurl}

		// Fetch an OAuth2 access token from the metadata server
		oauthToken, errAuth := metadata.Get("instance/service-accounts/default/token")
		if errAuth != nil {
			fmt.Println(errAuth)
		}

		//load up oauth token
		data := OAuth2TokenInfo{}
		errJson := json.Unmarshal([]byte(oauthToken), &data)
		if errJson != nil {
			fmt.Println(errJson.Error())
		}
		fmt.Printf("OAuth2 token: %s", data.Token)

		//setup callback request
		workflowReq, errWorkflowReq := http.NewRequest("POST", callbackurl, strings.NewReader("{}"))
		if errWorkflowReq != nil {
			fmt.Println(errWorkflowReq.Error())
		}

		//add oauth header
		workflowReq.Header.Add("authorization", "Bearer "+data.Token)
		workflowReq.Header.Add("accept", "application/json")
		workflowReq.Header.Add("content-type", "application/json")

		//inboke callback url
		client := &http.Client{}
		workflowResp, workflowErr := client.Do(workflowReq)

		if workflowErr != nil {

			fmt.Printf("Error making callback request: %s\n", workflowErr)
		}
		fmt.Printf("Status code: %d", workflowResp.StatusCode)

		return c.Render(http.StatusOK, "home", wf)
	})

	//simple startup
	e.Logger.Fatal(e.Start(":8080"))
}

The “get” request shows the details that came in via the querystrings. The “post” request generates the required OAuth2 token, adds it to the header, and calls back into Google Cloud Workflows. I got stuck for a while because I was sending an ID token and the service expects an access token. There’s a difference! My colleague Guillaume Laforge, who doesn’t even write Go, put together the code I needed to generate the necessary OAuth2 token.

From a local terminal, I ran a single command to push this source code into our fully managed Cloud Run environment:

gcloud run deploy

After a few moments, the app deployed and I loaded it up the browser with some dummy querystring values.

Step 3 – Create Workflow with event-driven trigger

That was it for coding! The rest of our system is composed of managed services. Specifically, Cloud Workflows, and Eventarc which processes events in Google Cloud and triggers consumers.

I created a new Workflow called “workflow-loans” and chose the new “Eventarc” trigger. This means that the Workflow starts up as a result of an event happening elsewhere in Google Cloud.

A new panel popped up and asked me to name my trigger and pick a source. We offer nearly every Google Cloud service as a source for events. See here that I chose Cloud Storage. Once I chose the event provider, I’m offered a contextual set of events. I selected the “finalized” event which fires for any new object added to the bucket.

Then, I’m asked to choose my storage bucket, and we have a nice picker interface. No need to manually type it in. Once I chose my bucket, which resides in a different region from my Workflow, I’m told as much.

The final step is to add the Workflow definition itself. These can be in YAML or JSON. My Workflow accepts some arguments (properties of the Cloud Storage doc, including the file name), and runs through a series of steps. It extracts the loan number from file name, creates a callback endpoint, logs the URL, waits for a callback, and processes the response.

The full Workflow definition is below, and also in my GitHub repo.

main:
    params: [args]
    steps:
        - setup_variables:
            #define and assign variables for use in the workflow
            assign:
                - version: 100                  #can be numbers
                - filename: ${args.data.name}   #name of doc
        - log_receipt:
            #write a log to share that we started up
            call: sys.log          
            args:
                text: ${"Loan doc received"}
        - extract_loan_number:
            #pull out substring containing loan number
            assign:
                - loan_number : ${text.substring(filename, 5, 8)}
        - create_callback:
            #establish a callback endpoint
            call: events.create_callback_endpoint
            args:
                http_callback_method: "POST"
            result: callback_details
        - print_callback_details:
            #print out formatted URL
            call: sys.log
            args:
                severity: "INFO"
                # update with the URL of your Cloud Run service
                text: ${"Callback URL is https://[INSERT CLOUD RUN URL HERE]?loanid="+ loan_number +"&callbackurl=" + callback_details.url}
        - await_callback:
            #wait impatiently
            call: events.await_callback
            args:
                callback: ${callback_details}
                timeout: 3600
            result: callback_request
        - print_callback_request:
            #wlog the result
            call: sys.log
            args:
                severity: "INFO"
                text: ${"Received " + json.encode_to_string(callback_request.http_request)}
        - return_callback_result:
            return: ${callback_request.http_request}

I deployed the Workflow which also generated the Eventarc trigger itself.

Step 4 – Testing it all out

Let’s see if this serverless, event-driven system now works! To start, I dropped a new PDF named “loan600.pdf” into the designated Storage bucket.

Immediately, Eventarc triggered a Workflow instance because that PDF was uploaded to Cloud Storage. See that the Workflow instance in an “await_callback” stage.

On the same page, notice the logs for the Workflow instance, including the URL for my Cloud Run with all the right querystring parameters loaded.

I plugged that URL into my browser and got my app loaded with the right callback URL.

After clicking the “acknowledge loan submission” button which called back to my running Workflow instance, I switched back to Cloud Workflows and saw that my instance completed successfully.

Summary

There are many ways to solve the problem I called out here. I like this solution. By using Google Cloud Eventarc and Workflows, I eliminated a LOT of code. And since all these services, including Cloud Run, are fully managed serverless services, it only costs me money when it does something. When idle, it costs zero. If you follow along and try it for yourself, let me know how it goes!

Author: Richard Seroter

Richard Seroter is Director of Developer Relations and Outbound Product Management at Google Cloud. He’s also an instructor at Pluralsight, a frequent public speaker, the author of multiple books on software design and development, and a former InfoQ.com editor plus former 12-time Microsoft MVP for cloud. As Director of Developer Relations and Outbound Product Management, Richard leads an organization of Google Cloud developer advocates, engineers, platform builders, and outbound product managers that help customers find success in their cloud journey. Richard maintains a regularly updated blog on topics of architecture and solution design and can be found on Twitter as @rseroter.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.