Richard Seroter's Architecture Musings

Category: SOA

My Co-Authors Interviewed on Microsoft endpoint.tv
You want this book!

-Ron Jacobs, Microsoft

Ron Jacobs (blog, twitter) runs the Channel9 show called endpoint.tv and he just interviewed Ewan Fairweather and Rama Ramani who were co-authors on my book, Applied Architecture Patterns on the Microsoft Platform. I’m thrilled that the book has gotten positive reviews and seems to fill a gap in the offerings of traditional technology books.

Ron made a few key observations during this interview:
- As people specialize, they lose perspective of other ways to solve similar problems, and this book helps developers and architects “fill the gaps.”
- Ron found the dimensions our “Decision Framework” to be novel and of critical importance when evaluating technology choices. Specifically, evaluating a candidate architecture against design, development, operational and organizational factors can lead you down a different path than you might have expected. Ron specifically liked the “organizational direction” facet which can be overlooked but should play a key role in technology choice.
- He found the technology primers and full examples of such a wide range of technologies (WCF, WF, Server AppFabric, Windows Azure, BizTalk, SQL Server, StreamInsight) to be among the unique aspects of the book.
- Ron liked how we actually addressed candidate architectures instead of jumping directly into a demonstration of a “best fit” solution.
Have you read the book yet? If so, I’d love to hear your (good or bad) feedback. If not, Christmas is right around the corner, and what better way to spend the holidays than curling up with a beefy technology book?
December 18, 2010
Error with One-Way WSDL Operations and BizTalk Receive Locations
Do you ever do WSDL-first web service development? Regardless of the reason that you do this (e.g. you’re an architectural-purist, your mother didn’t hold you enough), this style of service design typically works fine with BizTalk Server solutions. However, if you decide to build a one-way input service, you’ll encounter an annoying, but understandable error.

Let’s play this scenario out. I’ve hand-built a WSDL that takes in an “employee update” message through a one-way service. That is, no response is needed by the party that invokes the service.

The topmost WSDL node defines some default namespace values and then has a type declaration which describes our schema.
```
<wsdl:definitions name="EmployeeUpdateService"
targetNamespace="http://Seroter.OneWayWsdlTest.EmployeeProcessing"
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns:tns="http://Seroter.OneWayWsdlTest.EmployeeProcessing">
  
  <wsdl:types>
    <xs:schema xmlns="http://Seroter.OneWayWsdlTest.EmployeeProcessing" xmlns:b="http://schemas.microsoft.com/BizTalk/2003" targetNamespace="http://Seroter.OneWayWsdlTest.EmployeeProcessing" xmlns:xs="http://www.w3.org/2001/XMLSchema">
      <xs:element name="EmployeeUpdate">
        <xs:complexType>
          <xs:sequence>
            <xs:element name="EmpId" type="xs:string" />
            <xs:element name="UpdateType" type="xs:string" />
            <xs:element name="DateUpdated" type="xs:string" />
          </xs:sequence>
        </xs:complexType>
      </xs:element>
    </xs:schema>
  </wsdl:types>
```
Next, I defined my input message, port type with an operation that accepts that message, and then a binding that uses that port type.
```

  <wsdl:message name="Request">
    <wsdl:part name="part" element="tns:EmployeeUpdate" />
  </wsdl:message>
  
  <wsdl:portType name="EmployeeUpdate_PortType">
    <wsdl:operation name="PublishEmployeeRequest">
      <wsdl:input message="tns:Request" />
    </wsdl:operation>
  </wsdl:portType>
  
  <wsdl:binding name="EmployeeUpdate_Binding" type="tns:EmployeeUpdate_PortType">
    <soap:binding transport="http://schemas.xmlsoap.org/soap/http"/>
    <wsdl:operation name="PublishEmployeeRequest">
      <soap:operation soapAction="PublishEmployeeRequest"
      style="document"/>
      <wsdl:input>
        <soap:body use ="literal"/>
      </wsdl:input>
    </wsdl:operation>
  </wsdl:binding>
```
Finally, I created a service declaration that has an endpoint URL selected.
```

  <wsdl:service name="EmployeeUpdateService">
    <wsdl:port binding="tns:EmployeeUpdate_Binding" name="EmployeeUpdatePort">
      <soap:address location="http://localhost:8087/EmployeeUpdateService/Service.svc"/>
    </wsdl:port>
  </wsdl:service>
</wsdl:definitions>
```
I copied this WSDL to the root of my web server that so that it has a URL that can be referenced later.

Let’s jump into a BizTalk project now. Note that if you design a service this way (WSDL-first), you CAN use the BizTalk WCF Service Consuming Wizard to generate the schemas and orchestration messaging ports for a RECEIVE scenario. We typically use this wizard to build artifacts to consume a service, but this actually works pretty well for building services as well. Anyway, I’m going to take the schema definition from my WSDL and manually create a new XSD file.

This is the only artifact I need to develop. I deployed the BizTalk project and switched to the BizTalk Administration Console where I will build a receive port/location that hosts a WCF endpoint. First though, I created a one-way Send Port which subscribes to my message’s type property and emits the file to disk.

Next I added a new one-way receive port that will host the service. It uses the WCF-Custom adapter so that I can host the service in-process instead of forcing me to physically build a service to reside in IIS.

On the General tab I set the address to the value from the WSDL (http://localhost:8087/EmployeeUpdateService/Service.svc). On the Binding tab I chose the basicHttpBinding. Finally, on the Behavior tab, I added a Service Behavior and selected the serviceMetadata behavior from the list. I set the externalMetadataLocation to the URL of my custom WSDL and flipped the httpGetEnabled value to True.

If everything is configured correctly, the receive location is started, and the BizTalk host is started (and thus, the WCF service host is opened), I can hit the URL of my BizTalk endpoint and see the metadata page.

All that’s left to do is consume this service. Instead of building a custom application that calls this service, I can leverage the WCF Test Client that ships with the .NET Framework. After adding a reference to my BizTalk-hosted service, and invoking the service, two things happened. First, the message is successfully processed by BizTalk and a file is dropped to disk (via my Send Port). But secondly, and most important, my service call resulted in an error:

The one-way operation returned a non-null message with Action=”.

Yowza. While I could technically catch that error in code and just ignore it (since BizTalk processed everything just fine), that’d be pretty lazy. We want to know why this happened! I got this error because a“one way” BizTalk receive location still sends a message back to the caller and my service client wasn’t expecting it. A WSDL file with a true one-way operation results in a WCF client that expects an IsOneWay=true interaction pattern. However, BizTalk doesn’t support true one-way interactions. It supports operations that return no data (e.g. “void”) only. So, by putting a hand-built WSDL that demanded an asynchronous service on a BizTalk receive location that cannot support it, we end up with a mismatch.

How do I fix this? Actually, it’s fairly simple. I returned to my hand-built WSDL and added a new, empty message declaration.
```

  <wsdl:message name="Request">
    <wsdl:part name="part" element="tns:EmployeeUpdate" />
  </wsdl:message>
  <wsdl:message name="Response" />
```
I then made that message the output value of my operation in both my port type and binding.
```

  <wsdl:portType name="EmployeeUpdate_PortType">
    <wsdl:operation name="PublishEmployeeRequest">
      <wsdl:input message="tns:Request" />
      <wsdl:output message="tns:Response" />
    </wsdl:operation>
  </wsdl:portType>
  
  <wsdl:binding name="EmployeeUpdate_Binding" type="tns:EmployeeUpdate_PortType">
    <soap:binding transport="http://schemas.xmlsoap.org/soap/http"/>
    <wsdl:operation name="PublishEmployeeRequest">
      <soap:operation soapAction="PublishEmployeeRequest"
      style="document"/>
      <wsdl:input>
        <soap:body use ="literal"/>
      </wsdl:input>
      <wsdl:output>
        <soap:body use ="literal"/>
      </wsdl:output>
    </wsdl:operation>
  </wsdl:binding>
```
After copying the WSDL back to IIS (so that my service’s metadata was up to date), I refreshed the service in the WCF Test Client. I called the service again, and this time, got no error while the file was once again successfully written to disk by the send port.

BizTalk Server, and the .NET Framework in general, have decent, but not great support for WSDL-first development. Therefore, it’s wise to be aware of any gotchas or quirks when going this route.
December 5, 2010
Metadata Handling in BizTalk Server 2010 AppFabric Connect for Services

Microsoft just announced the new BizTalk Server 2010 AppFabric Connect for Services which is a set of tools used to expose BizTalk services to the cloud. Specifically, you can expose BizTalk endpoints and LOB adapter endpoints to the Azure Service Bus.

The blog post linked to above has a good overview, but seemed to leave out a lot of details. So, I downloaded, installed and walked through some scenarios and thought I’d share some findings. Specifically, I want to show how service metadata for BizTalk endpoints is exposed to cloud consumers. This has been a tricky thing up until now since AppFabric endpoints don’t respect the WCF metadataBehavior so you couldn’t just expose BizTalk receive location metadata without some slight of hand. I’ve shown previously how you could just handcraft a WSDL and use it in the port and with Azure clients, but that’s a suboptimal solution.

First off, I built a simple schema that I will expose as a web service.

Next up, I started the BizTalk WCF Service Publishing Wizard and noticed the new wording that came with installing the BizTalk Server 2010 Feature Pack.

Interesting. Next up, I’m asked about creating my on-premises service endpoint and optionally, a receive location and a new on-premise metadata exchange point.

On the next wizard page, I’m able to optionally choose to extend this service to the Azure AppFabric cloud.

After this, I choose whether to expose an orchestration or schemas as a web service. I chose to expose schemas as a service (i.e., build a service from scratch vs. using orchestration ports and messages to auto-produce a service).

As you can see, I have a one-way service to publish order messages. Following this screen is the same “choose an on-premises location” page where you set the IIS directory for the new service.

After this wizard page is a new one where you set up a Service Bus endpoint. You pick which Service Bus binding that you want and apply your own service namespace. You can then choose to enable both a discovery behavior and a metadata exchange behavior.

Finally, we apply our Service Bus credentials for listening to the cloud. Notice that I force authentication for the service endpoint itself, but not the metadata retrieval.

After the wizard is done, what I expected to see was a set of receive locations. However, the only receive location that I have is the one using the on-premises WCF binding.

What the what? I expected to see a receive location that used the Service Bus binding. So what happened to all those configuration values I just set? If you open up the WCF service created in IIS, you can see a whole host of Service Bus configuration settings. First, we see that there are now three service endpoints. The three service endpoints below include an on-premises MEX endpoint, a RelayEndpoint that binds to the cloud, and a MEX endpoint that binds to the cloud.

That’s a pretty smart way to go. Instead of trying to hack up the receive location, Microsoft instead beefed up the proxy services to do all the cloud binding.

I can use IIS 7.5 autostart to make sure that my cloud binding occurs as soon as the service starts (vs. waiting for the first invocation). Once my receive location and service are started, I can hit my local service, and, see my service is also in my cloud registry.

If I drill into my service, I can also see my primary service and my MEX endpoint.

When I click the primary service name in the Azure AppFabric registry, I get an HTTP 401 (unauthorized) error which makes sense since we have a client authentication requirement on this service.

If I click the MEX endpoint, I get a weird error. I seem to recall that you can’t retrieve a MEX WSDL over HTTP. Or maybe I’m crazy. But, to test that my MEX endpoint really works, I plugged the MEX URL into an Add Service Reference window in Visual Studio.NET, and sure enough, it pulls back the metadata for my BizTalk-exposed service.

All that said, this looks really promising. Seems like a smart decision to stay away from the receive location and move cloud configuration to the WCF service where it belongs.

October 28, 2010
Behavior of BizTalk WCF Publishing Wizard When Creating Multiple Operations or Services at Once

So what happens when you create a few operations or services during a single instance of the BizTalk WCF Service Publishing wizard? What types of web service projects and BizTalk messaging configurations get produced? I was messing around with this the other day, and thought I’d quickly show the output. Maybe this is common knowledge, but I hadn’t tried all these scenarios before.

Let’s assume you have a schema to use for the wizard, and a BizTalk application to hold the generated ports. In this first scenario, I’m creating two operations for the single service. I started out the wizard by saying that I wanted my receive locations (notice that “locations” is plural here) in a particular BizTalk application.

I then decided to build a service from existing schemas instead of an orchestration. I’ve created two one-way service operations on the root service.

I accept the rest of the wizard’s default values and complete this wizard instance.   The first thing that I end up with is a web service application in IIS that contains a single service file.

When I browse that service, I can clearly see in its metadata that there are two operations available for consumption.

On the BizTalk side, the application has a single new receive port and a single receive location that points to my web service.

In my next run through the wizard, I’ve once again selected to have a metadata endpoint enabled and to put my receive locations in a particular BizTalk application. I’ve also chosen to build my services from schemas. In this case, I’ve created two services, each with a single one-way operation underneath.

With all the other default values selected (but using a different output service location), I completed the wizard. As you’d probably expect, my generated web service application now has two services within it.

The generated BizTalk output surprised me slightly. What I ended up with where a distinct receive port and receive location for each service. I was expecting to have a single receive port with two locations. I could see why they’d have to be different if the exchange pattern (two-way vs. one-way) was different for the services, but both of my services are one-way, and could theoretically live fine in the same receive port.

I can only guess that the reason for doing this is because folks could use the single wizard instance to build completely unrelated services while trying to save the time of opening a new wizard for each new service.

To be thorough, let’s compare this against a multi-operation orchestration-generated service. I’ve built an orchestration that has two (public) receive ports.

After deploying this and starting the BizTalk WCF Service Publishing Wizard again, I chose to build a service from an orchestration. Here I get a different interface and both of my public ports are shown. I’ve NOT selected the “merge ports into a single service” option for this pass through the wizard.

With the wizard complete, I confirmed that I have a web service application with two services, and a BizTalk application containing two receive ports, each with a single receive location.

In my last pass through the wizard, I’ve again chosen to build a service from an orchestration and chose to merge my selected ports.    I have to change my orchestration itself before I proceed since my orchestration used the default Operation_1 operation name for each port and I get an error from the wizard saying that I cannot merge the operation because of the name collision. After dealing with that unpleasantness, I completed the wizard instance with my ports merged.

What did I get? I have a single service in my IIS web application. and a single receive port with a single receive location in my BizTalk application.

The wizard works consistently whether you are building services from schemas or orchestrations.   It’d be great if you had a choice to merge ports from a BizTalk messaging sense in addition to the orchestration sense, but such is life.

Do any of you use the wizard to build up a whole set of services at once, or do you typically fly through it for each individual service?

October 22, 2010
Book’s Sample Chapter, Articles and Press Release

The book is now widely available and our publisher is starting up the promotion machine. At the bottom of this post is the publisher’s press release. Also, we now have one sample chapter online (Mike Sexton’s Debatching Bulk Data) as well as two articles representing some of the material from my Content Based Routing chapter (Part 1 – Content Based Routing on the Microsoft Platform, Part II – Building the Content Based Routing Solution on the Microsoft Platform). This hopefully provides a good sneak peak into the book’s style.

## PRESS RELEASE ##

Solve business problems on the Microsoft application platform using Packt’s new book

Applied Architecture Patterns on the Microsoft Platform is a new book from Packt that offers an architectural methodology for choosing Microsoft application platform technologies. Written by a team of specialists in the Microsoft space, this book examines new technologies such as Windows Server AppFabric, StreamInsight, and Windows Azure Platform, and their application in real-world solutions.

Filled with live examples on how to use the latest Microsoft technologies, this book guides developers through thirteen architectural patterns utilizing code samples for a wide variety of technologies including Windows Server AppFabric, Windows Azure Platform AppFabric, SQL Server (including Integration Services, Service Broker, and StreamInsight), BizTalk Server, Windows Communication Foundation (WCF), and Windows Workflow Foundation (WF).

This book is broken down into 4 different sections. Part 1 starts with getting readers up to speed with various Microsoft technologies. Part 2 concentrates on messaging patterns and the inclusion of use cases highlighting content-based routing. Part 3 digs into bulk data processing, and multi-master synchronization. Finally the last part covers performance-related patterns including low latency, failover to the cloud, and reference data caching.

Developers can learn about the core components of BizTalk Server 2010, with an emphasis on BizTalk Server versus Windows Workflow and BizTalk Server versus SQL Server. They will not only be in a position to develop their first Windows Azure Platform AppFabric, and SQL Azure applications but will also learn to master data management and data governance of SQL Server Integration Services, Microsoft Sync Framework, and SQL Server Service Broker.

Architects, developers, and managers wanting to get up to speed on selecting the most appropriate platform for a particular problem will find this book to be a useful and beneficial read. This book is out now and is available from Packt. For more information, please visit the site.

[Cross posted on Book’s dedicated website]

September 17, 2010
And … The New Book is Released

Nearly 16 months after a book idea was born, the journey is now complete. Today, you can find our book, Applied Architecture Patterns on the Microsoft Platform, in stock at Amazon.com and for purchase and download at the Packt Publishing site.

I am currently in Stockholm along with co-authors Stephen Thomas and Ewan Fairweather delivering a 2 day workshop for the BizTalk User Group Sweden. We’re providing overviews of the core Microsoft application platform technologies and then excerpting the book to show how we analyzed a particular use case, chose a technology and then implemented it. It’s our first chance to see if this book was a crazy idea, or actually useful. So far, the reaction has been positive. Of course, the Swedes are such a nice bunch that they may just be humoring me.

I have absolutely no idea how this book will be received by you all. I hope you find it to be a unique tool for evaluating architecture and building solutions on Microsoft technology. If you DON’T like it, then I’ll blame this book idea on Ewan.

September 8, 2010
Do you know the Microsoft Customer Advisory Teams? You should.

For those who live and work with Microsoft application platform technologies, the Microsoft Customer Advisory Teams (CAT) are a great source of real-world info about products and technology. These are the small, expert-level teams whose sole job is to make sure customers are successful with Microsoft technology. Last month I had the pleasure of presenting to both the SQL CAT and Server AppFabric CAT teams about blogging and best practices and thought I’d throw a quick plug out for these groups here.

First off, the SQL CAT team (dedicated website here) has a regular blog of best practices, and link to the best whitepapers for SQL admins, architects, and developers. I’m not remotely a great SQL Server guy, but I love following this team’s work and picking up tidbits that make me slightly more dangerous at work. If you actually need to engage these guys on a project, contact your Microsoft rep.

As for the Windows Server AppFabric CAT team, they also have a team blog with great expert content. This team, which contains the artists-formerly-known-as-BizTalk-Rangers, provides deep expertise on BizTalk Server, Windows Server AppFabric, WCF, WF, AppFabric Caching and StreamInsight. You’ll find a great bunch of architects on this team including Tim Wieman, Mark Simms, Rama Ramani, Paolo Salvatori and more, all led by Suren Machiraju and the delightfully frantic Curt Peterson. They’ve recently produced posts about using BizTalk with the AppFabric Service Bus, material on the Entity Framework, and a ridiculously big and meaty post from Mark Simms about building StreamInsight apps.

I highly recommend subscribing to both these team blogs and following SQL CAT on twitter (@sqlcat).

August 26, 2010
Using “Houston” to Manage SQL Azure Databases

Up until now, your only option for managing SQL Azure cloud databases was using an on-premise SQL Server Management Console and pointing to your cloud database. The SQL Azure team has released a CTP of “Houston” which is a web-based, Silverlight environment for doing all sorts of stuff with your SQL Azure database. Instead of just telling you about it, I figured I’d show it.

First, you need to create a SQL Azure database (assuming that you don’t already have one). Mine is named SeroterSample. I’m feeling very inspired this evening.

Next up, we make sure to have a firewall rule allowing Microsoft services to access the database.

After this, we want to grab our database connection details via the button at the bottom of the Databases view.

Now go to the SQL Azure labs site and select the Project Houston CTP 1 tab.

We then see a futuristic console which either logs me into project Houston or launches a missile.

If the login is successful, we get the management dashboard. It contains basic management operations at the top (“new table”, “new stored procedure”, “open query”, etc), a summary of database schema objects on the left, and an unnecessary but interesting “cube of info” in the middle.

The section in the middle (aka “cube of info”) rotates as you click the arrows and shows various data points. Hopefully a future feature includes a jack-in-the-box that pops out of the top.

I chose to create a new table in my database. We are shown an interface where we build up our table structure by choosing columns, data types, default values, data types and more.

After creating a few columns and renaming my table, I clicked the Save button on the top left of the screen to commit my changes. I can now see my table in the list of artifacts belonging to my database.

It’s great to have a table, but let’s put some data into that bad boy. Clicking the table name re-opens the design view by default. We can select the Data view at the top to actually add rows to our table.

I’m not exactly sure how to delete artifacts except through manual queries. For kicks and giggles I clicked the New View option, and when I canceled out of it, I still ended up with a view in the artifact list. Right-clicking is not something that is available anywhere in the application, and there was no visible way to delete the view short of create a new Query and deleting it from there. That said, when I logged out and logged back in, the view was no longer there. So, because I didn’t explicitly save it, the view was removed when I disconnected.

All in all, this is a fine, light-weight management interface for our cloud database. It wasn’t until I was halfway through my demonstration that I realized that I did all my interactions on the portal through a Chrome browser. Cross-browser stuff is much more standard now, but, still nice to see.

Because I have no confidence that my Azure account is accurately tied to my MSDN Subscription, I predict that this demonstration has cost me roughly $14,000 in Azure data fees. You all are worth it though.

July 22, 2010
I’m Heading to Sweden to Deliver a 2-Day Workshop

The incomparable Mikael Håkansson has just published the details of my next visit to Sweden this September. After I told Mikael about my latest book, we thought it might be epic to put together a 2 day workshop that highlights the “when to use what” discussion. Two of my co-authors, Stephen Thomas and Ewan Fairweather, will be joining me for busy couple of days at the Microsoft Sweden office. This is the first time that Stephen and Ewan have seen my agenda, so, surprise guys!

We plan to summarize each core technology in the Microsoft application platform and then dig into six of the patterns that we discuss in the book. I hope this is a great way to introduce a broad audience to the nuances of each technology and have a spirited discussion of how to choose the best tool for a given situation.

If other user groups would be interested in us repeating this session, let me know. We take payment in the form of plane tickets, puppies or gold bullion.

June 14, 2010
Announcing My New Book: Applied Architecture Patterns on the Microsoft Platform

So my new book is available for pre-order here and I’ve also published our companion website. This is not like any technical book you’ve read before. Let me back up a bit.

Last May (2009) I was chatting with Ewan Fairweather of Microsoft and we agreed that with so many different Microsoft platform technologies, it was hard for even the most ambitious architect/developer to know when to use which tool. A book idea was born.

Over the summer, Ewan and I started crafting a series of standard architecture patterns that we wanted to figure out which Microsoft tool solved best. We also started the hunt for a set of co-authors to bring expertise in areas where we were less familiar. At the end of the summer, Ewan and I had suckered in Stephen Thomas (of BizTalk fame), Mike Sexton (top DB architect at Avanade) and Rama Ramani (Microsoft guy on AppFabric Caching team). All of us finally pared down our list of patterns to 13 and started off on this adventure. Packt Publishing eagerly jumped at the book idea and started cracking the whip on the writing phase.

So what did we write? Our book starts off by briefly explaining the core technologies in the Microsoft application platform including Windows Workflow Foundation, Windows Communication Foundation, BizTalk Server, SQL Server (SSIS and Service Broker), Windows Server AppFabric, Windows Azure Platform and StreamInsight. After these “primer” chapters, we have a discussion about our Decision Framework that contains our organized approach to assessing technology fit to a given problem area. We then jump into our Pattern chapters where we first give you a real world use case, discuss the pattern that would solve the problem, evaluate multiple candidate architectures based on different application technologies, and finally select a winner prior to actually building the “winning” solution.

In this book you’ll find discussion and deep demonstration of all the key parts of the Microsoft application platform. This book isn’t a tutorial on any one technology, but rather, it’s intended to provide the busy architect/developer/manager/executive with an assessment of the current state of Microsoft’s solution offerings and how to choose the right one to solve your problem.

This is a different kind of book. I haven’t seen anything like it. Either you will love it or hate it. I sincerely hope it’s the former, as we’ve spent over a year trying to write something interesting, had a lot of fun doing it, and hope that energy comes across to the reader.

So go out there and pre-order, or check out the site that I set up specifically for the book: http://AppliedArchitecturePatterns.com.

I’ll be sure to let you all know when the book ships!

June 3, 2010