Author: Richard Seroter

I was asked a couple days ago whether it was possible to receive a related but disjointed set of files into BizTalk Server and both aggregate and reorder them prior to passing the result to a web service.  Below is small sample I put together to demonstrate that it was indeed possible.

You can find some other resequencer patterns (most notably, in the Pro BizTalk Server 2006 book), but I was looking for something fairly simple and straightforward.  My related messages all come into BizTalk at roughly the same time, and, there are no more than 20 in a related batch.

Let’s first take a look at a simplified version of the schema I’m working with.

I’ve highlighted a few header values.  I know the unique ID of the batch of related records (which is a promoted value), how many items are in the batch, and the position of this individual message in the batch sequence.  These are crucial for creating the singleton, and being able to reorder the messages later on.  The message payload is a description of a document.  This same schema is used for the “aggregate” message because the “Document” node has an unbounded occurrence limit.

I need a helper component which stores, sorts and combines my batch messages.  My class starts out like this:

Notice that I’m using a SortedDictionary class which is going to take the integer-based sequence number as the “key” and an XML document as the “value.”  The SortedDictionary is pretty cool in that it will automatically sort my list based on the key.  No extra work needed on my part.  I’ve also got a couple member variables that hold values universal to the entire batch of records.  I accept those values in the constructor.

Next, I have an operation to take an inbound XML document and add it to the list.

You can see that I yank out the document-specific “SequenceID” and use that value as the “key” in the SortedDictionary.

Next I created an “aggregation” function which drains the SortedDictionary and creates a single XML message that jams all the “Document” nodes into a repeating collection.

As you can see, I extract values from the dictionary using a “for-each” loop and a KeyValuePair object.  I then create a new “Document” node, and suck out the guts of the dictionary value and slap it in there.

Now I can build my BizTalk singleton.  Because we promoted the “BatchID” value, I can create a correlation set based on it.  My initial receive shape takes in a “BatchRecord” message and initializes the correlation set.  In the “Set Variables” Expression Shape, I instantiate my loop counters (index at 1 and maximum based on the “BatchCount” distinguished field), and the helper class by passing in the “BatchID” and “BatchCount” to the constructor.  In the “AddDocToBatch” Expression Shape, I set my message equal to a variable of type “XmlDocument”, and pass that variable to the “AddDocumentToDictionary” method of my helper class.

Next, I have a loop where I receive the (following correlation) “BatchRecord” message, once again call “AddDocumentToDictionary”, and finally increment my loop counter.

Finally, I create the “BatchResult” message (same message type as the “BatchRecord”) by setting it equal to the result of the “GetAggregateDocument” method of the helper class.  Then, I send the message out of the orchestration.

So, if I drop in 5 messages at different times and completely out of order (e.g. sequence 3, 5, 4, 2, 1), I get the following XML output from the BizTalk process:

As you can see, all the documents show up in the correct order.

Some parting thoughts: this pattern clearly doesn’t scale as the number of items in a batch increases.  Because the batch aggregate is kept in memory, you will run into issues if either (a) the batch messages come in over a long period of time or (b) there are lots of messages in a batch.  If either case is true, you would want to consider stashing the batch records in an external storage (e.g. database) and doing the sorting and mashing at that layer.

Any other thoughts you wish to share?

Technorati Tags: BizTalk

Today, one of our BizTalk developers asked me how to take a delimited string stored in a single node, and extract all those values into separate destination nodes.  I put together a quick XSLT operation that makes this magic happen.

So let’s say I have a source XML structure like this:

I need to get this pipe-delimited value into an unbounded destination node.  Specifically, the above XML should be reshaped into the format here:

Notice that each pipe-delimited value is in its own “value” node.  Now I guess I could chained together 62 functoids to make this happen, but it seemed easier to write a bit of XSLT that took advantage of recursion to split the delimited string and emit the desired nodes.

My map has a scripting functoid that accepts the three values from the source (included the pipe-delimited “values” field) and maps to a parent destination record.

Because I want explicit input variables  to my functoid (vs. traversing the source tree just to get the individual nodes I need), I’m using the “Call Templates” action of the Scripting functoid.

My XSLT script is as follows:

<!-- This template accepts three inputs and creates the destination 
"Property" node.  Inside the template, it calls another template which 
builds up the potentially repeating "Value" child node -->
<xsl:template name="WritePropertyNodeTemplate">
<xsl:param name="name" />
<xsl:param name="type" />
<xsl:param name="value" />

<!-- create property node -->
<Property>
<!-- create single instance children nodes -->
<Name><xsl:value-of select="$name" /></Name>
<Type><xsl:value-of select="$type" /></Type>

<!-- call splitter template which accepts the "|" separated string -->
<xsl:call-template name="StringSplit">
<xsl:with-param name="val" select="$value" />
</xsl:call-template>
</Property>
</xsl:template>

<!-- This template accepts a string and pulls out the value before the 
designated delimiter -->
<xsl:template name="StringSplit">
<xsl:param name="val" />

<!-- do a check to see if the input string (still) has a "|" in it -->
<xsl:choose>
  <xsl:when test="contains($val, '|')">
   <!-- pull out the value of the string before the "|" delimiter -->
   <Value><xsl:value-of select="substring-before($val, '|')" /></Value>
     
     <!-- recursively call this template and pass in 
value AFTER the "|" delimiter -->
     <xsl:call-template name="StringSplit">
     <xsl:with-param name="val" select="substring-after($val, '|')" />
     </xsl:call-template>

  </xsl:when>
  <xsl:otherwise>
      <!-- if there is no more delimiter values, print out 
the whole string -->
      <Value><xsl:value-of select="$val" /></Value>
   </xsl:otherwise>
</xsl:choose>

</xsl:template>

Note that I use recursion to call the “string splitter” template and I keep passing in the shorter and shorter string into the template.   When I use this mechanism, I end up with the destination XML shown at the top.

Any other way you would have done this?

Technorati Tags: BizTalk

Today I’m sitting in a “BizTalk Administration” class being taught to 20 of my co-workers by my buddy Victor.  I’ve retired from teaching internal training classes on BizTalk, so the torch has been passed and I get to sit in the back and heckle the teacher.

One thing that I finally confirmed today after having it on my “todo” list for months was the behavior of the BizTalk Admin Console when terminating messages.  What I specifically wanted to confirm was the scenario presented below.

Let’s say that I have 10 suspended messages for a particular receive port.

What happens if while I’m looking at this, another 5 suspended messages come in for this service instance?  I’ll confirm that 5 more came in via another “query” in the console.

So we know for sure that 5 more came in, but, let’s say I was still only looking at the “Suspended (resumable)” query tab.  If I choose to”terminate” the 10 suspended messages, in reality, all suspended messages that match this search criteria (now 15) get terminated.

So even though the default query result set showed 10 suspended messages, the “terminate” operation kills anything that matches this suspension criteria (15 messages).   How do we avoid this potentially sticky situation?  The best way is to append an additional criteria on your Admin Console query.  The “Suspension Time” attribute allows you to put a date + time filter on your result set.  In the screenshot below, you can see that I’ve taken the greatest timestamp in my visible result set and used that.  Even though additional failures have occurred, then don’t get absorbed by this query.

So, if you are a regular BizTalk administrator, and don’t already do this (and maybe I’m the only sap who didn’t realize this all along), make sure that your suspension queries always have a date restriction prior to terminating (unless you don’t care about messages that have arrived since the query last executed).

Technorati Tags: BizTalk

If you have any responsibility for planning or maintaining BizTalk Server databases, I highly encourage you to check out the brand new Microsoft Knowledge Base article entitled How to maintain and troubleshoot BizTalk Server databases. 

The article contains details on SQL Server instance settings, the SQL Server Agent jobs, handling deadlocks, how to delete data from BizTalk databases, and how to troubleshoot database-related issues.

Much of this data is floating around elsewhere, but it’s a useful bookmark for a consolidated view.

On a completely unrelated note, I’ve enjoyed poking around the newly opened up Stack Overflow site and learning a few new C# language things and seeing some nice tool recommendations.  I suspect that this site will be added to my daily web browsing cycle.

Technorati Tags: BizTalk