Category: BizTalk

I was having issues with BizTalk’s promoted/distinguished fields today, so I ran a small test, and I’m not sure what I have here.

What I want to see is when I change various values in a message (e.g. promoted property fields, distinguished fields, the message directly), when do the other values (e.g. promoted property fields, distinguished fields, the message directly) get updated? For instance, when I set a promoted value for a message, when does it get “pushed down”?

I have a very simple BizTalk project with a schema, property schema, and orchestration. In the schema I have a field called Node1 and I have both distinguished and promoted that field (into a property field named RoutingNode). In my orchestration, I have a series of activities where I manipulate the message in all sorts of sinister ways. Construct Block #1 has the following code contained (I’ve subtracted most of my trace statements):

Got that? So I print out the current values stored in the distinguished field, promoted property field and then the whole message itself. Then I change the promoted property value, and print the whole thing out again. In Construct Block #2 I have the following code (all trace statements removed since you get the point):

So in that one I changed the distinguished field value. Finally, in Construct Block #3 I changed the underlying XML document itself:

So if you’re still with me, you may be interested in the results of this little experiment. What do you expect to be printed out after each Construct block? Here’s what I got:

I don’t think I expected that. From that experiment, it looks like the XLANG message either updates itself automatically, or, when I request the values (like distinguished fields or promoted properties) it’s checking for changes. Did I do something off here, or does this seem right to you people?

Technorati Tags: BizTalk

Sigh. Ok, so BizTalk still isn’t affected by the changes to Daylight Savings this year, BUT, the Microsoft KB article has been updated to explain general quirks with daylight savings, and reveals a hotfix to be available this month.

On the general Daylight Savings Microsoft page, you’ll see a note saying, in part, that:

Although related to BizTalk Server’s handling of DST, these issues are not specific to DST 2007,

Did you know that BizTalk Server has a hidden pipeline component that can add namespaces to inbound documents?

Often, you’ll find that you’re retrieving XML data from a system where no namespace has been provided. This can cause issues for BizTalk Server given that namespace#root is the global unique identifier for messages. If you had installed the BizTalk Adapters for Enterprise Applications, you’d find a Visual Studio.NET project located at:
C:\Program Files\Microsoft BizTalk Adapters for Enterprise Applications\Pipeline Component

This is a custom pipeline component which adds namespaces to messages. For instance, let’s say I have the following XML input coming in …

<InputSchema>
<Node1>Node1_0</Node1>
<Node2>Node2_0</Node2>
</InputSchema>

I can create a receive pipeline with my new SetNSForMsg custom pipeline component.

Notice that I can type in a namespace that will be applied to the message. After deploying this pipeline and running the above message through, I get an output XML message looking like this:

<InputSchema targetNamespace=”http://Blog.BizTalk.NSPipelineTest”&gt;
<Node1>Node1_0</Node1>
<Node2>Node2_0</Node2>
</InputSchema>

I stared at that for a few moments and something didn’t look right. I added an XML Disassembler pipeline to my receive pipeline and redeployed. Now when I processed the original message, it got suspended with a notice that an unrecognized format was received. I realized that the component is setting the targetNamespace value vs. setting up the xmlns value the message needed. So, I went into the provided custom pipeline component’s Execute method and changed the line message.DocumentElement.SetAttribute(“targetNamespace“, targetNS); to message.DocumentElement.SetAttribute(“xmlns“, targetNS);. So my current receive pipeline looks like this:

My output messages now look like this:

<InputSchema xmlns=”http://Blog.BizTalk.NSPipelineTest”&gt;
<Node1>Node1_0</Node1>
<Node2>Node2_0</Node2>
</InputSchema>
Now THAT’S what I’m looking for. Just to be sure that the disassembling actually succeeded, I stopped the send port, thus suspending the outbound message. Inspecting that message shows me that the Message Type was indeed set:

Sweet. One final cool thing. Now that I have this pipeline, I can use the BizTalk Server 2006 feature to modify pipeline configuration settings for EACH receive location that uses it. You can reuse this pipeline over and over, and just modify the namespace value and document schema.

While on the topic of pipelines, don’t forget to download Tomas’ fancy new PipelineTesting library for running unit tests on your pipeline components.

Technorati Tags: BizTalk

Ok … so in checking the Microsoft Preparing for Daylight Savings Changes in 2007 page, you’ll now see an update on Feb. 2nd that states:

BizTalk Server entries removed after determination product not directly affected by DST2007 issues.

So, my previous post is moot. You’ve got plenty of things to update, but now BizTalk Server isn’t one of them.

So I see today that BizTalk Server (among many other applications) is affected by the whole screwy US Daylight Savings change for 2007. Specifically for BizTalk, you’ll find the statement “Microsoft Biztalk Server 2006: Final update will be available in March 2007 through CSS. For more information see KB article 931961 (to be published by January 26).” The KB article isn’t online yet, but any thoughts as to what is affected by the change? My money is on the “service windows” for ports. Off the top of my head, I can’t think of what else would care about the change. Other guesses before the KB article appears?

Technorati Tags: BizTalk

Via Luke, I suspect that this updated page of articles, whitepapers and webcasts on the BizTalk 2006 adapters will be invaluable for folks working in a diverse environment (hey, like me!). Note all of the Line of Business adapter walkthroughs now available. They are fairly short, but each one is to the point and gives you enough screenshots to figure things out.

Technorati Tags: BizTalk

In my last post I showed how to add conditional tracing to your BizTalk application using a flag stored in an external configuration file. While this is the easiest “lookup” solution to set up, it also has a few downsides. Namely, being forced to restart the host instance to absorb changes and the requirement to keep configuration file(s) in sync across servers.

So how about using the Business Rules Engine (BRE)? Now I’ve seen discussion saying that this isn’t a great way to go. Mainly because the BRE isn’t set up to be a metadata repository and its strength lies in calculating complex rule sets, not doing simple lookups. Also, folks may complain that they have to create an XML document or .NET class to use as an input “fact.” All valid complaints, but, I wanted to see if I could build something that was very low maintenance and still high performing. For me, the BRE provides the cache refresh capability I want, and the central management feature that’s key to maintainability.

So instead of creating some custom fact type (XML doc, class, etc), I decided to use a standard .NET type as an input. You can’t just use a “string”, as it’s viable for the rule “condition” but can’t be used for the “action.” So, I used the StringBuilder class found in the mscorlib assembly. You’ll see here, that I’ve built a rule policy with four versions. I have a versions to turn OFF tracing, turn ALL tracing on, and to turn tracing on for just pipelines or orchestrations. See that the rule “action” just adds text to the StringBuilder fact. So NO custom facts needed, just standard .NET built-in types.

I also set up a Vocabulary that allows me to use a drop-down list of possible tracing flags.

Now I can promote and demote (actually deploy and undeploy) trace flag settings without modifying code or updating files. I can be confident that the policy cache will be refreshed (60 seconds by default) without restarting any processes. Within my orchestration, now my “trace block” looks like this …

I have an orchestration variable of type StringBuilder and pass that into my rule policy. My decision condition then checks the value of “traceFlag.ToString()” which equals the value designated in the deployed rule version. If you remember from Part Iof this saga, I also created a pipeline component that used the configuration file trace flag. Now, let’s update this component to use the BRE instead. After adding a project reference to Microsoft.RulesEngine.dll, I updated my pipeline component’s “Execute” method to look like this …

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{
//create StringBuilder to pass into ruleset
StringBuilder sb = new StringBuilder();
Policy p = new Policy(“Blog.BizTalk.Tracing.Flag”);
p.Execute(sb);

if (sb.ToString() == “ALL” || sb.ToString() == “Pipeline”)
{
Stream s = inmsg.BodyPart.GetOriginalDataStream();
byte[] buffer = new byte[s.Length];
s.Read(buffer, 0, Convert.ToInt32(s.Length));
string output = System.Text.UTF8Encoding.UTF8.GetString(buffer);

System.Diagnostics.Debug.WriteLine(“Pipeline Trace: Body is ” + output);
}
p = null;
sb = null;

return inmsg;

So you can see, very little code to call the rule and check the value. Create a StringBuilder, create the Policy object, and execute the rule set. Now, from a performance standpoint, calling into the BRE should have a negligible impact. Once you call a rule policy the first time, it’s cached in memory for future requests. So, while I haven’t run the numbers on this, I can’t imagine doing such a simple rule call would have any noticeable effect on pipeline performance.

To test this, I processed my orchestration, first with the “Pipeline” trace deployed, then with the “Orchestration” trace deployed, and finally with the “None” scenario deployed. You can see the result in the log here …

So, over these two posts you’ve seen how to trace using a configuration file and the BRE. You could also potentially use the SSO API (assuming you also implemented a caching algorithm), but at this very moment, I’m a fan of using the BRE. Any other favorite methods you have?

Technorati Tags: BizTalk

I just finished up my first week in the new job, and so far, so good. One aspect of my job as Architect is to help identify reusable frameworks and encourage their usage. I spent a brief time considering “tracing” and how best to add *conditional* tracing to my BizTalk application. This is Part I of a two part post. [UPDATE: Part II]

I’d like a way to turn application tracing on and off, without requiring a code update or deployment. For this post, let’s consider the usage of a configuration file to store the trace switch. I started by modifying my btsntsvc.exe.config file. My configuration file now looks like this:

<?xml version=”1.0″ ?>
<configuration>
<runtime>
…
</runtime>
  <appSettings>
<add key=”TraceFlag” value=”Off” />

</appSettings>
<system.runtime.remoting>

…
</system.runtime.remoting>

</configuration>

Now my BizTalk components can read this value at runtime. I want the tracing to be embedded within orchestrations, but in a fairly inconspicuous way. So, I used the Group shape to hold any tracing code. My sample workflow looks like this …

If you peek inside the Group shape, you see this …

Note that a given orchestration might have a few of these trace points strategically placed at key points in the workflow. The code inside the “Lookup Flag” Expression shape is:

traceFlag = System.Configuration.ConfigurationSettings.AppSettings[“TraceFlag”];

I now have the value that the decision shape can use to determine whether to output trace data or not.

I may want to trace more than just my orchestration, so I wrote a simple pipeline component that also uses this same configuration flag. This “any” pipeline component can be used on either a Send or Receive pipeline. The pipeline component’s “Execute” method looks like this:

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{
//get trace setting from config file
string traceFlag = System.Configuration.ConfigurationSettings.AppSettings[“TraceFlag”];

if (traceFlag == “On”)
{
Stream s = inmsg.BodyPart.GetOriginalDataStream();
byte[] buffer = new byte[s.Length];
s.Read(buffer, 0, Convert.ToInt32(s.Length));
//store value of message (whatever format) in a string
string output = System.Text.UTF8Encoding.UTF8.GetString(buffer);

//output string to trace
System.Diagnostics.Debug.WriteLine(“Pipeline Trace: Body is ” + output);
}

return inmsg;
}

I then threw this into a Send pipeline that looks like this …

Now, when I run my process normally, I get the standard message that my orchestration ALWAYS writes, and if I flip the trace flag in the configuration file, I get full tracing as well …

Now, a huge caveat is that you have to restart your host instance whenever you update the configuration file in order to clear your configuration cache and force a reload of the new setting(s). This may not be ideal for your production environment, so, that’s why I’m writing a Part II to this post. [UPDATE: Part II]

Technorati Tags: BizTalk

So I spent a little time thinking about generic web service on ramps after reading Peter Kelcey’s post on the Microsoft ESB Guidance.  The shockingly observant of you may recall that I wrote a post a few months back on how to build a BizTalk web service that accepted generic input as well.

I’m just not completely sold on the “one ramp to the bus” concept yet.  Primarily because I worry about loosely typed service/object input.  It’s nice that I can use the same snippet of code to publish a message to a service regardless of the message I’m using, but, ensuring that you’re passing the *right* data is arguably more important than the convenience of only having a few web services in the infrastructure.  Now that we have such mature, powerful web service management offerings from companies like SOA Software, the physical number of services seems less of a concern.

All that said, I was having dinner with some techie buddies the other night and I raised this issue.  We agreed that being able to validate your input PRIOR to sending the message to the generic service would be a way to mitigate this concern.  So, I thought I’d try and build a simple “validation service” with BizTalk using orchestration and pipelines.

I started with a very basic couple of schemas.  What I want to do is provide a simple service that take in any XML input, and returns back any validation errors.

Next comes the pipeline itself.  In my Disassemble stage, I’m using the XML Disassembler with each available schema defined in the Document Schemas property.  I set Validate Document to false here, and then used an XML Validator pipeline component later on to do the actual schema validation.

To start with, this is all I need to test.  I wanted to test 4 different scenarios:

• Invalid data types (e.g. pass in string when int is expected)
• Missing nodes
• Adding additional records even though maxOccurs equals 1
• Passing in a message where no corresponding schema exists

I think that covers the basics of validation for now.  So I built a pure messaging solution (FILE receive location, with FILE send port) and applied my custom receive pipeline to the inbound receive location.  As expected, the result of each of the above scenarios was a suspended message.  So my pipeline works.  Now, I wanted to call this pipeline from an orchestration so that I would have a synchronous service that gracefully captured exceptions.

So, in my orchestration I referenced Microsoft.XLANGS.Pipelines.dll and Microsoft.BizTalk.Pipeline.dll.  The process receives a document in (of type XmlDocument), and then within an atomic transaction (contained within a larger long-running transaction that catches exceptions), I called into my pipeline component.  This is new functionality in BizTalk Server 2006.  The code in my Expression Shape is:

RcvPipeOutMsgs = Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(Microsoft.Demo.Blog.ValidationSvc.Rcv_ValidateOnRampMsg), OnRampInput);

The RcvPipeOutMsgs is a variable of type Microsoft.XLANGS.Pipeline.ReceivePipelineOutputMessages, you see that I use the typeof my pipeline component, and the OnRampInput is the inbound message of type XmlDocument.  The whole orchestration looks like this:

You can see that I catch any exceptions, set status variables, and then construct a “response” message to the service caller.  I then walked through the Web Services Publishing Wizard (after changing my operation name from Operation_1 to PostDocToValidate) and built a web service that takes in any XML, and returns a common “response” message.

I then built a simple WinForm to call the service, passing in a variety of inputs.  If I pass in a valid message, I get a “success” message.  If I pass in a message with invalid content (scenarios 1-3 above), I get an error:

Looks good.  HOWEVER, if I pass in a garbage message (that is, no schema exists), I get this:

I’m calling the exact same pipeline that I did in my “pure messaging” solution which DID raise an error for garbage messages, but when calling this pipeline from an orchestration, the pipeline isn’t raising an exception.  I have tried about 37 different combinations of pipeline components, property settings, message types (tried switching from XML input to string) and I can’t see to get an error to occur.

So, there you have a 3/4 validation service.  However, without properly handling bad messages, it’s not entirely useful.  Thoughts?  I really wanted to use the standard BizTalk components to validate (e.g. out of the box pipeline components), instead of using any sort of cool code-based solution.  Anything I might be missing here?