A project team recently asked me if they could use the BizTalk BRE to validate incoming data. I asked what sort of validation we were talking about, and it came down to four areas:
- Setting default values when the incoming fields were empty
- Doing look-ups based on existing data and populating related fields
- Doing concatenation of fields
- Catching any business validation errors and recording them
Before answering “of course it can”, I figured I’d quickly roll up my sleeves and prove that the BRE can do those things fairly easily.
The Setup
First, I had to get my environment set up. This started with a BizTalk XSD schema to represent the data coming into the BRE. In this scenario, the content is actually metadata about a physical file that arrives into the company.
Notice that I have a “validation errors” record at the bottom with a repeating “error” element.
Next, I have a really simple database table that can be used for lookups. Based on providing a “study” and “product” value (which comes in via the inbound XML), I want to be able to figure out which “investigator” to add to the message.
Next I need a .NET class to perform a few functions that the BRE doesn’t provide for me out of the box. First, I need to be able to concatenate two strings. Yes, it’s ridiculous that I have to write an operation to do this vs. the BRE having something built in. Deal with it.
public class RuleFunctions
{
public string ConcatValues(string input1, string input2)
{
return input1 + input2;
}
}
Next, I need a function that can add nodes to my XML message when validation errors are discovered. After adding references to the Microsoft.RulesEngine.dll and System.Xml, I wrote the following quick-and-dirty code.
public void AddValidationError(TypedXmlDocument activeDoc, string err)
{
XmlDocument doc = activeDoc.Document.OwnerDocument;
XmlNode errorRoot = doc.SelectSingleNode("//ValidationErrors");
XmlElement newError = doc.CreateElement("Error");
newError.InnerText = err;
errorRoot.AppendChild(newError);
}
I accept a “TypedXmlDocument”, yank out the underlying XmlDocument, grab a pointer to the “ValidationErrors” record, and append a new node containing whatever error message was defined within the BRE.
After GAC-ing that helper DLL, I’m ready to roll. Before building my rules in the BRE, I wanted to establish a friendly vocabulary for my XML nodes, database columns and helper functions. So, I defined a new vocabulary and created definitions for all the “get”, “set” and “function” operations that my rules required.
Setting Default Values
The first requirement was to be able to set default values on nodes. This one’s pretty easy. I just check for either an empty string or null value, and set the default value.
Doing Lookups
The next requirement was to take values from the inbound message, perform a lookup and fill in an empty node from the message. In the rule below, I say that if the value in the XML message equals a value from the database, and another value also equals a value from the database, then set the XML node (Investigator Name) equal to another column in the database.
What this generates behind the scenes when you execute the rule is this:
So the proper T-SQL is built and executed to grab my lookup value.
Performing Concatenation
There’s no out-of-box string concatenation function in the BRE, so here I use the custom class I built earlier. I want this rule to fire at all times, so my condition is set to something that will always be true (unless math itself starts failing). I take a static file path location and then append the file name of the document coming into my company and turn that whole thing into an absolute path to the file.
Catching and Recording Business Exceptions
The final requirement was to detect business validation errors and record them. So, we can use my custom built function from earlier to create rules that look for business errors and add nodes to the message itself which outline the error details.
Remember that my custom function takes in a “TypedXmlDocument” type, so I pass the document itself into the function by dragging the root node of my schema from the XML Schemas Fact Explorer view into the operation parameter.
The Result
So what happens when I pass in a document that flexes all these conditions? I start off with an instance document that only has a few values.
So I should see some default values get set, the “investigator” node set based on database lookups, the “file path” node be populated with the concatenated value, and I should see some errors at the end because nodes like “document sub type” and “site” are empty.
To test this rule set, I’m using the built in rule tester, so I have to pass in a valid XML instance, a database connection, and a fact creator that provides the BRE with an instance of my custom .NET class.
When the rule policy is finished executing, my input XML is changed to this:
You can see that all the rules we built earlier got applied successfully. So yes, you can use the BRE to do some pretty easy-to-maintain business validation. This may not be a fit if you’re dealing with a nightly batch of 25,000 records, but if you are doing a messaging based solution, the BRE can do some nice work for you.
Richard Seroter's Architecture Musings 
Sure would be nice if there were a Microsoft Rules Server 2012 with a full rules management suite, an Oslo repository, a VS or Quadrant dev environment, and maybe an Office family / SharePoint app that domain experts could make heads or tails of. And especially now that ILog is off the table I’m constantly amazed that Rules simply don’t get more respect or traction at Microsoft. Hopefully CEP will start to change that a bit, but I am not particularly optimistic that will be the case.
Nice article,
Could you elaborate a bit more on how you set up the DB lookup ?
Hey Patrick. So to get the database lookup working, I first created a new vocabulary item, choosing “Database Table or Column”, then browsed to the database, table and column that the vocab item should point to. I kept the “binding type” drop down list set to “Data Connection” vs. “Data Table/Row” which would require me to get the data table myself and pass it into the rule. Instead, all I need is the connection string. I then built the rule as shown in the post, had the T-SQL automatically generated during execution. To test, I had to create a valid connection to pass in, and during actual execution, I’d have to pass in a real SqlConnection.
Does that help?
nice one ….
for these kinds of “defaulting”, “lookups”, “concat” etc. could a map on the inbound port not be a better solution?
i agree a map isn’t the best place for very complicated rules processing, but simple stuff like this works well in a map … and it has a Concat functoid out the box 😀
Ryan, fair point. You could definitely allow a map to do some of these. I guess one criteria that would steer you one way or another is the general volatility of the logic, and who is expected to maintain it.
Good article – great to see the BRE being positioned as a mainstream tool.
I did something similar a while back, and was equally annoyed, as it seemed as if I’d have to write a helper class to perform simple functions. After some digging, I found an alternative.
You can directly reference the “mscorlib” under .NET Classes. This is the Common Object Runtime Library, and as it contains the System namespace, you have access to very useful objects like Convert, Math and String.
For string concatenation, you’d expand the String object, and drag the “Concat(String str0, String str1)” method across to your Conditions or Actions pane in the same way that you’d use the methods from a custom helper class.
But wait! There’s more!
Because “Concat(String str0, String str1)” is a static method, you don’t need to provide a fact creator to the BRE. In effect, you now have access to a host of utility functions out of the box for free (provided you’re calling static methods).
Hope that you guys find this useful and that it saves you time.
Thanks It helped me a lot:)
The SWIFT Accelerator makes extensive use of the Business Rules Engine for SWIFT Network Rules validation. It works pretty well to fill in the huge hole left by standard XSD validation. It’s worth checking out if you want to see a mature (long lived) use of BRE.
How to make this work when you’re validating multiple records in the xml?
something like:
condition: //Employees/Employee/testColum == 1
action: add error node to //Employees/Employee/ErrorNodes/ – the error node should be added to the current Employee node.
Using the example above, the error node is always added to the first Employee record in the Employees xml.
Hey Calvin,
Might have to use an external class to help out.
Wondering if you can help? I am new to Biztalk and I am needing to validate multiple fact values. My example:
The rule is for revenue code data from a hospital bill. This is how it is on the XML:
“/Claim/Charges/Charge/RevenueCode”
On these bills there are lots of different revenue codes. I want the rule to fire when there is more than one 045x revenue code on the hospital bill. Specifically more than one revenue code that is from 0450 to 0459. Thanks.
Hi Shirley. it’s been a bit since I’ve played with the Rules Engine, but I think that if you send the RevenueCode node in as a set of facts, then all the nodes get assessed. However, in your rule condition, you can check for codes in that range, and if the fact matches, only then will the rule’s Action occur.
Appreciate it.
I am trying a sample to get multiple errors. I have created all the schemas as described. But Error messages are not showing up. What may be the reason for this?
Is there any way to debug the class created with typedXMLDocument. I am getting object reference error in AddValidationErrors method
AddValidationErrors method in the created class is not working when I use the rule in Orchestration. But it works when I use it in TestPolicy with FactCreator. Is there anything else that i should do?
Hmm. Unfortunately I don’t have any tips here for you, but I suggest you add your question to the BizTalk Forums to see if someone has some insight into troubleshooting this!
I am creating one DB vocabulary but getting below error :
“Invalid Object Name testdata”
testdata is my table name and it’s schema is mdm.
So is there any limitation for BizTalk that it can only access the tables having dbo schema?