Debatching Flat Files Into XML (With Header Intact)

Today I was approached with the problem of how to take a flat file input, debatch it, add the “header” to each individual record, and return the results as XML messages.

The first part of the equation was to successfully debatch the flat file. So, I took the header and body schemas that the developer built and added them to a new Receive pipeline’s “flat file disassembler”. On the flat file disassembler, I flipped the Preserve header property to “True” to ensure that the header would be injected into the context collection of each debatched record.

I can prove that Preserve header works by stopping my send port, and checking out the temporarily suspended messages. If I view the context collection for a given message, I’ll see the FlatFileHeaderDocument value and my header message (in XML format) embedded inside.

To transmit a flat file result (with header attached), all I have to do is create a custom Send pipeline, use the flat file assembler component, and once again choose the document and header schemas. This will cause the FlatFileHeaderDocument to be sucked out of context and converted back to delimited text, and slapped on top of each message.

That’s cool, but in our case, we wanted XML output with the header appended. I could have tried some gnarly solution involving orchestrations calling a “flat file” send pipeline (to add the header) and then an “xml” send pipeline to generate XML content, but that didn’t seem clean. Instead, I created an orchestration that first grabs the FlatFileHeaderDocument out of context and creates a valid orchestration message from it.

The XMLNORM.FlatFileHeaderDocument is the specific context value from which I can yank a string representation of the XML header message. I then loaded that into an XmlDocument variable and used that variable to instantiate an orchestration message. Next, I passed both the body AND the newly-constructed header message into a multi-part map. This map takes those two values and outputs a single document format containing both header and body values.

So now, BizTalk can take the flat file containing 20 records, debatch it, and reassemble it (header and all) into valid XML output.

Technorati Tags:

Author: Richard Seroter

Richard Seroter is Director of Developer Relations and Outbound Product Management at Google Cloud. He’s also an instructor at Pluralsight, a frequent public speaker, the author of multiple books on software design and development, and a former editor plus former 12-time Microsoft MVP for cloud. As Director of Developer Relations and Outbound Product Management, Richard leads an organization of Google Cloud developer advocates, engineers, platform builders, and outbound product managers that help customers find success in their cloud journey. Richard maintains a regularly updated blog on topics of architecture and solution design and can be found on Twitter as @rseroter.

5 thoughts

  1. Thank you very much. I just used this technique to solve a very similar situation (debatching a flat file to individual XML files with the original header inserted into each).

    I can’t believe that there still is no elegant way (even in Biztalk 2009) to do this without using an orchestration or custom coding.

    I experimented with another technique (property demotion) briefly but it didn’t work due to the fact that I have to map my document before outputing using the XML assembler, which is needed for property demotion.

    Anyway, thanks again.

  2. Hi Richard.
    I’m working with Composed Message Processor pattern and it works well, but I’m getting crazy trying to retrieve Header info. I know about “preserve header” property, but the output FlatFile has empty the header section.
    I have looked at SDK example and it has header Invoice with Default Value “XXX”.
    My goal is split one FF into 2 FF, each one w/ Header.
    Thanks for any help

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.