While trying to do what I thought would be a relatively straightforward implementation in BizTalk 2006 R2 to add a header and trailer to a flat file, I discovered that it was a bit of a nightmare with little documentation and a lot of people finding issues with no solutions.
The scenario we have is that we want to build up a flat file over time as files are collected using a sequential convoy in an orchestration (basically the aggregator pattern). When all the files have been collected (due to some condition being met) the file had to be written out with header and trailer information.
The trailer would contain aggregated information that could only be complete once all the files had been collected (total number of rows, total values etc.).
All sounded fairly simple and like a scenario that the FFAssembler pipeline component was built to handle as it had properties for Document, Header and Trailer schemas exposed.
I based the solution on the ComposedMessageProcessor in the BizTalk SDK.
I added a new schema called InvoiceTrailer.xsd and updated the pipeline configuration so it looked like the following:
I constructed test header and trailer messages and set their values through promoted properties (instead of the default xsd value in the sample) and added them to the send pipeline input message collection (Microsoft.XLANGs.Pipeline.SendPipelineInputMessages) in the same way as the document messages.
When I ran this with the test message I got the following error:
Error Description: There was a failure executing pipeline “CMP.PipelinesAndSchemas.FFSendPipeline”. Error details: “The document type “http://CMP.InvoiceHeader#InvoiceHeader” does not match any of the given schemas.”
I also tried removing the header and trailer configuration to see if that made any difference (was thinking it may look them up dynamically), but I got the same issue.
After a lot of frustration, head scratching and trial and error I came to the conclusion that BizTalk did not support what I wanted to do. I found a few blog and community posts with people trying the same thing, but no real resolution.
Then I had a kind of eureka (one last thing to try moment). I noticed on the documentation it said:
“If no schema is specified, run-time schema discovery is done.”
and I had suspected that adding the header message failed as it was only checking against the schema configured against the ‘Document schema’ property in the pipeline (although if you specify Header and Trailer schema the logical thing would have been to check these as well).
So i tried removing all configuration from the FFAssembler component as follows:
and it worked correctly, obviously resolving each of the header, document and trailer schema at runtime and building the flat file document as I wanted!
The only thing to be aware of is that the order the messages are added are the order they are written. So you need to add the header, all documents and then trailer.