I'm looking to make some enhancements to the Flatpack component (JIRA
tickets forthcoming) and have been doing some testing/research and have
come across an odd behavior. As background, the Flatpack component will
translate a fixed width or delimited file into a common DataSet object. It
also provides converters to go from the DataSet object to either a Map or
List format. It also has a flag that allows you to split the file into one
exchange for each row in your input file. So, if you have an input file
like this:
Bob Smith
Joe Johns
John Doe
and a route like this
from("file://...").to("flatpack:fixed:config.pzmap.xml?splitRows=true");
from("flatpack:fixed:config.pzmap.xml?splitRows=true").to("mock:result");
you would get three exchanges to the mock:result endpoint. If you changed
"splitRows=true" to "splitRows=false" then you would get one exchange to
mock:result.
With the Converters you can use them like this:
from("file://...").to("flatpack:fixed:config.pzmap.xml?splitRows=true");
from("flatpack:fixed:config.pzmap.xml?splitRows=true").convertBodyTo(java.util.Map).to("mock:result");
which gives you three exchanges all as Maps.
from("file://...").to("flatpack:fixed:config.pzmap.xml?splitRows=false");
from("flatpack:fixed:config.pzmap.xml?splitRows=false").convertBodyTo(java.util.List).to("mock:result");
will give you one exchange as a List<Map>.
That all works like expected. The problem comes in when you do something
like this.
from("file://...").to("flatpack:fixed:config.pzmap.xml?splitRows=true");
from("flatpack:fixed:config.pzmap.xml?splitRows=true").convertBodyTo(java.util.List).to("mock:result");
This is asking the component to split the file into multiple exchanges and
then convert each exchange into a List<Map>, presumably each with one item
in the list. What you actually get is one exchange with a List<Map>
containing three items. This is caused by the fact that the
FlatpackProducer is iterating over the rows of the DataSet, creating an
exchange for each row but is then sending the whole dataset (preset on the
row in question) in each exchange (by Reference). When the List<Map>
converter gets it (in the same thread) it iterates through all the rows in
the DataSet and creates the list and sends it on. Then when the Producer
goes to get the next row the DataSet is now empty because the List looped
through the whole thing.
The reason I ran into this was that I wanted to make an Converter that
would translate the DataSet into XML and it would be able to work on either
the whole DataSet or just a single row. I wasn't expecting every exchange
to include the whole DataSet. There are some simple solutions to make the
XML Converter work the way I want but I thought this general behavior was
unexpected and want to get other's thoughts on if something should be done
to make it more predictable. Maybe since Flatpack doesn't natively have a
way to split a DataSet having Camel try and do it isn't the best option in
the component?
Something that makes sense to me would be that instead of having this be a
full fledged component that requires a <to:flatpack> and <from:flatpack> to
function, that it work more like a normal processor where you could do
<from:file><to:flatpack><convertBodyTo><to:splitter>.
Thoughts?
Chris
