Hi Brad! To read and split a file you need a combination of streaming unmarshalling and splitter components. Is tmxPaymentTechIn something you wrote?
You can try to read the file by camel as xml, splitting groups by xpath and unmarshall just the group from xml to a bean model... Unfortunately I do not use BeanIO so I do not know how the unmarshalled instance model looks like. Is there a List<Group> involved? If yes, it's to late for splitting the unmarshalled beans anyway. You can write your own processor which makes use of template-producer to create individual messages per group while you read the file on your own. Because you need additional header fields set I would try to write a camel processor to create the messages. Jens Von meinem iPhone gesendet > Am 05.04.2016 um 00:54 schrieb Brad Johnson <brad.john...@mediadriver.com>: > > I have a beanio mapping that's rather standard and I read it in but it > appears that it reads the entire file into memory instead of streaming it. > Is there a way to change that behavior so that I can take items as they are > read and put them on a queue? > > <from uri="file:/dir/inbound" /> > <unmarshal ref="tmxPaymentTechIn" /> > <log message="Process and convert records." /> > <to uri="seda:processRecords" /> > > The file is an ole timey batch record with headers and mutli-line, fixed > width fields per record. What I'd like to see is each element header, > transaction, etc. read from the file and then put on the queue. What I get > is an array of the entire file which I can, of course, split but that will > get memory intensive. > > BeanIO works very well for this as there are variable row records (Address > for example) and it snaps that stuff up well. But I really want records or > groups to come through individually. > > > <stream name="fileIn" format="fixedlength"> > > > <record name="header" order="1" minOccurs="1" maxOccurs="1" > class="foo.Header"> > <field name="identifier" rid="true" literal="P" > position="0" length="1" /> > <field name="pid" position="4" length="6" /> > <field name="pidPassword" position="11" length="8" /> > <field name="sid" position="24" length="6" /> > <field name="sidPassword" position="31" length="8" /> > <field name="directive" position="40" length="6" /> > </record> > > <group name="transaction" order="2" minOccurs="1" > maxOccurs="unbounded" class="foo.Transaction"> > > <record name="summary" minOccurs="0" maxOccurs="1" > class="foo.Summary"> > <field name="identifier" rid="true" literal="S" > position="0" length="1" /> > > <field name="reportGroup" position="1" length="10" /> > <field name="orderId" position="11" length="8" /> > <field name="batchId" position="19" length="14" /> > <field name="actionCode" position="33" length="2" /> > > <segment name="group" class="foo.Group"> > <field name="my" position="35" length="2" /> > <field name="group" position="37" length="19" /> > <field name="info" position="56" length="4" /> > </segment> > <field name="source" position="60" length="12" /> > <field name="integerOrderSource" position="78" > length="1" /> > <field name="activityBranch" position="112" length="4" > /> > </record> > > <record name="company" class="foo.Company" minOccurs="0" > maxOccurs="1"> > <field name="companyName" rid="true" literal="AB" > position="0" length="2" /> > <field name="companyName" position="2" length="29" /> > </record> > <record name="addressList" collection="list" > class="foo.Addressline" minOccurs="0" maxOccurs="unbounded"> > <field name="identifier" rid="true" literal="A" > position="0" length="1" /> > <field name="address" position="2" length="29" /> > </record> > </group> > > <record name="BRec" order="4" class="foo.Rec" minOccurs="0" > maxOccurs="1"> > <field name="identifier" rid="true" literal="B" > position="0" length="1" /> > <field name="record" position="1" length="unbounded" /> > </record> > <record name="TRec" order="5" class="foo.Rec" minOccurs="0" > maxOccurs="1"> > <field name="identifier" rid="true" literal="T" > position="0" length="1" /> > <field name="record" position="1" length="unbounded" /> > </record> > <record name="footer" order="6" class="foo.Footer" > minOccurs="0" maxOccurs="1"> > <field name="identifier" rid="true" literal="PID" > position="0" length="3" /> > > </record> > > </stream>
