Hi, I don't understand what you propose.
Regards, Charles Moulliard Claus Ibsen wrote: > > Hi Charles > > A workaround for your current problem is to make a mistake in the > header/trailer definition and flatpack will skip it, at least in my > current unit test. > > Med venlig hilsen > > Claus Ibsen > ...................................... > Silverbullet > Skovsgårdsvænget 21 > 8362 Hørning > Tlf. +45 2962 7576 > Web: www.silverbullet.dk > > -----Original Message----- > From: cmoulliard [mailto:[EMAIL PROTECTED] > Sent: 16. juli 2008 11:11 > To: [email protected] > Subject: Re: Component to read FixedLength file ? > > > The error comes from the fact that the convertBody(Map.class) does not > take > into account that the first record and the last one are the header/trailer > of the message. > > In this case, three different map have to be created : > - Header, > - Body > - Trailer > > The property of the message and attribute of the DSL/URI must be adapted > to > use a new keywork -> flatpack:fixedHeader instead of flatpack:fixed to > inform the Body converter How to transform the > net.sf.flatpack.DefaultDataSet into their three Maps > > Regards, > > Charles > > > > cmoulliard wrote: >> >> I have a new error when I use the following routes. >> >> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10") >> >> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml"); >> >> >> >> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml") >> .convertBodyTo(Map.class) >> .to("activemq:queue:cutas.in.queue"); >> >> and xml file >> >> <?xml version='1.0'?> >> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" > >> >> <PZMAP> >> <RECORD id="header" startPosition="1" endPosition="3" indicator="HBT"> >> <COLUMN name="INDICATOR" length="3" /> >> <COLUMN name="HEADERDATA" length="26" /> >> </RECORD> >> <COLUMN name="RECORDTYPE" length="1" /> >> <COLUMN name="SYSTEMID" length="3" /> >> <COLUMN name="THIRD_PARTY_REF" length="15" /> >> <COLUMN name="CUTAS_ACCOUNT_NR" length="15" /> >> <COLUMN name="FUND_CODE" length="5" /> >> <COLUMN name="PRODUCT_CODE" length="5" /> >> <COLUMN name="PRODUCT_TYPE" length="4" /> >> <COLUMN name="CUTAS_REPLY_DATE" length="8" /> >> <COLUMN name="CUTAS_REPLY_HOUR" length="4" /> >> <COLUMN name="TRANSACTION_TYPE" length="2" /> >> <COLUMN name="FUND_TYPE" length="1" /> >> <COLUMN name="UNIT_SIGN" length="1" /> >> <COLUMN name="DATA" length="412" /> >> >> <RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT"> >> <COLUMN name="INDICATOR" length="3" /> >> <COLUMN name="TRAILERDATA" length="26" /> >> </RECORD> >> >> </PZMAP> >> >> Remark : In my case, we have a header / Trailer in the message. >> >> Error reported : >> >> >> 43org.apache.camel.RuntimeCamelException: >> java.util.NoSuchElementException: Column [recordtype] does not exist, >> check case/spelling. key:trailer >> at >> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411) >> at >> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42) >> at >> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67) >> at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59) >> at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50) >> at >> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41) >> at >> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50) >> at >> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79) >> at >> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57) >> at >> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69) >> at >> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155) >> at >> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91) >> at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) >> at org.apache.camel.processor.Pipeline.process(Pipeline.java:85) >> at >> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57) >> at >> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41) >> at >> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50) >> at >> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50) >> at >> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79) >> at >> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57) >> at >> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43) >> at >> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39) >> at >> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41) >> at >> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66) >> at >> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41) >> at >> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67) >> at >> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41) >> at >> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61) >> at >> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50) >> at >> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79) >> at >> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57) >> at >> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69) >> at >> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155) >> at >> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91) >> at >> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57) >> at >> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41) >> at >> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50) >> at >> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50) >> at >> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79) >> at >> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57) >> at >> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43) >> at >> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39) >> at >> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128) >> at >> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77) >> at >> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84) >> at >> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57) >> at >> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65) >> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) >> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source) >> at java.util.concurrent.FutureTask.runAndReset(Unknown Source) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown >> Source) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown >> Source) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown >> Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown >> Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) >> at java.lang.Thread.run(Unknown Source) >> Caused by: java.util.NoSuchElementException: Column [recordtype] does not >> exist, check case/spelling. key:trailer >> at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785) >> at >> net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184) >> at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262) >> at >> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48) >> at >> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) >> at java.lang.reflect.Method.invoke(Unknown Source) >> at >> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407) >> ... 55 more >> >> Content of the message : >> >> 4349422 [Thread: 1 [EMAIL PROTECTED] >> INFO org.apache.camel.processor.interceptor.TraceInterceptor - >> ID-dell-charles/1446-1216195452000/2-0 -> to7 >> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml] >> InOnly Properties:{CamelCauseException=null, >> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock, >> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807 >> exclusive valid], >> [EMAIL PROTECTED] >> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File >> Body:HBT00032700000327190620071714 >> C00111a30571b6b0008Z001412 FIEU BFM 200805271554 >> +0000000010000000000009076000000138137A000090760EUR >> GBPSS808800714636FIEU >> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000 >> 00001522000 000000000000000O 0000000000000000 >> >> 20070907155200 >> >> X >> C00111a30571b170007B000005 APAA 200805271554 >> +0000000001835910000003941700000039417A000332147USD >> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417 >> >> 000000000000000000000000000 00000000000 000000000000000B >> 0000000000000000 20070619101000 >> >> X >> C00111a30571a600006B000005 APAA 200805271554 >> +0000000001855550000003941700000039417A000052147GBP >> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417 >> >> 000000000000000000000000000 00000000000 000000000000000X >> 0000000000000000 20070619101000 >> >> X >> C00111a30571c2a0009B000005 APAA 200805271554 >> +0000000001855550000003941700000039417A000052147GBP >> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417 >> >> 000000000000000000000000000 00000000000 000000000000000 >> 0000000000000000 20070619101000 >> >> X >> FBT00032700000001190620071714 >> >> Info displayed on the console : >> >> 4349609 [Thread: 1 [EMAIL PROTECTED] >> INFO org.apache.camel.processor.interceptor.TraceInterceptor - >> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4 >> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[ >> null]]), sendTo(Endpoint[seda:errors]), >> RedeliveryPolicy[maximumRedeliveries=1]], >> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]), >> sendTo(Endpoint[seda:errors]), >> RedeliveryPolicy[maximumRedeliveries=1]]])] >> InOnly Properties:{} Headers:{camelFlatpackCounter=0} >> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0 >> Rows:6 >> Position:0 >> Conversion >> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble, >> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal, >> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger} >> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0 >> , Name:SYSTEMID Length:3 Start:0 End:0 >> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0 >> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0 >> , Name:FUND_CODE Length:5 Start:0 End:0 >> , Name:PRODUCT_CODE Length:5 Start:0 End:0 >> , Name:PRODUCT_TYPE Length:4 Start:0 End:0 >> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0 >> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0 >> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0 >> , Name:FUND_TYPE Length:1 Start:0 End:0 >> , Name:UNIT_SIGN Length:1 Start:0 End:0 >> , Name:DATA Length:412 Start:0 End:0 >> ] >> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10, >> systemid=1, >> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0, >> data=12, >> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8} >> XML Record Elements:[EMAIL PROTECTED], >> [EMAIL PROTECTED] >> >> Regards, >> >> Charles >> >> >> >> James.Strachan wrote: >>> >>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier >>> to use. The downside is the data format kinda has to turn the DataSet >>> into a List of Maps to make it easier to work with; but at least the >>> routes are a bit simpler. e.g. this is from the >>> DelimitedWithUnmarshalTest (the convertBodyTo is not required >>> currently) >>> >>> <camelContext xmlns="http://activemq.apache.org/camel/schema/spring";> >>> <route> >>> <from uri="file://src/test/data/delim?noop=true"/> >>> <unmarshal ref="delimitedFormat"/> >>> <splitter> >>> <simple>in.body</simple> >>> <convertBodyTo type="java.util.Map"/> >>> <to uri="mock:results"/> >>> </splitter> >>> </route> >>> </camelContext> >>> >>> <bean id="delimitedFormat" >>> class="org.apache.camel.component.flatpack.FlatpackDataFormat"> >>> <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/> >>> </bean> >>> >>> 2008/7/15 James Strachan <[EMAIL PROTECTED]>: >>>> Yeah - I did ponder if it should be a dataformat or a component; or >>>> maybe even both. The bit I wasn't sure about was flatpack really just >>>> provides an iterator over rows of the file; I wasn't sure if this >>>> properly matched the dataformat model of marshalling or unmarshalling >>>> a payload (which currently assumes that unmarshalling generates a >>>> single result). >>>> >>>> Maybe we need to enhance the dataformat concept to work with this use >>>> case too? >>>> >>>> So maybe if we want to process each row of a file we do something >>>> like... >>>> >>>> from("file://whatnot"). >>>> >>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml"). >>>> split().body(). >>>> to("whatever"); >>>> >>>> where the split().body() would do the iteration over each DataSet? >>>> >>>> >>>> 2008/7/15 Claus Ibsen <[EMAIL PROTECTED]>: >>>>> Hi >>>>> >>>>> Yes I thought also at first it was a dataformat ;) But how James >>>>> created it it works as if it is a dataformat. You just have to do it >>>>> in >>>>> to..from from..to steps. >>>>> >>>>> >>>>> from("file://c:/test").to("flatpack:foo: >>>>> locationOftheXMLConfigurationfile.pzmap.xml"); >>>>> >>>>> from("flatpack:foo").to("whatever you want"); >>>>> >>>>> As a component it gets more attention, as people can find it on the >>>>> component list. The dataformats are kinda hidden in the wiki >>>>> documentation. >>>>> Maybe they need a "dataformat" top menu as well ;) >>>>> >>>>> Med venlig hilsen >>>>> >>>>> Claus Ibsen >>>>> ...................................... >>>>> Silverbullet >>>>> Skovsgårdsvænget 21 >>>>> 8362 Hørning >>>>> Tlf. +45 2962 7576 >>>>> Web: www.silverbullet.dk >>>>> >>>>> -----Original Message----- >>>>> From: cmoulliard [mailto:[EMAIL PROTECTED] >>>>> Sent: 15. juli 2008 09:32 >>>>> To: [email protected] >>>>> Subject: Re: Component to read FixedLength file ? >>>>> >>>>> >>>>> Many thanks. >>>>> >>>>> I will test the integration of this component within Camel. >>>>> >>>>> Remark : I'm a little confused about how Flatpack has been integrated. >>>>> The >>>>> project has been integrated like a component but from my point of >>>>> view, >>>>> it >>>>> should be integrated (for consistency reason) like a Dataformat. >>>>> Because >>>>> like the other framework/technology implemented in Camel (JAXB, Artix, >>>>> XStream, ....), they transform the content from a format (XML, CSV, >>>>> ...) >>>>> into another format (Collection of Java objects) >>>>> >>>>> E.g. >>>>> >>>>> DataFormat flatpack = new >>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml") >>>>> >>>>> from("file:///c:/test") >>>>> .unmarshall(flatpack) >>>>> .to("whatever you want") >>>>> >>>>> Regards, >>>>> >>>>> Charles >>>>> >>>>> >>>>> >>>>> James.Strachan wrote: >>>>>> >>>>>> BTW trunk has an early spike of a flatpack component. Needs more >>>>>> work, >>>>>> but there's a basic fixed-width reader example there for the curious >>>>>> >>>>>> 2008/7/14 James Strachan <[EMAIL PROTECTED]>: >>>>>>> I've raised a JIRA to track this... >>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717 >>>>>>> >>>>>>> 2008/7/14 James Strachan <[EMAIL PROTECTED]>: >>>>>>>> 2008/7/14 cmoulliard <[EMAIL PROTECTED]>: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Is there a component expected to read fixedlength files in Camel >>>>>>>>> like >>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ? >>>>>>>> >>>>>>>> It should be pretty easy to write a flatpack component. >>>>>>>> >>>>>>>> there's a commercial tool called Artix Data Services which comes >>>>>>>> with >>>>>>>> a nice design UI that does this with Camel... >>>>>>>> http://activemq.apache.org/camel/artix-data-services.html >>>>>>>> >>>>>>>> but it'd be nice to have an OSS option :) >>>>>>>> >>>>>>>> -- >>>>>>>> James >>>>>>>> ------- >>>>>>>> http://macstrac.blogspot.com/ >>>>>>>> >>>>>>>> Open Source Integration >>>>>>>> http://open.iona.com >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> James >>>>>>> ------- >>>>>>> http://macstrac.blogspot.com/ >>>>>>> >>>>>>> Open Source Integration >>>>>>> http://open.iona.com >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> James >>>>>> ------- >>>>>> http://macstrac.blogspot.com/ >>>>>> >>>>>> Open Source Integration >>>>>> http://open.iona.com >>>>>> >>>>>> >>>>> >>>>> >>>>> ----- >>>>> Enterprise Architect >>>>> >>>>> Xpectis >>>>> 12, route d'Esch >>>>> L-1470 Luxembourg >>>>> >>>>> Phone +352 25 10 70 470 >>>>> Mobile +352 621 45 36 22 >>>>> >>>>> e-mail : [EMAIL PROTECTED] >>>>> web site : www.xpectis.com >>>>> -- >>>>> View this message in context: >>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html >>>>> Sent from the Camel - Users mailing list archive at Nabble.com. >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> James >>>> ------- >>>> http://macstrac.blogspot.com/ >>>> >>>> Open Source Integration >>>> http://open.iona.com >>>> >>> >>> >>> >>> -- >>> James >>> ------- >>> http://macstrac.blogspot.com/ >>> >>> Open Source Integration >>> http://open.iona.com >>> >>> >> >> > > > ----- > Enterprise Architect > > Xpectis > 12, route d'Esch > L-1470 Luxembourg > > Phone +352 25 10 70 470 > Mobile +352 621 45 36 22 > > e-mail : [EMAIL PROTECTED] > web site : www.xpectis.com > -- > View this message in context: > http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18483512.html > Sent from the Camel - Users mailing list archive at Nabble.com. > > > ----- Enterprise Architect Xpectis 12, route d'Esch L-1470 Luxembourg Phone +352 25 10 70 470 Mobile +352 621 45 36 22 e-mail : [EMAIL PROTECTED] web site : www.xpectis.com -- View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18489977.html Sent from the Camel - Users mailing list archive at Nabble.com.
