Hi , Camel is using java.util.Scanner for splitting the input stream by using token "\n". So it makes sense that it is slower then using the BufferedReader to read the file.
You can read the file yourself by implementing a customer splitter just like the ZipFile does[1] [1]https://issues.apache.org/jira/browse/CAMEL-6139 -- Willem Jiang Red Hat, Inc. FuseSource is now part of Red Hat Web: http://www.fusesource.com | http://www.redhat.com Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) (English) http://jnn.iteye.com (http://jnn.javaeye.com/) (Chinese) Twitter: willemjiang Weibo: 姜宁willem On Thursday, July 18, 2013 at 10:28 AM, Vic wrote: > I am processing big file - line by line with camel. The average amount of > processed messages per second is 30k. When I do the same in java using > BufferedReader - the average amount of processed messages per second is > 500k. I am processing the same file. This is significant performance lost. > Am I doing something wrong in Camel? > > camel route : > > from("file:C:/Test?fileName=test_file.txt&noop=true") > .split().tokenize("\n").streaming() > .to("log:INFO?groupSize=10000"); > > java code: > > FileReader fr = new FileReader("C:/Test/test_file.txt"); > BufferedReader br = new BufferedReader(fr); > > long count = 0; > long start = System.currentTimeMillis(); > while(br.readLine() != null) > { > count++; > if(count % 10000 == 0) > { > long now = System.currentTimeMillis(); > long msgPerSecond = 1000*count/(now-start); > System.out.println(msgPerSecond); > } > } > > br.close(); > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Message-Processing-Performance-while-splitting-tp5735824.html > Sent from the Camel - Users mailing list archive at Nabble.com > (http://Nabble.com).
