Re: supported file formats today ?

Thanks for the response Devendra. Regards, Raja. From: Devendra Tagare mailto:devend...@datatorrent.com>> Reply-To: "users@apex.apache.org" mailto:users@apex.apache.org>> Date: Tuesday, August 2, 2016 at 6:31 PM To: "users@apex.apache.org

Re: supported file formats today ?

Hi Raja, We have a proposal for Parquet writer backed by a WAL in Malhar.I believe Avro write support is also planned. As of now there is nothing out of the box to support ORC/RC support. You can propose it to the community and someone would likely take it up or you can pick it up and contribute

Re: Information Needed

The problem is likely in the parser. Please check it's documentation to see if parser can be reused to parse tuples. If not and you need to construct a new parser for each tuple, read configuration file into a byte array and construct new input stream using the byte array. Vlad On 8/2/16 14:0

RE: Information Needed

Hi, Below is the method which gets called inside the emit() method. I am using the input stream to parse each line. The highlighted inputStream I would want to create only Once. But the stream is getting closed if I create the inputStream inside setup() or beginWindow() method. private KeyValu

RE: Information Needed

Hi, inputConfStream is used to parse the input line from the feed. This is used for all the lines from the feed. Not sure why the stream is getting closed? Regards, Surya Vamshi From: Vlad Rozov [mailto:v.ro...@datatorrent.com] Sent: 2016, August, 02 4:26 PM To: users@apex.apache.org Subject: R

Re: Information Needed

Both setup() and beginWindow() should work. It will be more correct to open the configuration stream and parse the configuration file in setup() as you tried in the initial implementation as long as configuration path does not depend on window Id. Where the inputConfStream is used? Most likely

Information Needed

Hi Team, When I am trying to read input line from feed, to parse the line I am reading another configuration file from HDFS. To avoid reading the configuration file for every line I would like to read it in the beginWindow() method. But the Input stream is getting closed and operator is not hol

Re: supported file formats today ?

Hi Can someone help me with my question in below email … ? I want to write data as and when arrives into Hadoop HDFS but want to write into different hadoop supported file formats like RC Files/ ORC Files ? Do we some operator in Apex in this regard ? Thanks a lot in advance. Regards, Raj