Re: Change field separator in Metron to make it Hive and ORC friendly

2018-08-14 Thread Ali Nazemian
Hi Simon, We have temporarily decided to just change it with "_" for HDFS to avoid all the headaches of the bugs and issues that can be raised by using unsupported separators for ORC/Hive and Spark. However, I am not quite confident with "_" as an option for the community as it becomes similar to

Re: Change field separator in Metron to make it Hive and ORC friendly

2018-08-14 Thread deepak kumar
I agree Ali. May be it can be configuration parameter. On Tue, Aug 14, 2018 at 3:24 PM Ali Nazemian wrote: > Hi Simon, > > We have temporarily decided to just change it with "_" for HDFS to avoid > all the headaches of the bugs and issues that can be raised by using > unsupported separators for

Re: Change field separator in Metron to make it Hive and ORC friendly

2018-08-14 Thread Simon Elliston Ball
The challenge with making it configurable is that every query, every profile, every analytic, template, pre-installed dashboard and use case built by any third party who wanted to extend metron would have to honour the configuration and paramaterize every query they run. My worry is that that wo