Re: csv reader performance with multiline option

Hi yes, multiline would only use one thread in that case. The csv parser used by spark is uniVocity Le 18 août 2018 à 18:07, Nirav Patel écrivait : > does enabling 'multiLine' option impact performance? how? would it run read > entire file with just one thread? > > Thanks > > > > What's

csv reader performance with multiline option

does enabling 'multiLine' option impact performance? how? would it run read entire file with just one thread? Thanks --        

Re: Use Spark extension points to implement row-level security

Thanks, this looks promising. I am trying to do it without a dependency on Hive and was hoping that the extension hooks could be used to add a filter transformation to the logical plan. I've seen some other email saying that in the optimisation hook the logical is expected to stay the same (