Re: DataSourceV2Reader Q

2019-05-21 Thread Andrew Melo
Hi Ryan, On Tue, May 21, 2019 at 2:48 PM Ryan Blue wrote: > > Are you sure that your schema conversion is correct? If you're running with a > recent Spark version, then that line is probably `name.hashCode()`. That file > was last updated 6 months ago so I think it is likely that `name` is the

Re: DataSourceV2Reader Q

2019-05-21 Thread Ryan Blue
Are you sure that your schema conversion is correct? If you're running with a recent Spark version, then that line is probably `name.hashCode()`. That file was last updated 6 months ago so I think it is likely that `name` is the null in your version. On Tue, May 21, 2019 at 11:39 AM Andrew Melo

DataSourceV2Reader Q

2019-05-21 Thread Andrew Melo
Hello, I'm developing a DataSourceV2 reader for the ROOT (https://root.cern/) file format to replace a previous DSV1 source that was in use before. I have a bare skeleton of the reader, which can properly load the files and pass their schema into Spark 2.4.3, but any operation on the resulting