Re: Welcoming Hyukjin Kwon and Sameer Agarwal as committers

Congratulations Hyukjin and Sameer ! On 7 Aug 2017 9:23 p.m., "Matei Zaharia" wrote: > Hi everyone, > > The Spark PMC recently voted to add Hyukjin Kwon and Sameer Agarwal as > committers. Join me in congratulating both of them and thanking them for > their

Re: [How-To][SQL] Create a dataframe inside the TableScan.buildScan method of a relation

On Thu, Jun 22, 2017 at 7:51 PM, OBones wrote: > Hello, > > I'm trying to extend Spark so that it can use our own binary format as a > read-only source for pipeline based computations. > I already have a java class that gives me enough elements to build a > complete StructType

Catalyst fails to eliminate common subexpression

I was examining the filters passed down to the Data Source API and noticed that a common subexpression in the SQL Select Where clause was not eliminated. Notice the "gender = M" is listed twice in the plan explain. Is this expected ? I haven't tried more complex examples with AND in the where