Re: [DISCUSS] Spark Columnar Processing

2019-04-13 Thread Bobby Evans
I'll link the two. On Thu, Apr 11, 2019 at 12:34 PM Reynold Xin wrote: > I just realized we had an earlier SPIP on a similar topic: > https://issues.apache.org/jira/browse/SPARK-24579 > > Perhaps we should tie the two together. IIUC, you'd want to expose the > existing ColumnBatch API, but also

Re: Dataset schema incompatibility bug when reading column partitioned data

2019-04-13 Thread Felix Cheung
I kinda agree it is confusing when a parameter is not used... From: Ryan Blue Sent: Thursday, April 11, 2019 11:07:25 AM To: Bruce Robbins Cc: Dávid Szakállas; Spark Dev List Subject: Re: Dataset schema incompatibility bug when reading column partitioned data

ApacheCon NA 2019 Call For Proposal and help promoting Spark project

2019-04-13 Thread Felix Cheung
Hi Spark community! As you know ApacheCon NA 2019 is coming this Sept and it’s CFP is now open! This is an important milestone as we celebrate 20 years of ASF. We have tracks like Big Data and Machine Learning among many others. Please submit your talks/thoughts/challenges/learnings here: