Spark 3.0 and ORC 1.6

2020-01-28 Thread David Christle
Hi all, I am a heavy user of Spark at LinkedIn, and am excited about the ZStandard compression option recently incorporated into ORC 1.6. I would love to explore using it for storing/querying of large (>10 TB) tables for my own disk I/O intensive workloads, and other users & companies may be

Re: `Target Version` management on correctness/data-loss Issues

2020-01-28 Thread Dongjoon Hyun
Thanks, Tom. I agree that emails are good for urgent announcement and reaching fast agreement. Also, more visible in a short time period. However, some correctness issues are long-standing and sometime they changes their faces with different JIRA IDs. We can see the relationship easily in the

Re: `Target Version` management on correctness/data-loss Issues

2020-01-28 Thread Tom Graves
I was just thinking an info email  (perhaps tagged with correctness/dataloss) to dev rather than an official vote, that way its more visible and if anyone sees it and disagrees with the targeting it can be discussed on that thread.   It might also just bring more visibility to those important