Here are my notes from this week’s sync.
*Attendees*:
Ryan Blue
John Zhuge
Dale Richardson
Gabor Somogyi
Matt Cheah
Yifei Huang
Xin Ren
Jose Torres
Gengliang Wang
Kevin Yu
*Topics*:
- Metadata columns or function push-down for Kafka v2 source
- Open PRs
- REPLACE TABLE
I think that with a few reviews, we could have a minimally working DSv2 SQL
API ready in a couple weeks. We have PRs ready for the last of the major
items, like ALTER TABLE, INSERT INTO, REPLACE TABLE, and a v2 catalog
interface for the session catalog. Mainly, we're just waiting for final
reviews
That's a good question. Although we had penciled in 'middle of the
year' I don't think we're in sight of a QA phase just yet, as I
believe some key items are still in progress. I'm thinking of the Hive
update, and DS v2 work (?).
I'm also curious to hear what broad TODOs people see for 3.0? we
Hey Friends,
Is there a timeline for spark 3.0 in terms of the first RC and final release?
Cheers Andrew
The PR of bumping Jackson to 2.9.6 gives some examples of the behavioral
changes that Sean is referring to:
https://github.com/apache/spark/pull/21596
Cheers,
Fokko Driesprong
Op vr 28 jun. 2019 om 14:13 schreef Sean Owen :
> https://github.com/apache/spark/blob/branch-2.4/pom.xml#L161
>
https://github.com/apache/spark/blob/branch-2.4/pom.xml#L161
Correct, because it would introduce behavior changes.
On Fri, Jun 28, 2019 at 3:54 AM Pavithra R wrote:
> In spark master branch, the version of Jackson jars have been upgraded to
> 2.9.9
>
>
>
Be interesting to see how well a Pi4 works; with only 4GB of RAM you
wouldn't compile with it, but you could try installing the spark jar bundle
and then run against some NFS mounted disks:
https://www.raspberrypi.org/magpi/raspberry-pi-4-specs-benchmarks/ ;
unlikely to be fast, but it'd be an
In spark master branch, the version of Jackson jars have been upgraded to 2.9.9
https://github.com/apache/spark/commit/bd8732300385ad99d2cec3a4af49953d8925eaf6
[SPARK-27757][CORE] Bump Jackson to 2.9.9 -
This has been done to address CVE-2019-12086.
Could you confirm why Jackson jars are not
