Hi Ryan,
Thanks for summarizing and sending out the notes! I've created the JIRA
ticket to add v2 statements for all the commands that need to resolve a
table: https://issues.apache.org/jira/browse/SPARK-29481
Contributions to it are appreciated!
Thanks,
Wenchen
On Fri, Oct 11, 2019 at 7:05 AM
Here are my notes from last week's DSv2 sync.
*Attendees*:
Ryan Blue
Terry Kim
Wenchen Fan
*Topics*:
- SchemaPruning only supports Parquet and ORC?
- Out of order optimizer rules
- 3.0 work
- Rename session catalog to spark_catalog
- Finish TableProvider update to avoid
Here are my notes from the last DSv2 sync. Sorry it's a bit late!
*Attendees*:
Ryan Blue
John Zhuge
Raynmond McCollum
Terry Kim
Gengliang Wang
Jose Torres
Wenchen Fan
Priyanka Gomatam
Matt Cheah
Russel Spitzer
Burak Yavuz
*Topics*:
- Check in on blockers
- Remove SaveMode
-
I agree that the long-term solution is much farther away, but I'm not sure
it is a good idea to do this in the optimizer. Maybe we could find a good
way to do it, but the initial complication required before we moved to
push-down to the conversion to physical plan was really bad. Plus, this has
Hi Ryan,
Thanks for summarizing and sending out the meeting notes! Unfortunately, I
missed the last sync, but the topics are really interesting, especially the
stats integration.
The ideal solution I can think of is to refactor the optimizer/planner and
move all the stats-based optimization to
Here are my notes from the last sync. If you’d like to be added to the
invite or have topics, please let me know.
*Attendees*:
Ryan Blue
Matt Cheah
Yifei Huang
Jose Torres
Burak Yavuz
Gengliang Wang
Michael Artz
Russel Spitzer
*Topics*:
- Existing PRs
- V2 session catalog:
Here are the latest DSv2 sync notes. Please reply with updates or
corrections.
*Attendees*:
Ryan Blue
Michael Armbrust
Gengliang Wang
Matt Cheah
John Zhuge
*Topics*:
Wenchen’s reorganization proposal
Problems with TableProvider - property map isn’t sufficient
New PRs:
- ReplaceTable:
Here are my notes from last night’s sync. I had to leave early, so there
may be more discussion. Others can fill in the details for those topics.
*Attendees*:
John Zhuge
Ryan Blue
Yifei Huang
Matt Cheah
Yuanjian Li
Russell Spitzer
Kevin Yu
*Topics*:
- Atomic extensions for the TableCatalog
Sorry these notes are so late, I didn’t get to the write up until now. As
usual, if anyone has corrections or comments, please reply.
*Attendees*:
John Zhuge
Ryan Blue
Andrew Long
Wenchen Fan
Gengliang Wang
Russell Spitzer
Yuanjian Li
Yifei Huang
Matt Cheah
Amardeep Singh Dhilon
Zhilmil Dhion
Thanks Ryan!
On Tue, Mar 5, 2019 at 7:19 PM Ryan Blue wrote:
> Everyone is welcome to join this discussion. Just send me an e-mail to get
> added to the invite.
>
> Stavros, I'll add you.
>
> rb
>
> On Tue, Mar 5, 2019 at 5:43 AM Stavros Kontopoulos <
> stavros.kontopou...@lightbend.com> wrote:
Everyone is welcome to join this discussion. Just send me an e-mail to get
added to the invite.
Stavros, I'll add you.
rb
On Tue, Mar 5, 2019 at 5:43 AM Stavros Kontopoulos <
stavros.kontopou...@lightbend.com> wrote:
> Thanks for the update, is this meeting open for other people to join?
>
>
Thanks for the update, is this meeting open for other people to join?
Stavros
On Thu, Feb 21, 2019 at 10:56 PM Ryan Blue
wrote:
> Here are my notes from the DSv2 sync last night. As always, if you have
> corrections, please reply with them. And if you’d like to be included on
> the invite to
Here are my notes from the DSv2 sync last night. As always, if you have
corrections, please reply with them. And if you’d like to be included on
the invite to participate in the next sync (6 March), send me an email.
Here’s a quick summary of the topics where we had consensus last night:
-
Hi everyone, here are notes from the last DSv2 sync on 23 January 2019.
Here are the highlights:
- Agreed that using v2 should not change behavior for file sources.
(Cannot make this guarantee for all v1 sources)
- Consensus for the approach proposed on the dev list for identifying
Here are my notes from the DSv2 sync last night.
*As usual, I didn’t take great notes because I was participating in the
discussion. Feel free to send corrections or clarification.*
*Attendees*:
Ryan Blue
John Zhuge
Xiao Li
Reynold Xin
Felix Cheung
Anton Okolnychyi
Bruce Robbins
Dale Richardson
Thanks for sending out the meeting notes from last week's discussion Ryan!
For technical unknown reasons, I could not unmute myself and be heard when I
was trying to pitch in during one of the topic discussions regarding default
value handling for traditional databases. Had posted response in
Hi everyone, sorry these notes are late. I didn’t have the time to write
this up last week.
For anyone interested in the next sync, we decided to skip next week and
resume in early January. I’ve already sent the invite. As usual, if you
have topics you’d like to discuss or would like to be added
17 matches
Mail list logo
