Re: [VOTE] SPIP: Spark API for Table Metadata

[Wenchen Fan]

+1, thanks for making it clear that this SPIP focuses on
high-level direction!

On Sat, Mar 2, 2019 at 9:35 AM Reynold Xin  wrote:

> Thanks Ryan. +1.
>
>
>
>
> On Fri, Mar 01, 2019 at 5:33 PM, Ryan Blue  wrote:
>
>> Actually, I went ahead and removed the confusing section. There is no
>> public API in the doc now, so that it is clear that it isn't a relevant
>> part of this vote.
>>
>> On Fri, Mar 1, 2019 at 4:58 PM Ryan Blue  wrote:
>>
>> I moved the public API to the "Implementation Sketch" section. That API
>> is not an important part of this, as that section notes.
>>
>> I completely agree that SPIPs should be high-level and that the
>> specifics, like method names, are not hard requirements. The proposal was
>> more of a sketch, but I was asked by Xiao in the DSv2 sync to make sure the
>> list of methods was complete. I think as long as we have agreement that the
>> intent is not to make the exact names binding, we should be okay.
>>
>> I can remove the user-facing API sketch, but I'd prefer to leave it in
>> the sketch section so we have it documented somewhere.
>>
>> On Fri, Mar 1, 2019 at 4:51 PM Reynold Xin  wrote:
>>
>> Ryan - can you take the public user facing API part out of that SPIP?
>>
>> In general it'd be better to have the SPIPs be higher level, and put the
>> detailed APIs in a separate doc. Alternatively, put them in the SPIP but
>> explicitly vote on the high level stuff and not the detailed APIs.
>>
>> I don't want to get to a situation in which two months later the
>> identical APIs were committed with the justification that they were voted
>> on a while ago. In this case, it's even more serious because while I think
>> we all have consensus on the higher level internal API, not much discussion
>> has happened with the user-facing API and we should just leave that out
>> explicitly.
>>
>>
>>
>>
>>
>>
>> On Fri, Mar 01, 2019 at 1:00 PM, Anthony Young-Garner <
>> anthony.young-gar...@cloudera.com.invalid> wrote:
>>
>> +1 (non-binding)
>>
>> On Thu, Feb 28, 2019 at 5:54 PM John Zhuge  wrote:
>>
>> +1 (non-binding)
>>
>> On Thu, Feb 28, 2019 at 9:11 AM Matt Cheah  wrote:
>>
>> +1 (non-binding)
>>
>>
>>
>> *From: *Jamison Bennett 
>> *Date: *Thursday, February 28, 2019 at 8:28 AM
>> *To: *Ryan Blue , Spark Dev List > >
>> *Subject: *Re: [VOTE] SPIP: Spark API for Table Metadata
>>
>>
>>
>> +1 (non-binding)
>>
>>
>> *Jamison Bennett*
>>
>> Cloudera Software Engineer
>>
>> jamison.benn...@cloudera.com
>>
>> 515 Congress Ave, Suite 1212   |   Austin, TX   |   78701
>>
>>
>>
>>
>>
>> On Thu, Feb 28, 2019 at 10:20 AM Ryan Blue 
>> wrote:
>>
>> +1 (non-binding)
>>
>>
>>
>> On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer <
>> russell.spit...@gmail.com> wrote:
>>
>> +1 (non-binding)
>>
>> On Wed, Feb 27, 2019, 6:28 PM Ryan Blue 
>> wrote:
>>
>> Hi everyone,
>>
>>
>>
>> In the last DSv2 sync, the consensus was that the table metadata SPIP was
>> ready to bring up for a vote. Now that the multi-catalog identifier SPIP
>> vote has passed, I'd like to start one for the table metadata API,
>> TableCatalog.
>>
>>
>>
>> The proposal is for adding a TableCatalog interface that will be used by
>> v2 plans. That interface has methods to load, create, drop, alter, refresh,
>> rename, and check existence for tables. It also specifies the set of
>> metadata used to configure tables: schema, partitioning, and key-value
>> properties. For more information, please read the SPIP proposal doc
>> [docs.google.com]
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1zLFiA1VuaWeVxeTDXNg8bL6GP3BVoOZBkewFtEnjEoo_edit-23heading-3Dh.m45webtwxf2d&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=hzwIMNQ9E99EMYGuqHI0kXhVbvX3nU3OSDadUnJxjAs&m=JmgvL6ffL9tyoLWWZtWujDe9FNiSguMApA53YK9NTP8&s=eSx5nMZvdB5hS9VepuvvFZFXjTCrdde-AdzkHC5jRYk&e=>
>> .
>>
>>
>>
>> Please vote in the next 3 days.
>>
>>
>>
>> [ ] +1: Accept the proposal as an official SPIP
>>
>> [ ] +0
>>
>> [ ] -1: I don't think this is a good idea because ...
>>
>>
>>
>>
>>
>> Thanks!
>>
>>
>>
>> --
>>
>> Ryan Blue
>>
>> Software Engineer
>>
>> Netflix
>>
>>
>>
>>
>> --
>>
>> Ryan Blue
>>
>> Software Engineer
>>
>> Netflix
>>
>>
>>
>> --
>> John Zhuge
>>
>>
>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Netflix
>>
>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Netflix
>>
>
>

Re: [VOTE] SPIP: Spark API for Table Metadata

[Reynold Xin]

Thanks Ryan. +1.

On Fri, Mar 01, 2019 at 5:33 PM, Ryan Blue < rb...@netflix.com > wrote:

> 
> Actually, I went ahead and removed the confusing section. There is no
> public API in the doc now, so that it is clear that it isn't a relevant
> part of this vote.
> 
> On Fri, Mar 1, 2019 at 4:58 PM Ryan Blue < rblue@ netflix. com (
> rb...@netflix.com ) > wrote:
> 
> 
>> I moved the public API to the "Implementation Sketch" section. That API is
>> not an important part of this, as that section notes.
>> 
>> 
>> I completely agree that SPIPs should be high-level and that the specifics,
>> like method names, are not hard requirements. The proposal was more of a
>> sketch, but I was asked by Xiao in the DSv2 sync to make sure the list of
>> methods was complete. I think as long as we have agreement that the intent
>> is not to make the exact names binding, we should be okay.
>> 
>> 
>> I can remove the user-facing API sketch, but I'd prefer to leave it in the
>> sketch section so we have it documented somewhere.
>> 
>> On Fri, Mar 1, 2019 at 4:51 PM Reynold Xin < rxin@ databricks. com (
>> r...@databricks.com ) > wrote:
>> 
>> 
>>> Ryan - can you take the public user facing API part out of that SPIP?
>>> 
>>> 
>>> 
>>> In general it'd be better to have the SPIPs be higher level, and put the
>>> detailed APIs in a separate doc. Alternatively, put them in the SPIP but
>>> explicitly vote on the high level stuff and not the detailed APIs. 
>>> 
>>> 
>>> 
>>> I don't want to get to a situation in which two months later the identical
>>> APIs were committed with the justification that they were voted on a while
>>> ago. In this case, it's even more serious because while I think we all
>>> have consensus on the higher level internal API, not much discussion has
>>> happened with the user-facing API and we should just leave that out
>>> explicitly.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On Fri, Mar 01, 2019 at 1:00 PM, Anthony Young-Garner < anthony. 
>>> young-garner@
>>> cloudera. com. invalid ( anthony.young-gar...@cloudera.com.invalid ) > 
>>> wrote:
>>> 
>>> 
>>>> +1 (non-binding)
>>>> 
>>>> 
>>>> On Thu, Feb 28, 2019 at 5:54 PM John Zhuge < jzhuge@ apache. org (
>>>> jzh...@apache.org ) > wrote:
>>>> 
>>>> 
>>>>> +1 (non-binding)
>>>>> 
>>>>> 
>>>>> On Thu, Feb 28, 2019 at 9:11 AM Matt Cheah < mcheah@ palantir. com (
>>>>> mch...@palantir.com ) > wrote:
>>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> +1 (non-binding)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> *From:* Jamison Bennett < jamison. bennett@ cloudera. com. INVALID (
>>>>>> jamison.benn...@cloudera.com.INVALID ) >
>>>>>> *Date:* Thursday, February 28, 2019 at 8:28 AM
>>>>>> *To:* Ryan Blue < rblue@ netflix. com ( rb...@netflix.com ) >, Spark Dev
>>>>>> List < dev@ spark. apache. org ( dev@spark.apache.org ) >
>>>>>> *Subject:* Re: [VOTE] SPIP: Spark API for Table Metadata
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> +1 (non-binding)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> *Jamison Bennett*
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Cloudera Software Engineer
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> jamison. bennett@ cloudera. com ( jamison.benn...@cloudera.com )
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 515 Congress Ave, Suite 1212  |   Austin, TX  |   78701
>>>>>> 
>>>>>> 
>>>>>> 
>>>

Re: [VOTE] SPIP: Spark API for Table Metadata

[Ryan Blue]

Actually, I went ahead and removed the confusing section. There is no
public API in the doc now, so that it is clear that it isn't a relevant
part of this vote.

On Fri, Mar 1, 2019 at 4:58 PM Ryan Blue  wrote:

> I moved the public API to the "Implementation Sketch" section. That API is
> not an important part of this, as that section notes.
>
> I completely agree that SPIPs should be high-level and that the specifics,
> like method names, are not hard requirements. The proposal was more of a
> sketch, but I was asked by Xiao in the DSv2 sync to make sure the list of
> methods was complete. I think as long as we have agreement that the intent
> is not to make the exact names binding, we should be okay.
>
> I can remove the user-facing API sketch, but I'd prefer to leave it in the
> sketch section so we have it documented somewhere.
>
> On Fri, Mar 1, 2019 at 4:51 PM Reynold Xin  wrote:
>
>> Ryan - can you take the public user facing API part out of that SPIP?
>>
>> In general it'd be better to have the SPIPs be higher level, and put the
>> detailed APIs in a separate doc. Alternatively, put them in the SPIP but
>> explicitly vote on the high level stuff and not the detailed APIs.
>>
>> I don't want to get to a situation in which two months later the
>> identical APIs were committed with the justification that they were voted
>> on a while ago. In this case, it's even more serious because while I think
>> we all have consensus on the higher level internal API, not much discussion
>> has happened with the user-facing API and we should just leave that out
>> explicitly.
>>
>>
>>
>>
>>
>>
>> On Fri, Mar 01, 2019 at 1:00 PM, Anthony Young-Garner <
>> anthony.young-gar...@cloudera.com.invalid> wrote:
>>
>>> +1 (non-binding)
>>>
>>> On Thu, Feb 28, 2019 at 5:54 PM John Zhuge  wrote:
>>>
>>> +1 (non-binding)
>>>
>>> On Thu, Feb 28, 2019 at 9:11 AM Matt Cheah  wrote:
>>>
>>> +1 (non-binding)
>>>
>>>
>>>
>>> *From: *Jamison Bennett 
>>> *Date: *Thursday, February 28, 2019 at 8:28 AM
>>> *To: *Ryan Blue , Spark Dev List <
>>> dev@spark.apache.org>
>>> *Subject: *Re: [VOTE] SPIP: Spark API for Table Metadata
>>>
>>>
>>>
>>> +1 (non-binding)
>>>
>>>
>>> *Jamison Bennett*
>>>
>>> Cloudera Software Engineer
>>>
>>> jamison.benn...@cloudera.com
>>>
>>> 515 Congress Ave, Suite 1212   |   Austin, TX   |   78701
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Feb 28, 2019 at 10:20 AM Ryan Blue 
>>> wrote:
>>>
>>> +1 (non-binding)
>>>
>>>
>>>
>>> On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer <
>>> russell.spit...@gmail.com> wrote:
>>>
>>> +1 (non-binding)
>>>
>>> On Wed, Feb 27, 2019, 6:28 PM Ryan Blue 
>>> wrote:
>>>
>>> Hi everyone,
>>>
>>>
>>>
>>> In the last DSv2 sync, the consensus was that the table metadata SPIP
>>> was ready to bring up for a vote. Now that the multi-catalog identifier
>>> SPIP vote has passed, I'd like to start one for the table metadata API,
>>> TableCatalog.
>>>
>>>
>>>
>>> The proposal is for adding a TableCatalog interface that will be used by
>>> v2 plans. That interface has methods to load, create, drop, alter, refresh,
>>> rename, and check existence for tables. It also specifies the set of
>>> metadata used to configure tables: schema, partitioning, and key-value
>>> properties. For more information, please read the SPIP proposal doc
>>> [docs.google.com]
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1zLFiA1VuaWeVxeTDXNg8bL6GP3BVoOZBkewFtEnjEoo_edit-23heading-3Dh.m45webtwxf2d&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=hzwIMNQ9E99EMYGuqHI0kXhVbvX3nU3OSDadUnJxjAs&m=JmgvL6ffL9tyoLWWZtWujDe9FNiSguMApA53YK9NTP8&s=eSx5nMZvdB5hS9VepuvvFZFXjTCrdde-AdzkHC5jRYk&e=>
>>> .
>>>
>>>
>>>
>>> Please vote in the next 3 days.
>>>
>>>
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>>
>>> [ ] +0
>>>
>>> [ ] -1: I don't think this is a good idea because ...
>>>
>>>
>>>
>>>
>>>
>>> Thanks!
>>>
>>>
>>>
>>> --
>>>
>>> Ryan Blue
>>>
>>> Software Engineer
>>>
>>> Netflix
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Ryan Blue
>>>
>>> Software Engineer
>>>
>>> Netflix
>>>
>>>
>>>
>>> --
>>> John Zhuge
>>>
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>


-- 
Ryan Blue
Software Engineer
Netflix

Re: [VOTE] SPIP: Spark API for Table Metadata

[Ryan Blue]

I moved the public API to the "Implementation Sketch" section. That API is
not an important part of this, as that section notes.

I completely agree that SPIPs should be high-level and that the specifics,
like method names, are not hard requirements. The proposal was more of a
sketch, but I was asked by Xiao in the DSv2 sync to make sure the list of
methods was complete. I think as long as we have agreement that the intent
is not to make the exact names binding, we should be okay.

I can remove the user-facing API sketch, but I'd prefer to leave it in the
sketch section so we have it documented somewhere.

On Fri, Mar 1, 2019 at 4:51 PM Reynold Xin  wrote:

> Ryan - can you take the public user facing API part out of that SPIP?
>
> In general it'd be better to have the SPIPs be higher level, and put the
> detailed APIs in a separate doc. Alternatively, put them in the SPIP but
> explicitly vote on the high level stuff and not the detailed APIs.
>
> I don't want to get to a situation in which two months later the identical
> APIs were committed with the justification that they were voted on a while
> ago. In this case, it's even more serious because while I think we all have
> consensus on the higher level internal API, not much discussion has
> happened with the user-facing API and we should just leave that out
> explicitly.
>
>
>
>
>
>
> On Fri, Mar 01, 2019 at 1:00 PM, Anthony Young-Garner <
> anthony.young-gar...@cloudera.com.invalid> wrote:
>
>> +1 (non-binding)
>>
>> On Thu, Feb 28, 2019 at 5:54 PM John Zhuge  wrote:
>>
>> +1 (non-binding)
>>
>> On Thu, Feb 28, 2019 at 9:11 AM Matt Cheah  wrote:
>>
>> +1 (non-binding)
>>
>>
>>
>> *From: *Jamison Bennett 
>> *Date: *Thursday, February 28, 2019 at 8:28 AM
>> *To: *Ryan Blue , Spark Dev List > >
>> *Subject: *Re: [VOTE] SPIP: Spark API for Table Metadata
>>
>>
>>
>> +1 (non-binding)
>>
>>
>> *Jamison Bennett*
>>
>> Cloudera Software Engineer
>>
>> jamison.benn...@cloudera.com
>>
>> 515 Congress Ave, Suite 1212   |   Austin, TX   |   78701
>>
>>
>>
>>
>>
>> On Thu, Feb 28, 2019 at 10:20 AM Ryan Blue 
>> wrote:
>>
>> +1 (non-binding)
>>
>>
>>
>> On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer <
>> russell.spit...@gmail.com> wrote:
>>
>> +1 (non-binding)
>>
>> On Wed, Feb 27, 2019, 6:28 PM Ryan Blue 
>> wrote:
>>
>> Hi everyone,
>>
>>
>>
>> In the last DSv2 sync, the consensus was that the table metadata SPIP was
>> ready to bring up for a vote. Now that the multi-catalog identifier SPIP
>> vote has passed, I'd like to start one for the table metadata API,
>> TableCatalog.
>>
>>
>>
>> The proposal is for adding a TableCatalog interface that will be used by
>> v2 plans. That interface has methods to load, create, drop, alter, refresh,
>> rename, and check existence for tables. It also specifies the set of
>> metadata used to configure tables: schema, partitioning, and key-value
>> properties. For more information, please read the SPIP proposal doc
>> [docs.google.com]
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1zLFiA1VuaWeVxeTDXNg8bL6GP3BVoOZBkewFtEnjEoo_edit-23heading-3Dh.m45webtwxf2d&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=hzwIMNQ9E99EMYGuqHI0kXhVbvX3nU3OSDadUnJxjAs&m=JmgvL6ffL9tyoLWWZtWujDe9FNiSguMApA53YK9NTP8&s=eSx5nMZvdB5hS9VepuvvFZFXjTCrdde-AdzkHC5jRYk&e=>
>> .
>>
>>
>>
>> Please vote in the next 3 days.
>>
>>
>>
>> [ ] +1: Accept the proposal as an official SPIP
>>
>> [ ] +0
>>
>> [ ] -1: I don't think this is a good idea because ...
>>
>>
>>
>>
>>
>> Thanks!
>>
>>
>>
>> --
>>
>> Ryan Blue
>>
>> Software Engineer
>>
>> Netflix
>>
>>
>>
>>
>> --
>>
>> Ryan Blue
>>
>> Software Engineer
>>
>> Netflix
>>
>>
>>
>> --
>> John Zhuge
>>
>>
>

-- 
Ryan Blue
Software Engineer
Netflix

Re: [VOTE] SPIP: Spark API for Table Metadata

[Reynold Xin]

Ryan - can you take the public user facing API part out of that SPIP?

In general it'd be better to have the SPIPs be higher level, and put the 
detailed APIs in a separate doc. Alternatively, put them in the SPIP but 
explicitly vote on the high level stuff and not the detailed APIs. 

I don't want to get to a situation in which two months later the identical APIs 
were committed with the justification that they were voted on a while ago. In 
this case, it's even more serious because while I think we all have consensus 
on the higher level internal API, not much discussion has happened with the 
user-facing API and we should just leave that out explicitly.

On Fri, Mar 01, 2019 at 1:00 PM, Anthony Young-Garner < 
anthony.young-gar...@cloudera.com.invalid > wrote:

> 
> +1 (non-binding)
> 
> 
> On Thu, Feb 28, 2019 at 5:54 PM John Zhuge < jzhuge@ apache. org (
> jzh...@apache.org ) > wrote:
> 
> 
>> +1 (non-binding)
>> 
>> 
>> On Thu, Feb 28, 2019 at 9:11 AM Matt Cheah < mcheah@ palantir. com (
>> mch...@palantir.com ) > wrote:
>> 
>> 
>>> 
>>> 
>>> +1 (non-binding)
>>> 
>>> 
>>> 
>>>  
>>> 
>>> 
>>> 
>>> *From:* Jamison Bennett < jamison. bennett@ cloudera. com. INVALID (
>>> jamison.benn...@cloudera.com.INVALID ) >
>>> *Date:* Thursday, February 28, 2019 at 8:28 AM
>>> *To:* Ryan Blue < rblue@ netflix. com ( rb...@netflix.com ) >, Spark Dev
>>> List < dev@ spark. apache. org ( dev@spark.apache.org ) >
>>> *Subject:* Re: [VOTE] SPIP: Spark API for Table Metadata
>>> 
>>> 
>>> 
>>> 
>>>  
>>> 
>>> 
>>> 
>>> 
>>> +1 (non-binding)
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> *Jamison Bennett*
>>> 
>>> 
>>> 
>>> Cloudera Software Engineer
>>> 
>>> 
>>> 
>>> jamison. bennett@ cloudera. com ( jamison.benn...@cloudera.com )
>>> 
>>> 
>>> 
>>> 515 Congress Ave, Suite 1212  |   Austin, TX  |   78701
>>> 
>>> 
>>> 
>>> 
>>>  
>>> 
>>> 
>>> 
>>> 
>>>  
>>> 
>>> 
>>> 
>>> On Thu, Feb 28, 2019 at 10:20 AM Ryan Blue < rblue@ netflix. com. invalid (
>>> rb...@netflix.com.invalid ) > wrote:
>>> 
>>> 
>>> 
>>>> 
>>>> 
>>>> +1 (non-binding)
>>>> 
>>>> 
>>>> 
>>>> 
>>>>  
>>>> 
>>>> 
>>>> 
>>>> On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer < russell. spitzer@ gmail.
>>>> com ( russell.spit...@gmail.com ) > wrote:
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> 
>>>>> +1 (non-binding) 
>>>>> 
>>>>> 
>>>>> 
>>>>> On Wed, Feb 27, 2019, 6:28 PM Ryan Blue < rblue@ netflix. com. invalid (
>>>>> rb...@netflix.com.invalid ) > wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Hi everyone,
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> In the last DSv2 sync, the consensus was that the table metadata SPIP was
>>>>>> ready to bring up for a vote. Now that the multi-catalog identifier SPIP
>>>>>> vote has passed, I'd like to start one for the table metadata API,
>>>>>> TableCatalog.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> The proposal is for adding a TableCatalog interface that will be used by
>>>>>> v2 plans. That interface has methods to load, create, drop, alter,
>>>>>> refresh, rename, and check existence for tables. It also specifies the 
>>>>>> set
>>>>>> of metadata used to configure tables: schema, partitioning, and key-value
>>>>>> properties. For more information, please read the SPIP proposal doc 
>>>>>> [docs.
>>>>>> google. com] (
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1zLFiA1VuaWeVxeTDXNg8bL6GP3BVoOZBkewFtEnjEoo_edit-23heading-3Dh.m45webtwxf2d&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=hzwIMNQ9E99EMYGuqHI0kXhVbvX3nU3OSDadUnJxjAs&m=JmgvL6ffL9tyoLWWZtWujDe9FNiSguMApA53YK9NTP8&s=eSx5nMZvdB5hS9VepuvvFZFXjTCrdde-AdzkHC5jRYk&e=
>>>>>> ).
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Please vote in the next 3 days.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> [ ] +1: Accept the proposal as an official SPIP
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> [ ] +0
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> [ ] -1: I don't think this is a good idea because ...
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Thanks!
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>  
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Ryan Blue
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Software Engineer
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Netflix
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>>  
>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> 
>>>> 
>>>> 
>>>> Ryan Blue
>>>> 
>>>> 
>>>> 
>>>> Software Engineer
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Netflix
>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> 
>> --
>> John Zhuge
>> 
> 
>

Re: [VOTE] SPIP: Spark API for Table Metadata

[Anthony Young-Garner]

+1 (non-binding)

On Thu, Feb 28, 2019 at 5:54 PM John Zhuge  wrote:

> +1 (non-binding)
>
> On Thu, Feb 28, 2019 at 9:11 AM Matt Cheah  wrote:
>
>> +1 (non-binding)
>>
>>
>>
>> *From: *Jamison Bennett 
>> *Date: *Thursday, February 28, 2019 at 8:28 AM
>> *To: *Ryan Blue , Spark Dev List > >
>> *Subject: *Re: [VOTE] SPIP: Spark API for Table Metadata
>>
>>
>>
>> +1 (non-binding)
>>
>>
>> *Jamison Bennett*
>>
>> Cloudera Software Engineer
>>
>> jamison.benn...@cloudera.com
>>
>> 515 Congress Ave, Suite 1212   |   Austin, TX   |   78701
>>
>>
>>
>>
>>
>> On Thu, Feb 28, 2019 at 10:20 AM Ryan Blue 
>> wrote:
>>
>> +1 (non-binding)
>>
>>
>>
>> On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer <
>> russell.spit...@gmail.com> wrote:
>>
>> +1 (non-binding)
>>
>> On Wed, Feb 27, 2019, 6:28 PM Ryan Blue 
>> wrote:
>>
>> Hi everyone,
>>
>>
>>
>> In the last DSv2 sync, the consensus was that the table metadata SPIP was
>> ready to bring up for a vote. Now that the multi-catalog identifier SPIP
>> vote has passed, I'd like to start one for the table metadata API,
>> TableCatalog.
>>
>>
>>
>> The proposal is for adding a TableCatalog interface that will be used by
>> v2 plans. That interface has methods to load, create, drop, alter, refresh,
>> rename, and check existence for tables. It also specifies the set of
>> metadata used to configure tables: schema, partitioning, and key-value
>> properties. For more information, please read the SPIP proposal doc
>> [docs.google.com]
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1zLFiA1VuaWeVxeTDXNg8bL6GP3BVoOZBkewFtEnjEoo_edit-23heading-3Dh.m45webtwxf2d&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=hzwIMNQ9E99EMYGuqHI0kXhVbvX3nU3OSDadUnJxjAs&m=JmgvL6ffL9tyoLWWZtWujDe9FNiSguMApA53YK9NTP8&s=eSx5nMZvdB5hS9VepuvvFZFXjTCrdde-AdzkHC5jRYk&e=>
>> .
>>
>>
>>
>> Please vote in the next 3 days.
>>
>>
>>
>> [ ] +1: Accept the proposal as an official SPIP
>>
>> [ ] +0
>>
>> [ ] -1: I don't think this is a good idea because ...
>>
>>
>>
>>
>>
>> Thanks!
>>
>>
>>
>> --
>>
>> Ryan Blue
>>
>> Software Engineer
>>
>> Netflix
>>
>>
>>
>>
>> --
>>
>> Ryan Blue
>>
>> Software Engineer
>>
>> Netflix
>>
>>
>
> --
> John Zhuge
>

Re: [VOTE] SPIP: Spark API for Table Metadata

[John Zhuge]

+1 (non-binding)

On Thu, Feb 28, 2019 at 9:11 AM Matt Cheah  wrote:

> +1 (non-binding)
>
>
>
> *From: *Jamison Bennett 
> *Date: *Thursday, February 28, 2019 at 8:28 AM
> *To: *Ryan Blue , Spark Dev List 
> *Subject: *Re: [VOTE] SPIP: Spark API for Table Metadata
>
>
>
> +1 (non-binding)
>
>
> *Jamison Bennett*
>
> Cloudera Software Engineer
>
> jamison.benn...@cloudera.com
>
> 515 Congress Ave, Suite 1212   |   Austin, TX   |   78701
>
>
>
>
>
> On Thu, Feb 28, 2019 at 10:20 AM Ryan Blue 
> wrote:
>
> +1 (non-binding)
>
>
>
> On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer 
> wrote:
>
> +1 (non-binding)
>
> On Wed, Feb 27, 2019, 6:28 PM Ryan Blue  wrote:
>
> Hi everyone,
>
>
>
> In the last DSv2 sync, the consensus was that the table metadata SPIP was
> ready to bring up for a vote. Now that the multi-catalog identifier SPIP
> vote has passed, I'd like to start one for the table metadata API,
> TableCatalog.
>
>
>
> The proposal is for adding a TableCatalog interface that will be used by
> v2 plans. That interface has methods to load, create, drop, alter, refresh,
> rename, and check existence for tables. It also specifies the set of
> metadata used to configure tables: schema, partitioning, and key-value
> properties. For more information, please read the SPIP proposal doc
> [docs.google.com]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1zLFiA1VuaWeVxeTDXNg8bL6GP3BVoOZBkewFtEnjEoo_edit-23heading-3Dh.m45webtwxf2d&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=hzwIMNQ9E99EMYGuqHI0kXhVbvX3nU3OSDadUnJxjAs&m=JmgvL6ffL9tyoLWWZtWujDe9FNiSguMApA53YK9NTP8&s=eSx5nMZvdB5hS9VepuvvFZFXjTCrdde-AdzkHC5jRYk&e=>
> .
>
>
>
> Please vote in the next 3 days.
>
>
>
> [ ] +1: Accept the proposal as an official SPIP
>
> [ ] +0
>
> [ ] -1: I don't think this is a good idea because ...
>
>
>
>
>
> Thanks!
>
>
>
> --
>
> Ryan Blue
>
> Software Engineer
>
> Netflix
>
>
>
>
> --
>
> Ryan Blue
>
> Software Engineer
>
> Netflix
>
>

-- 
John Zhuge

Re: [VOTE] SPIP: Spark API for Table Metadata

[Matt Cheah]

+1 (non-binding)

 

From: Jamison Bennett 
Date: Thursday, February 28, 2019 at 8:28 AM
To: Ryan Blue , Spark Dev List 
Subject: Re: [VOTE] SPIP: Spark API for Table Metadata

 

+1 (non-binding) 


Jamison Bennett

Cloudera Software Engineer

jamison.benn...@cloudera.com

515 Congress Ave, Suite 1212   |   Austin, TX   |   78701

 

 

On Thu, Feb 28, 2019 at 10:20 AM Ryan Blue  wrote:

+1 (non-binding)

 

On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer  
wrote:

+1 (non-binding) 

On Wed, Feb 27, 2019, 6:28 PM Ryan Blue  wrote:

Hi everyone, 

 

In the last DSv2 sync, the consensus was that the table metadata SPIP was ready 
to bring up for a vote. Now that the multi-catalog identifier SPIP vote has 
passed, I'd like to start one for the table metadata API, TableCatalog.

 

The proposal is for adding a TableCatalog interface that will be used by v2 
plans. That interface has methods to load, create, drop, alter, refresh, 
rename, and check existence for tables. It also specifies the set of metadata 
used to configure tables: schema, partitioning, and key-value properties. For 
more information, please read the SPIP proposal doc [docs.google.com].

 

Please vote in the next 3 days.

 

[ ] +1: Accept the proposal as an official SPIP

[ ] +0

[ ] -1: I don't think this is a good idea because ...

 

 

Thanks!

 

-- 

Ryan Blue 

Software Engineer

Netflix


 

-- 

Ryan Blue 

Software Engineer

Netflix



smime.p7s
Description: S/MIME cryptographic signature

Re: [VOTE] SPIP: Spark API for Table Metadata

[Jamison Bennett]

+1 (non-binding)

Jamison Bennett

Cloudera Software Engineer

jamison.benn...@cloudera.com

515 Congress Ave, Suite 1212   |   Austin, TX   |   78701


On Thu, Feb 28, 2019 at 10:20 AM Ryan Blue 
wrote:

> +1 (non-binding)
>
> On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer 
> wrote:
>
>> +1 (non-binding)
>>
>> On Wed, Feb 27, 2019, 6:28 PM Ryan Blue 
>> wrote:
>>
>>> Hi everyone,
>>>
>>> In the last DSv2 sync, the consensus was that the table metadata SPIP
>>> was ready to bring up for a vote. Now that the multi-catalog identifier
>>> SPIP vote has passed, I'd like to start one for the table metadata API,
>>> TableCatalog.
>>>
>>> The proposal is for adding a TableCatalog interface that will be used by
>>> v2 plans. That interface has methods to load, create, drop, alter, refresh,
>>> rename, and check existence for tables. It also specifies the set of
>>> metadata used to configure tables: schema, partitioning, and key-value
>>> properties. For more information, please read the SPIP proposal doc
>>> 
>>> .
>>>
>>> Please vote in the next 3 days.
>>>
>>> [ ] +1: Accept the proposal as an official SPIP
>>> [ ] +0
>>> [ ] -1: I don't think this is a good idea because ...
>>>
>>>
>>> Thanks!
>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Netflix
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>

Re: [VOTE] SPIP: Spark API for Table Metadata

[Ryan Blue]

+1 (non-binding)

On Wed, Feb 27, 2019 at 8:34 PM Russell Spitzer 
wrote:

> +1 (non-binding)
>
> On Wed, Feb 27, 2019, 6:28 PM Ryan Blue  wrote:
>
>> Hi everyone,
>>
>> In the last DSv2 sync, the consensus was that the table metadata SPIP was
>> ready to bring up for a vote. Now that the multi-catalog identifier SPIP
>> vote has passed, I'd like to start one for the table metadata API,
>> TableCatalog.
>>
>> The proposal is for adding a TableCatalog interface that will be used by
>> v2 plans. That interface has methods to load, create, drop, alter, refresh,
>> rename, and check existence for tables. It also specifies the set of
>> metadata used to configure tables: schema, partitioning, and key-value
>> properties. For more information, please read the SPIP proposal doc
>> 
>> .
>>
>> Please vote in the next 3 days.
>>
>> [ ] +1: Accept the proposal as an official SPIP
>> [ ] +0
>> [ ] -1: I don't think this is a good idea because ...
>>
>>
>> Thanks!
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Netflix
>>
>

-- 
Ryan Blue
Software Engineer
Netflix

Re: [VOTE] SPIP: Spark API for Table Metadata

[Russell Spitzer]

+1 (non-binding)

On Wed, Feb 27, 2019, 6:28 PM Ryan Blue  wrote:

> Hi everyone,
>
> In the last DSv2 sync, the consensus was that the table metadata SPIP was
> ready to bring up for a vote. Now that the multi-catalog identifier SPIP
> vote has passed, I'd like to start one for the table metadata API,
> TableCatalog.
>
> The proposal is for adding a TableCatalog interface that will be used by
> v2 plans. That interface has methods to load, create, drop, alter, refresh,
> rename, and check existence for tables. It also specifies the set of
> metadata used to configure tables: schema, partitioning, and key-value
> properties. For more information, please read the SPIP proposal doc
> 
> .
>
> Please vote in the next 3 days.
>
> [ ] +1: Accept the proposal as an official SPIP
> [ ] +0
> [ ] -1: I don't think this is a good idea because ...
>
>
> Thanks!
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>

[VOTE] SPIP: Spark API for Table Metadata

[Ryan Blue]

Hi everyone,

In the last DSv2 sync, the consensus was that the table metadata SPIP was
ready to bring up for a vote. Now that the multi-catalog identifier SPIP
vote has passed, I'd like to start one for the table metadata API,
TableCatalog.

The proposal is for adding a TableCatalog interface that will be used by v2
plans. That interface has methods to load, create, drop, alter, refresh,
rename, and check existence for tables. It also specifies the set of
metadata used to configure tables: schema, partitioning, and key-value
properties. For more information, please read the SPIP proposal doc

.

Please vote in the next 3 days.

[ ] +1: Accept the proposal as an official SPIP
[ ] +0
[ ] -1: I don't think this is a good idea because ...


Thanks!

-- 
Ryan Blue
Software Engineer
Netflix