Spark-avro 4.0.0 is released

[Gengliang Wang]

The 4.0.0 release adds support for Spark 2.2. The published artifact is
compatible with both Spark 2.1 and 2.2.

New Features:

   - Support for Spark 2.2 (#242
   ): resolve
   compatibility issue with datasource write API changes
   

   .

Bug fixes:

   - Fix name conflict in nested records (#249
   )


Release history:

   - https://github.com/databricks/spark-avro/releases


Thanks for the contributions from Imran Rashid, Gerard Solà and Jacky Shen!

-- 
Wang Gengliang
Software Engineer
Databricks Inc.

Re: Timeline for Spark 2.3

[Sameer Agarwal]

Sounds good to me too. In addition to what has already been pointed out
about the Spark History Server and the Kubernetes support, this would also
give us enough time to further polish the new data source v2 API and the
vectorized UDF API to iron out any kinks.

I'd like to volunteer to serve as the release manager for Spark 2.3. In
terms of bandwidth, I will be available during this holiday season as I
have no vacation planed during the Dec-Jan timeframe. And I'm fairly
familiar with most of the major efforts targeted for the 2.3 release.

Thanks,
Sameer

On Fri, Nov 10, 2017 at 2:07 AM, Sean Owen  wrote:

> The original timeline was just +6 months from last planned release, so
> there was nothing too magic about it. That was pushed from +4 . The only
> risk here is that an extra month becomes 2, 3, and so users aren't getting
> the other 1000 fixes. But no particular problem with moving it back.
>
> On Thu, Nov 9, 2017, 5:54 PM Michael Armbrust 
> wrote:
>
>> According to the timeline posted on the website, we are nearing branch
>> cut for Spark 2.3.  I'd like to propose pushing this out towards mid to
>> late December for a couple of reasons and would like to hear what people
>> think.
>>
>> 1. I've done release management during the Thanksgiving / Christmas time
>> before and in my experience, we don't actually get a lot of testing during
>> this time due to vacations and other commitments. I think beginning the RC
>> process in early January would give us the best coverage in the shortest
>> amount of time.
>> 2. There are several large initiatives in progress that given a little
>> more time would leave us with a much more exciting 2.3 release.
>> Specifically, the work on the history server, Kubernetes and continuous
>> processing
>> 3. Given the actual release date of Spark 2.2, I think we'll still get
>> Spark 2.3 out roughly 6 months after.
>>
>> Thoughts?
>>
>> Michael
>>
>


-- 
Sameer Agarwal
Software Engineer | Databricks Inc.
http://cs.berkeley.edu/~sameerag

unsubscribe

[marco rocchi]

unsubscribe

unsubscribe

[Steve Johnston]

unsubscribe

[Al Pivonka]

unsubscribe

-- 
Those who say it can't be done, are usually interrupted by those doing it.

Re: Timeline for Spark 2.3

[Marco Gaido]

I would love too to have SPARK-18016. I think it would help a lot of users.

2017-11-10 5:58 GMT+01:00 Nick Pentreath :

> +1 I think that’s practical
>
> On Fri, 10 Nov 2017 at 03:13, Erik Erlandson  wrote:
>
>> +1 on extending the deadline. It will significantly improve the logistics
>> for upstreaming the Kubernetes back-end.  Also agreed, on the general
>> realities of reduced bandwidth over the Nov-Dec holiday season.
>> Erik
>>
>> On Thu, Nov 9, 2017 at 6:03 PM, Matei Zaharia 
>> wrote:
>>
>>> I’m also +1 on extending this to get Kubernetes and other features in.
>>>
>>> Matei
>>>
>>> > On Nov 9, 2017, at 4:04 PM, Anirudh Ramanathan
>>>  wrote:
>>> >
>>> > This would help the community on the Kubernetes effort quite a bit -
>>> giving us additional time for reviews and testing for the 2.3 release.
>>> >
>>> > On Thu, Nov 9, 2017 at 3:56 PM, Justin Miller <
>>> justin.mil...@protectwise.com> wrote:
>>> > That sounds fine to me. I’m hoping that this ticket can make it into
>>> Spark 2.3: https://issues.apache.org/jira/browse/SPARK-18016
>>> >
>>> > It’s causing some pretty considerable problems when we alter the
>>> columns to be nullable, but we are OK for now without that.
>>> >
>>> > Best,
>>> > Justin
>>> >
>>> >> On Nov 9, 2017, at 4:54 PM, Michael Armbrust 
>>> wrote:
>>> >>
>>> >> According to the timeline posted on the website, we are nearing
>>> branch cut for Spark 2.3.  I'd like to propose pushing this out towards mid
>>> to late December for a couple of reasons and would like to hear what people
>>> think.
>>> >>
>>> >> 1. I've done release management during the Thanksgiving / Christmas
>>> time before and in my experience, we don't actually get a lot of testing
>>> during this time due to vacations and other commitments. I think beginning
>>> the RC process in early January would give us the best coverage in the
>>> shortest amount of time.
>>> >> 2. There are several large initiatives in progress that given a
>>> little more time would leave us with a much more exciting 2.3 release.
>>> Specifically, the work on the history server, Kubernetes and continuous
>>> processing.
>>> >> 3. Given the actual release date of Spark 2.2, I think we'll still
>>> get Spark 2.3 out roughly 6 months after.
>>> >>
>>> >> Thoughts?
>>> >>
>>> >> Michael
>>> >
>>> >
>>>
>>>
>>> -
>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>
>>>
>>