Re: 1.2.1-rc3 - Avro input format for Hadoop 2 broken/fix?

2015-02-04 Thread M. Dale

   On 02/04/2015 02:04 PM, Josh Rosen wrote:

It looks like you replied just to me; mind CC’ing the mailing list, too?


On February 4, 2015 at 11:02:34 AM, M. Dale (medal...@yahoo.com 
) wrote:



Josh,
That was a bug that was present earlier. It was marked as fixed in 
1.2.0 but it had not been fixed.


Thanks,
Markus


On Wednesday, February 4, 2015 1:04 PM, Josh Rosen 
 wrote:



Just to clarify, is this a regression from Spark 1.2.0 or is it a bug 
that was present in older versions?  Unless this is a 1.2.0->1.2.1 
regression, I don't think this can make it into 1.2.1 and will have 
to wait for 1.2.2.


- Josh

Sent from my phone

> On Feb 4, 2015, at 7:30 AM, "M. Dale" > wrote:

>
> SPARK-3039 "Spark assembly for new hadoop API (hadoop 2) contains 
avro-mapred for hadoop 1 API" was reopened
> and prevents v.1.2.1-rc3 from using Avro Input format for Hadoop 2 
API/instances (it includes the hadoop1 avro-mapred library files).

>
> What are the chances of getting the fix outlined here 
(https://github.com/medale/spark/compare/apache:v1.2.1-rc3...avro-hadoop2-v1.2.1-rc2) 
included in 1.2.1? My apologies, I do not know how to generate a pull 
request against a tag version.

>
> I did add pull request https://github.com/apache/spark/pull/4315 
for the current 1.3.0-SNAPSHOT master on this issue. Even though 
1.3.0 build already does not include avro-mapred in the spark 
assembly jar this minor change improves dependence convergence.

>
> Thanks,
> Markus

>
> -
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org 

> For additional commands, e-mail: dev-h...@spark.apache.org 



>






Re: 1.2.1-rc3 - Avro input format for Hadoop 2 broken/fix?

2015-02-04 Thread Patrick Wendell
Hi Markus,

That won't be included in 1.2.1 most likely because the release votes
have already started, and at that point we don't hold the release
except for major regression issues from 1.2.0. However, if this goes
through we can backport it into the 1.2 branch and it will end up in a
future maintenance release, or you can just build spark from that
branch as soon as it's in there.

- Patric

On Wed, Feb 4, 2015 at 7:30 AM, M. Dale  wrote:
> SPARK-3039 "Spark assembly for new hadoop API (hadoop 2) contains
> avro-mapred for hadoop 1 API" was reopened
> and prevents v.1.2.1-rc3 from using Avro Input format for Hadoop 2
> API/instances (it includes the hadoop1 avro-mapred library files).
>
> What are the chances of getting the fix outlined here
> (https://github.com/medale/spark/compare/apache:v1.2.1-rc3...avro-hadoop2-v1.2.1-rc2)
> included in 1.2.1? My apologies, I do not know how to generate a pull
> request against a tag version.
>
> I did add pull request https://github.com/apache/spark/pull/4315 for the
> current 1.3.0-SNAPSHOT master on this issue. Even though 1.3.0 build already
> does not include avro-mapred in the spark assembly jar this minor change
> improves dependence convergence.
>
> Thanks,
> Markus
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org