Re: [VOTE] Release Apache Spark 1.2.1 (RC2)

The windows issue reported only affects actually running Spark on Windows (not job submission). However, I agree it's worth cutting a new RC. I'm going to cancel this vote and propose RC3 with a single additional patch. Let's try to vote that through so we can ship Spark 1.2.1. - Patrick On Sat,

[VOTE] Release Apache Spark 1.2.1 (RC3)

Please vote on releasing the following candidate as Apache Spark version 1.2.1! The tag to be voted on is v1.2.1-rc3 (commit b6eaf77): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=b6eaf77d4332bfb0a698849b1f5f917d20d70e97 The release files, including signatures, digests, etc.

Re: Questions about Spark standalone resource scheduler

Hey Jerry, I think standalone mode will still add more features over time, but the goal isn't really for it to become equivalent to what Mesos/YARN are today. Or at least, I doubt Spark Standalone will ever attempt to manage _other_ frameworks outside of Spark and become a general purpose

Re: Questions about Spark standalone resource scheduler

Hey Jerry, I think standalone mode will still add more features over time, but the goal isn't really for it to become equivalent to what Mesos/YARN are today. Or at least, I doubt Spark Standalone will ever attempt to manage _other_ frameworks outside of Spark and become a general purpose

[jira] [Resolved] (SPARK-5492) Thread statistics can break with older Hadoop versions

[ https://issues.apache.org/jira/browse/SPARK-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5492. Resolution: Fixed Fix Version/s: 1.3.0 Thanks, Sandy. Thread statistics can break

[jira] [Updated] (SPARK-5478) Add miss right parenthesis in Stage page Pending stages label

[ https://issues.apache.org/jira/browse/SPARK-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5478: --- Affects Version/s: 1.3.0 Add miss right parenthesis in Stage page Pending stages label

[jira] [Resolved] (SPARK-5478) Add miss right parenthesis in Stage page Pending stages label

[ https://issues.apache.org/jira/browse/SPARK-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5478. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Saisai Shao Add miss

[jira] [Resolved] (SPARK-5208) Add more documentation to Netty-based configs

[ https://issues.apache.org/jira/browse/SPARK-5208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5208. Resolution: Won't Fix Add more documentation to Netty-based configs

[jira] [Resolved] (SPARK-5353) Log failures in ExceutorClassLoader

[ https://issues.apache.org/jira/browse/SPARK-5353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5353. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Tobias Schlatter Log

[jira] [Commented] (SPARK-5492) Thread statistics can break with older Hadoop versions

[ https://issues.apache.org/jira/browse/SPARK-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300905#comment-14300905 ] Patrick Wendell commented on SPARK-5492: [~sandyr] I share your confusion, Sandy

[jira] [Resolved] (SPARK-3996) Shade Jetty in Spark deliverables

[ https://issues.apache.org/jira/browse/SPARK-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3996. Resolution: Fixed I've merged a new patch so closing this for now. Shade Jetty in Spark

[jira] [Commented] (SPARK-5515) Build fails with spark-ganglia-lgpl profile

[ https://issues.apache.org/jira/browse/SPARK-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300894#comment-14300894 ] Patrick Wendell commented on SPARK-5515: [~andrewor14] is this fixed now? Build

[jira] [Updated] (SPARK-5508) [hive context] Unable to query array once saved as parquet

[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5508: --- Component/s: (was: spark sql) SQL [hive context] Unable to query array

[jira] [Updated] (SPARK-5508) [hive context] Unable to query array once saved as parquet

[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5508: --- Component/s: (was: Spark Core) spark sql [hive context] Unable to query

[jira] [Updated] (SPARK-1517) Publish nightly snapshots of documentation, maven artifacts, and binary builds

[ https://issues.apache.org/jira/browse/SPARK-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1517: --- Assignee: Nicholas Chammas Publish nightly snapshots of documentation, maven artifacts

[jira] [Updated] (SPARK-5500) Document that feeding hadoopFile into a shuffle operation will cause problems

[ https://issues.apache.org/jira/browse/SPARK-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5500: --- Priority: Critical (was: Major) Document that feeding hadoopFile into a shuffle operation

[jira] [Updated] (SPARK-5197) Support external shuffle service in fine-grained mode on mesos cluster

[ https://issues.apache.org/jira/browse/SPARK-5197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5197: --- Fix Version/s: (was: 1.3.0) Support external shuffle service in fine-grained mode

[jira] [Updated] (SPARK-1517) Publish nightly snapshots of documentation, maven artifacts, and binary builds

[ https://issues.apache.org/jira/browse/SPARK-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1517: --- Target Version/s: 1.4.0 Publish nightly snapshots of documentation, maven artifacts

[jira] [Updated] (SPARK-1517) Publish nightly snapshots of documentation, maven artifacts, and binary builds

[ https://issues.apache.org/jira/browse/SPARK-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-1517: --- Target Version/s: (was: 1.3.0) Publish nightly snapshots of documentation, maven artifacts

[jira] [Commented] (SPARK-4114) Use stable Hive API (if one exists) for communication with Metastore

[ https://issues.apache.org/jira/browse/SPARK-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296562#comment-14296562 ] Patrick Wendell commented on SPARK-4114: Dropping target version 1.3 becasue we

[jira] [Updated] (SPARK-4628) Put external projects and examples behind a build flag

[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4628: --- Priority: Major (was: Blocker) Put external projects and examples behind a build flag

[jira] [Updated] (SPARK-4114) Use stable Hive API (if one exists) for communication with Metastore

[ https://issues.apache.org/jira/browse/SPARK-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4114: --- Target Version/s: (was: 1.3.0) Use stable Hive API (if one exists) for communication

[jira] [Updated] (SPARK-4628) Put external projects and examples behind a build flag

[ https://issues.apache.org/jira/browse/SPARK-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4628: --- Target Version/s: (was: 1.3.0) Put external projects and examples behind a build flag

[jira] [Updated] (SPARK-4923) Add Developer API to REPL to allow re-publishing the REPL jar

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4923: --- Fix Version/s: 1.3.0 Add Developer API to REPL to allow re-publishing the REPL jar

[jira] [Commented] (SPARK-5466) Build Error caused by Guava shading in Spark

[ https://issues.apache.org/jira/browse/SPARK-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296573#comment-14296573 ] Patrick Wendell commented on SPARK-5466: Okay Maven is reproducing this now even

[jira] [Resolved] (SPARK-5466) Build Error caused by Guava shading in Spark

[ https://issues.apache.org/jira/browse/SPARK-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5466. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Marcelo Vanzin Thanks

[jira] [Updated] (SPARK-3778) newAPIHadoopRDD doesn't properly pass credentials for secure hdfs on yarn

[ https://issues.apache.org/jira/browse/SPARK-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3778: --- Priority: Critical (was: Major) newAPIHadoopRDD doesn't properly pass credentials

[jira] [Resolved] (SPARK-3996) Shade Jetty in Spark deliverables

[ https://issues.apache.org/jira/browse/SPARK-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3996. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Patrick Wendell

[jira] [Commented] (SPARK-5492) Thread statistics can break with older Hadoop versions

[ https://issues.apache.org/jira/browse/SPARK-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14298075#comment-14298075 ] Patrick Wendell commented on SPARK-5492: /cc [~sandyr] Thread statistics can

[jira] [Updated] (SPARK-5492) Thread statistics can break with older Hadoop versions

[ https://issues.apache.org/jira/browse/SPARK-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5492: --- Priority: Blocker (was: Major) Thread statistics can break with older Hadoop versions

[jira] [Updated] (SPARK-3778) newAPIHadoopRDD doesn't properly pass credentials for secure hdfs on yarn

[ https://issues.apache.org/jira/browse/SPARK-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3778: --- Target Version/s: 1.3.0 (was: 1.1.1, 1.2.0) newAPIHadoopRDD doesn't properly pass

[jira] [Created] (SPARK-5492) Thread statistics can break with older Hadoop versions

Patrick Wendell created SPARK-5492: -- Summary: Thread statistics can break with older Hadoop versions Key: SPARK-5492 URL: https://issues.apache.org/jira/browse/SPARK-5492 Project: Spark

[jira] [Reopened] (SPARK-3996) Shade Jetty in Spark deliverables

[ https://issues.apache.org/jira/browse/SPARK-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-3996: This was causing compiler failures in the master build, so I reverted it. I think it's

[jira] [Commented] (SPARK-5428) Declare the 'assembly' module at the bottom of the modules element in the parent POM

[ https://issues.apache.org/jira/browse/SPARK-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294935#comment-14294935 ] Patrick Wendell commented on SPARK-5428: [~tzolov] Do you mind explaining a bit

[RESULT] [VOTE] Release Apache Spark 1.2.1 (RC1)

And Scale OK Fixed : org.apache.spark.SparkException in zip ! 2.5. rdd operations OK State of the Union Texts - MapReduce, Filter,sortByKey (word count) 2.6. recommendation OK Cheers k/ On Mon, Jan 26, 2015 at 11:02 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote

Re: [VOTE] Release Apache Spark 1.2.1 (RC2)

://issues.apache.org/jira/browse/SPARK-5144 Thanks, Aniket On Wed Jan 28 2015 at 15:39:43 Patrick Wendell [via Apache Spark Developers List] ml-node+s1001551n1031...@n3.nabble.com wrote: Minor typo in the above e-mail - the tag is named v1.2.1-rc2 (not v1.2.1-rc1). On Wed, Jan 28, 2015 at 2:06 AM

[jira] [Resolved] (SPARK-5415) Upgrade sbt to 0.13.7

[ https://issues.apache.org/jira/browse/SPARK-5415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5415. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Ryan Williams Upgrade sbt

[jira] [Updated] (SPARK-5341) Support maven coordinates in spark-shell and spark-submit

[ https://issues.apache.org/jira/browse/SPARK-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5341: --- Priority: Critical (was: Major) Support maven coordinates in spark-shell and spark-submit

[jira] [Commented] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14294928#comment-14294928 ] Patrick Wendell commented on SPARK-5420: How about just load and store

[jira] [Resolved] (SPARK-5144) spark-yarn module should be published

[ https://issues.apache.org/jira/browse/SPARK-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5144. Resolution: Duplicate spark-yarn module should be published

[jira] [Resolved] (SPARK-4809) Improve Guava shading in Spark

[ https://issues.apache.org/jira/browse/SPARK-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4809. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Marcelo Vanzin Improve

[VOTE] Release Apache Spark 1.2.1 (RC2)

Please vote on releasing the following candidate as Apache Spark version 1.2.1! The tag to be voted on is v1.2.1-rc1 (commit b77f876): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=b77f87673d1f9f03d4c83cf583158227c551359b The release files, including signatures, digests, etc.

Re: [VOTE] Release Apache Spark 1.2.1 (RC2)

Minor typo in the above e-mail - the tag is named v1.2.1-rc2 (not v1.2.1-rc1). On Wed, Jan 28, 2015 at 2:06 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.2.1! The tag to be voted on is v1.2.1-rc1 (commit b77f876

[jira] [Resolved] (SPARK-5458) Refer to aggregateByKey instead of combineByKey in docs

[ https://issues.apache.org/jira/browse/SPARK-5458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5458. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Sandy Ryza Refer

[jira] [Resolved] (SPARK-5188) make-distribution.sh should support curl, not only wget to get Tachyon

[ https://issues.apache.org/jira/browse/SPARK-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5188. Resolution: Fixed Assignee: Kousuke Saruta make-distribution.sh should support curl

[jira] [Updated] (SPARK-5188) make-distribution.sh should support curl, not only wget to get Tachyon

[ https://issues.apache.org/jira/browse/SPARK-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5188: --- Fix Version/s: 1.3.0 make-distribution.sh should support curl, not only wget to get Tachyon

Re: spark akka fork : is the source anywhere?

It's maintained here: https://github.com/pwendell/akka/tree/2.2.3-shaded-proto Over time, this is something that would be great to get rid of, per rxin On Wed, Jan 28, 2015 at 3:33 PM, Reynold Xin r...@databricks.com wrote: Hopefully problems like this will go away entirely in the next couple

[jira] [Comment Edited] (SPARK-4049) Storage web UI fraction cached shows as 100%

[ https://issues.apache.org/jira/browse/SPARK-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296479#comment-14296479 ] Patrick Wendell edited comment on SPARK-4049 at 1/29/15 6:58 AM

[jira] [Commented] (SPARK-4049) Storage web UI fraction cached shows as 100%

[ https://issues.apache.org/jira/browse/SPARK-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296479#comment-14296479 ] Patrick Wendell commented on SPARK-4049: [~skrasser] Yes - I agree that behavior

[jira] [Updated] (SPARK-5466) Build Error caused by Guava shading in Spark

[ https://issues.apache.org/jira/browse/SPARK-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5466: --- Component/s: Build Build Error caused by Guava shading in Spark

[jira] [Commented] (SPARK-5466) Build Error caused by Guava shading in Spark

[ https://issues.apache.org/jira/browse/SPARK-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296483#comment-14296483 ] Patrick Wendell commented on SPARK-5466: Also - [~srowen] can you reproduce

[jira] [Updated] (SPARK-5466) Build Error caused by Guava shading in Spark

[ https://issues.apache.org/jira/browse/SPARK-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5466: --- Priority: Blocker (was: Major) Build Error caused by Guava shading in Spark

[jira] [Commented] (SPARK-5466) Build Error caused by Guava shading in Spark

[ https://issues.apache.org/jira/browse/SPARK-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14296482#comment-14296482 ] Patrick Wendell commented on SPARK-5466: I sent [~vanzin] and e-mail today about

[jira] [Resolved] (SPARK-5471) java.lang.NumberFormatException: For input string:

[ https://issues.apache.org/jira/browse/SPARK-5471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5471. Resolution: Not a Problem Resolving per your own comment

[jira] [Resolved] (SPARK-2476) Have sbt-assembly include runtime dependencies in jar

[ https://issues.apache.org/jira/browse/SPARK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2476. Resolution: Not a Problem [~srowen] Nope, I think we found a workaround. Have sbt

[jira] [Resolved] (SPARK-2487) Follow up from SBT build refactor (i.e. SPARK-1776)

[ https://issues.apache.org/jira/browse/SPARK-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2487. Resolution: Fixed Follow up from SBT build refactor (i.e. SPARK-1776

Re: [VOTE] Release Apache Spark 1.2.1 (RC1)

, Sean On Jan 27, 2015, at 12:04 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.2.1! The tag to be voted on is v1.2.1-rc1 (commit 3e2d7d3): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h

[jira] [Resolved] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5308. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Assignee

[jira] [Resolved] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5299. Resolution: Fixed Thanks I've fixed this. Is http://www.apache.org/dist/spark/KEYS out

[jira] [Reopened] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-5299: Actually I need to deal with past releases as well, so re-opening. Is http://www.apache.org

Re: [VOTE] Release Apache Spark 1.2.1 (RC1)

, at 11:35 AM, Patrick Wendell pwend...@gmail.com wrote: Hey Sean, Right now we don't publish every 2.11 binary to avoid combinatorial explosion of the number of build artifacts we publish (there are other parameters such as whether hive is included, etc). We can revisit this in future feature

[jira] [Resolved] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5299. Resolution: Fixed Okay I've now added every key ever used to publish a Spark release (I

Re: [VOTE] Release Apache Spark 1.2.1 (RC1)

create: configuration createChecksumtrue/createChecksum /configuration As for the key issue, I think it's just a matter of uploading the new key in both places. We should all of course test the release anyway. On Tue, Jan 27, 2015 at 5:55 PM, Patrick Wendell pwend...@gmail.com wrote: Hey

[jira] [Resolved] (SPARK-5199) Input metrics should show up for InputFormats that return CombineFileSplits

[ https://issues.apache.org/jira/browse/SPARK-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5199. Resolution: Fixed Fix Version/s: 1.3.0 Input metrics should show up

Friendly reminder/request to help with reviews!

Hey All, Just a reminder, as always around release time we have a very large volume of patches show up near the deadline. One thing that can help us maximize the number of patches we get in is to have community involvement in performing code reviews. And in particular, doing a thorough review

[jira] [Updated] (SPARK-5441) SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

[ https://issues.apache.org/jira/browse/SPARK-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5441: --- Assignee: Michael Nazario SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

[jira] [Updated] (SPARK-5441) SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

[ https://issues.apache.org/jira/browse/SPARK-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5441: --- Target Version/s: 1.3.0 SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

[jira] [Resolved] (SPARK-5339) build/mvn doesn't work because of invalid URL for maven's tgz.

[ https://issues.apache.org/jira/browse/SPARK-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5339. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Kousuke Saruta build/mvn

[VOTE] Release Apache Spark 1.2.1 (RC1)

Please vote on releasing the following candidate as Apache Spark version 1.2.1! The tag to be voted on is v1.2.1-rc1 (commit 3e2d7d3): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e2d7d310b76c293b9ac787f204e6880f508f6ec The release files, including signatures, digests, etc.

[jira] [Updated] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5420: --- Summary: Cross-langauge load/store functions for creating and saving DataFrames (was: Create

[jira] [Created] (SPARK-5420) Create cross-langauge load/store functions for creating and saving DataFrames

Patrick Wendell created SPARK-5420: -- Summary: Create cross-langauge load/store functions for creating and saving DataFrames Key: SPARK-5420 URL: https://issues.apache.org/jira/browse/SPARK-5420

[jira] [Resolved] (SPARK-5052) com.google.common.base.Optional binary has a wrong method signatures

[ https://issues.apache.org/jira/browse/SPARK-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5052. Resolution: Fixed Fix Version/s: 1.3.0 com.google.common.base.Optional binary has

[jira] [Updated] (SPARK-5052) com.google.common.base.Optional binary has a wrong method signatures

[ https://issues.apache.org/jira/browse/SPARK-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5052: --- Assignee: Elmer Garduno com.google.common.base.Optional binary has a wrong method signatures

[jira] [Resolved] (SPARK-4147) Reduce log4j dependency

[ https://issues.apache.org/jira/browse/SPARK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4147. Resolution: Fixed Reduce log4j dependency --- Key

[jira] [Updated] (SPARK-4147) Reduce log4j dependency

[ https://issues.apache.org/jira/browse/SPARK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4147: --- Affects Version/s: 1.2.0 Reduce log4j dependency

[jira] [Updated] (SPARK-4147) Reduce log4j dependency

[ https://issues.apache.org/jira/browse/SPARK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4147: --- Fix Version/s: 1.2.1 1.3.0 Reduce log4j dependency

[jira] [Updated] (SPARK-4147) Reduce log4j dependency

[ https://issues.apache.org/jira/browse/SPARK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4147: --- Assignee: Sean Owen Reduce log4j dependency --- Key

Re: Standardized Spark dev environment

, this will at least serve as an up-to-date list of packages/versions they should try to install locally in whatever environment they have. - Patrick On Wed, Jan 21, 2015 at 5:42 AM, Will Benton wi...@redhat.com wrote: - Original Message - From: Patrick Wendell pwend...@gmail.com To: Sean

[jira] [Updated] (SPARK-5275) pyspark.streaming is not included in assembly jar

[ https://issues.apache.org/jira/browse/SPARK-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5275: --- Fix Version/s: 1.2.1 1.3.0 pyspark.streaming is not included in assembly

[jira] [Updated] (SPARK-4939) Python updateStateByKey example hang in local mode

[ https://issues.apache.org/jira/browse/SPARK-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4939: --- Target Version/s: 1.3.0 (was: 1.3.0, 1.2.1) Python updateStateByKey example hang in local

[jira] [Commented] (SPARK-4939) Python updateStateByKey example hang in local mode

[ https://issues.apache.org/jira/browse/SPARK-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14286234#comment-14286234 ] Patrick Wendell commented on SPARK-4939: [~tdas] [~davies] [~kayousterhout

[jira] [Resolved] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3958. Resolution: Fixed Target Version/s: (was: 1.2.1) At this point I'm not aware

[jira] [Resolved] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4105. Resolution: Fixed Target Version/s: (was: 1.2.1) At this point I'm not aware

Re: Standardized Spark dev environment

If the goal is a reproducible test environment then I think that is what Jenkins is. Granted you can only ask it for a test. But presumably you get the same result if you start from the same VM image as Jenkins and run the same steps. But the issue is when users can't reproduce Jenkins

[jira] [Updated] (SPARK-4923) Add Developer API to REPL to allow re-publishing the REPL jar

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4923: --- Summary: Add Developer API to REPL to allow re-publishing the REPL jar (was: Maven build

[jira] [Updated] (SPARK-4923) Add Developer API to REPL to allow re-publishing the REPL jar

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4923: --- Assignee: Chip Senkbeil Add Developer API to REPL to allow re-publishing the REPL jar

[jira] [Updated] (SPARK-5289) Backport publishing of repl, yarn into branch-1.2

[ https://issues.apache.org/jira/browse/SPARK-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5289: --- Fix Version/s: 1.2.1 Backport publishing of repl, yarn into branch-1.2

[jira] [Resolved] (SPARK-4923) Add Developer API to REPL to allow re-publishing the REPL jar

[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4923. Resolution: Fixed Target Version/s: 1.3.0 (was: 1.3.0, 1.2.1) I updated

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Priority: Blocker (was: Critical) Attributes are case sensitive when using a select query

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Assignee: Cheng Hao Attributes are case sensitive when using a select query from

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Fix Version/s: 1.3.0 Attributes are case sensitive when using a select query from

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Fix Version/s: (was: 1.2.1) Attributes are case sensitive when using a select query from

[jira] [Commented] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285262#comment-14285262 ] Patrick Wendell commented on SPARK-4959: Excuse my last comment

[jira] [Comment Edited] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285258#comment-14285258 ] Patrick Wendell edited comment on SPARK-4959 at 1/21/15 6:47 AM

[jira] [Resolved] (SPARK-5276) pyspark.streaming is not included in assembly jar

[ https://issues.apache.org/jira/browse/SPARK-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5276. Resolution: Duplicate pyspark.streaming is not included in assembly jar

[jira] [Commented] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14285258#comment-14285258 ] Patrick Wendell commented on SPARK-4959: Note that in the 1.2 branch

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Fix Version/s: 1.2.1 Attributes are case sensitive when using a select query from

Re: Standardized Spark dev environment

To respond to the original suggestion by Nick. I always thought it would be useful to have a Docker image on which we run the tests and build releases, so that we could have a consistent environment that other packagers or people trying to exhaustively run Spark tests could replicate (or at least

[jira] [Updated] (SPARK-5297) File Streams do not work with custom key/values

[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5297: --- Target Version/s: 1.3.0, 1.2.1 File Streams do not work with custom key/values

[jira] [Updated] (SPARK-5297) File Streams do not work with custom key/values

[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5297: --- Fix Version/s: 1.3.0 File Streams do not work with custom key/values

<    6   7   8   9   10   11   12   13   14   15   >