[jira] [Resolved] (SPARK-3450) Enable specifiying the --jars CLI option multiple times

2015-01-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3450. Resolution: Won't Fix I'd prefer not to do this one, it complicates our parsing substantiall

[jira] [Updated] (SPARK-3340) Deprecate ADD_JARS and ADD_FILES

2015-01-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3340: --- Labels: starter (was: ) > Deprecate ADD_JARS and ADD_FILES >

[jira] [Updated] (SPARK-5166) Stabilize Spark SQL APIs

2015-01-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5166: --- Priority: Blocker (was: Critical) > Stabilize Spark SQL APIs > > >

[jira] [Commented] (SPARK-3561) Allow for pluggable execution contexts in Spark

2015-01-11 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14273225#comment-14273225 ] Patrick Wendell commented on SPARK-3561: So if the question is: "Is Spark only API

[jira] [Updated] (SPARK-5102) CompressedMapStatus needs to be registered with Kryo

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5102: --- Target Version/s: 1.2.1 Assignee: Lianhui Wang > CompressedMapStatus needs to be r

[jira] [Resolved] (SPARK-5102) CompressedMapStatus needs to be registered with Kryo

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5102. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Fixed by: https://git

[jira] [Resolved] (SPARK-5078) Allow setting Akka host name from env vars

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5078. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 > Allow setting Akka

[jira] [Resolved] (SPARK-5172) spark-examples-***.jar shades a wrong Hadoop distribution

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5172. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Sean Owen > spark-examples-

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274239#comment-14274239 ] Patrick Wendell commented on SPARK-4923: Hey All, Sorry this has caused a disrupt

[jira] [Comment Edited] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274239#comment-14274239 ] Patrick Wendell edited comment on SPARK-4923 at 1/12/15 9:58 PM: ---

[jira] [Commented] (SPARK-4923) Maven build should keep publishing spark-repl

2015-01-12 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14274263#comment-14274263 ] Patrick Wendell commented on SPARK-4923: [~senkwich] definitely prefer github. >

[jira] [Resolved] (SPARK-4857) Add Executor Events to SparkListener

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4857. Resolution: Fixed Fix Version/s: 1.3.0 > Add Executor Events to SparkListener > -

[jira] [Resolved] (SPARK-4092) Input metrics don't work for coalesce()'d RDD's

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4092. Resolution: Fixed Fix Version/s: 1.3.0 > Input metrics don't work for coalesce()'d RD

[jira] [Resolved] (SPARK-2630) Input data size of CoalescedRDD is incorrect

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2630. Resolution: Duplicate I think this is a dup of SPARK-4092. > Input data size of CoalescedRD

[jira] [Updated] (SPARK-4955) Dynamic allocation doesn't work in YARN cluster mode

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4955: --- Priority: Blocker (was: Critical) > Dynamic allocation doesn't work in YARN cluster mode > --

[jira] [Updated] (SPARK-4955) Dynamic allocation doesn't work in YARN cluster mode

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4955: --- Target Version/s: 1.3.0 > Dynamic allocation doesn't work in YARN cluster mode > -

[jira] [Commented] (SPARK-5216) Spark Ui should report estimated time remaining for each stage.

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279863#comment-14279863 ] Patrick Wendell commented on SPARK-5216: This has been proposed before, but in the

[jira] [Updated] (SPARK-5176) Thrift server fails with confusing error message when deploy-mode is cluster

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5176: --- Labels: starter (was: ) > Thrift server fails with confusing error message when deploy-mode i

[jira] [Commented] (SPARK-5176) Thrift server fails with confusing error message when deploy-mode is cluster

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279869#comment-14279869 ] Patrick Wendell commented on SPARK-5176: Yes, we should add a check here similar t

[jira] [Comment Edited] (SPARK-5176) Thrift server fails with confusing error message when deploy-mode is cluster

2015-01-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279869#comment-14279869 ] Patrick Wendell edited comment on SPARK-5176 at 1/16/15 6:28 AM: ---

[jira] [Resolved] (SPARK-4357) Modify release publishing to work with Scala 2.11

2015-01-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4357. Resolution: Fixed Sorry this is actually working now. We now publish artifacts for Scala 2.1

[jira] [Updated] (SPARK-5270) Elegantly check if RDD is empty

2015-01-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5270: --- Target Version/s: 1.3.0 > Elegantly check if RDD is empty > --- >

[jira] [Updated] (SPARK-5260) Expose JsonRDD.allKeysWithValueTypes() in a utility class

2015-01-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5260: --- Fix Version/s: (was: 1.3.0) > Expose JsonRDD.allKeysWithValueTypes() in a utility class >

[jira] [Created] (SPARK-5289) Backport publishing of repl, yarn, and hive-thriftserver into branch-1.2

2015-01-16 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-5289: -- Summary: Backport publishing of repl, yarn, and hive-thriftserver into branch-1.2 Key: SPARK-5289 URL: https://issues.apache.org/jira/browse/SPARK-5289 Project: S

[jira] [Updated] (SPARK-5289) Backport publishing of repl, yarn into branch-1.2

2015-01-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5289: --- Summary: Backport publishing of repl, yarn into branch-1.2 (was: Backport publishing of repl,

[jira] [Updated] (SPARK-5289) Backport publishing of repl, yarn into branch-1.2

2015-01-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5289: --- Description: In SPARK-3452 we did some clean-up of published artifacts that turned out to adv

[jira] [Updated] (SPARK-5289) Backport publishing of repl, yarn into branch-1.2

2015-01-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5289: --- Description: In SPARK-3452 we did some clean-up of published artifacts that turned out to adve

[jira] [Updated] (SPARK-5289) Backport publishing of repl, yarn into branch-1.2

2015-01-16 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5289: --- Description: In SPARK-3452 we did some clean-up of published artifacts that turned out to adv

[jira] [Updated] (SPARK-5096) SparkBuild.scala assumes you are at the spark root dir

2015-01-17 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5096: --- Target Version/s: (was: 1.0.3) > SparkBuild.scala assumes you are at the spark root dir > --

[jira] [Resolved] (SPARK-5096) SparkBuild.scala assumes you are at the spark root dir

2015-01-17 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5096. Resolution: Fixed Fix Version/s: 1.3.0 > SparkBuild.scala assumes you are at the spar

[jira] [Resolved] (SPARK-5289) Backport publishing of repl, yarn into branch-1.2

2015-01-17 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5289. Resolution: Fixed > Backport publishing of repl, yarn into branch-1.2 >

[jira] [Resolved] (SPARK-3694) Allow printing object graph of tasks/RDD's with a debug flag

2015-01-17 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3694. Resolution: Duplicate > Allow printing object graph of tasks/RDD's with a debug flag > -

[jira] [Resolved] (SPARK-5249) Added setX functions to set a Boolean, Int, Float and Double parameters with a "specialized" function.

2015-01-18 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5249. Resolution: Won't Fix Per discussion on the issue we've decided to just ask users to convert

[jira] [Commented] (SPARK-5249) In SparkConf accept value with Any type and perform string conversion

2015-01-18 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282062#comment-14282062 ] Patrick Wendell commented on SPARK-5249: I also updated the title to reflect what

[jira] [Updated] (SPARK-5249) In SparkConf accept value with Any type and perform string conversion

2015-01-18 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5249: --- Summary: In SparkConf accept value with Any type and perform string conversion (was: Added se

[jira] [Resolved] (SPARK-5217) Spark UI should report pending stages during job execution on AllStagesPage.

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5217. Resolution: Fixed Fix Version/s: 1.3.0 > Spark UI should report pending stages during

[jira] [Resolved] (SPARK-3288) All fields in TaskMetrics should be private and use getters/setters

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3288. Resolution: Fixed Fix Version/s: 1.3.0 Target Version/s: 1.3.0 (was: 1.2.0)

[jira] [Updated] (SPARK-3288) All fields in TaskMetrics should be private and use getters/setters

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3288: --- Assignee: Ilya Ganelin (was: Dale Richardson) > All fields in TaskMetrics should be private a

[jira] [Resolved] (SPARK-3758) Script style checking

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3758. Resolution: Won't Fix This patch ended up being so large, I think we're gonna pass on it. >

[jira] [Resolved] (SPARK-2595) The driver run garbage collection, when the executor throws OutOfMemoryError exception

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-2595. Resolution: Won't Fix Per PR comment, closing this for now. > The driver run garbage collec

[jira] [Resolved] (SPARK-4417) New API: sample RDD to fixed number of items

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4417. Resolution: Won't Fix Assignee: Ilya Ganelin [~ilganeli] ended up taking a crack a thi

[jira] [Resolved] (SPARK-5088) Use spark-class for running executors directly on mesos

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5088. Resolution: Fixed Assignee: Jongyoul Lee > Use spark-class for running executors direc

[jira] [Updated] (SPARK-5297) File Streams do not work with custom key/values

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5297: --- Description: The following code: {code} stream_context.>fileStream(directory) .foreachRDD(new

[jira] [Updated] (SPARK-5270) Provide isEmpty utility function in RDD API

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5270: --- Assignee: Sean Owen > Provide isEmpty utility function in RDD API > --

[jira] [Updated] (SPARK-5270) Provide isEmpty utility function in RDD API

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5270: --- Summary: Provide isEmpty utility function in RDD API (was: Elegantly check if RDD is empty)

[jira] [Updated] (SPARK-5270) Provide isEmpty() function in RDD API

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5270: --- Summary: Provide isEmpty() function in RDD API (was: Provide isEmpty utility function in RDD

[jira] [Resolved] (SPARK-5270) Provide isEmpty() function in RDD API

2015-01-19 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5270. Resolution: Fixed Fix Version/s: 1.3.0 > Provide isEmpty() function in RDD API >

[jira] [Resolved] (SPARK-4660) JavaSerializer uses wrong classloader

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4660. Resolution: Fixed Fix Version/s: 1.2.1 1.1.2 1.3

[jira] [Updated] (SPARK-4660) JavaSerializer uses wrong classloader

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4660: --- Assignee: Piotr Kołaczkowski > JavaSerializer uses wrong classloader > ---

[jira] [Resolved] (SPARK-5329) UIWorkloadGenerator should stop SparkContext.

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5329. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Kousuke Saruta > UIWorkload

[jira] [Resolved] (SPARK-5276) pyspark.streaming is not included in assembly jar

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5276. Resolution: Duplicate > pyspark.streaming is not included in assembly jar >

[jira] [Updated] (SPARK-5289) Backport publishing of repl, yarn into branch-1.2

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5289: --- Fix Version/s: 1.2.1 > Backport publishing of repl, yarn into branch-1.2 > ---

[jira] [Updated] (SPARK-4923) Add Developer API to REPL to allow re-publishing the REPL jar

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4923: --- Summary: Add Developer API to REPL to allow re-publishing the REPL jar (was: Maven build shou

[jira] [Updated] (SPARK-4923) Add Developer API to REPL to allow re-publishing the REPL jar

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4923: --- Assignee: Chip Senkbeil > Add Developer API to REPL to allow re-publishing the REPL jar >

[jira] [Resolved] (SPARK-4923) Add Developer API to REPL to allow re-publishing the REPL jar

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4923. Resolution: Fixed Target Version/s: 1.3.0 (was: 1.3.0, 1.2.1) I updated the title

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Priority: Blocker (was: Critical) > Attributes are case sensitive when using a select query f

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Assignee: Cheng Hao > Attributes are case sensitive when using a select query from a projectio

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Fix Version/s: 1.3.0 > Attributes are case sensitive when using a select query from a projecti

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Target Version/s: 1.3.0, 1.2.1 (was: 1.3.0) > Attributes are case sensitive when using a sele

[jira] [Commented] (SPARK-4996) Memory leak?

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285255#comment-14285255 ] Patrick Wendell commented on SPARK-4996: I'm de-escalating this right now because

[jira] [Updated] (SPARK-4996) Memory leak?

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4996: --- Priority: Major (was: Blocker) > Memory leak? > > > Key: SPARK-4

[jira] [Resolved] (SPARK-5275) pyspark.streaming is not included in assembly jar

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5275. Resolution: Fixed Assignee: Davies Liu > pyspark.streaming is not included in assembly

[jira] [Commented] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285258#comment-14285258 ] Patrick Wendell commented on SPARK-4959: Note that in the 1.2 branch this was fixe

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Fix Version/s: 1.2.1 > Attributes are case sensitive when using a select query from a projecti

[jira] [Comment Edited] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285258#comment-14285258 ] Patrick Wendell edited comment on SPARK-4959 at 1/21/15 6:47 AM: ---

[jira] [Updated] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Fix Version/s: (was: 1.2.1) > Attributes are case sensitive when using a select query from

[jira] [Commented] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285262#comment-14285262 ] Patrick Wendell commented on SPARK-4959: Excuse my last comment, it was on the wro

[jira] [Issue Comment Deleted] (SPARK-4959) Attributes are case sensitive when using a select query from a projection

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4959: --- Comment: was deleted (was: Note that in the 1.2 branch this was fixed by https://github.com/a

[jira] [Commented] (SPARK-4296) Throw "Expression not in GROUP BY" when using same expression in group by clause and select clause

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14285263#comment-14285263 ] Patrick Wendell commented on SPARK-4296: Note this was fixed in https://github.com

[jira] [Updated] (SPARK-4296) Throw "Expression not in GROUP BY" when using same expression in group by clause and select clause

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4296: --- Fix Version/s: 1.2.1 1.3.0 > Throw "Expression not in GROUP BY" when using

[jira] [Updated] (SPARK-5297) File Streams do not work with custom key/values

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5297: --- Labels: backport-needed (was: ) > File Streams do not work with custom key/values > -

[jira] [Updated] (SPARK-5297) File Streams do not work with custom key/values

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5297: --- Target Version/s: 1.3.0, 1.2.1 > File Streams do not work with custom key/values > ---

[jira] [Updated] (SPARK-5297) File Streams do not work with custom key/values

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5297: --- Fix Version/s: 1.3.0 > File Streams do not work with custom key/values > -

[jira] [Updated] (SPARK-5297) File Streams do not work with custom key/values

2015-01-20 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5297: --- Assignee: Saisai Shao > File Streams do not work with custom key/values >

[jira] [Updated] (SPARK-5275) pyspark.streaming is not included in assembly jar

2015-01-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5275: --- Fix Version/s: 1.2.1 1.3.0 > pyspark.streaming is not included in assembly

[jira] [Updated] (SPARK-4939) Python updateStateByKey example hang in local mode

2015-01-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4939: --- Target Version/s: 1.3.0 (was: 1.3.0, 1.2.1) > Python updateStateByKey example hang in local m

[jira] [Commented] (SPARK-4939) Python updateStateByKey example hang in local mode

2015-01-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14286234#comment-14286234 ] Patrick Wendell commented on SPARK-4939: [~tdas] [~davies] [~kayousterhout] Becaus

[jira] [Resolved] (SPARK-3958) Possible stream-corruption issues in TorrentBroadcast

2015-01-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-3958. Resolution: Fixed Target Version/s: (was: 1.2.1) At this point I'm not aware of

[jira] [Resolved] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2015-01-21 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4105. Resolution: Fixed Target Version/s: (was: 1.2.1) At this point I'm not aware of

[jira] [Resolved] (SPARK-5339) build/mvn doesn't work because of invalid URL for maven's tgz.

2015-01-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5339. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Kousuke Saruta > build/mvn

[jira] [Updated] (SPARK-4147) Reduce log4j dependency

2015-01-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4147: --- Assignee: Sean Owen > Reduce log4j dependency > --- > > Ke

[jira] [Updated] (SPARK-4147) Reduce log4j dependency

2015-01-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4147: --- Affects Version/s: 1.2.0 > Reduce log4j dependency > --- > >

[jira] [Resolved] (SPARK-4147) Reduce log4j dependency

2015-01-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4147. Resolution: Fixed > Reduce log4j dependency > --- > > Ke

[jira] [Updated] (SPARK-4147) Reduce log4j dependency

2015-01-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-4147: --- Fix Version/s: 1.2.1 1.3.0 > Reduce log4j dependency >

[jira] [Resolved] (SPARK-5052) com.google.common.base.Optional binary has a wrong method signatures

2015-01-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5052. Resolution: Fixed Fix Version/s: 1.3.0 > com.google.common.base.Optional binary has a

[jira] [Updated] (SPARK-5052) com.google.common.base.Optional binary has a wrong method signatures

2015-01-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5052: --- Assignee: Elmer Garduno > com.google.common.base.Optional binary has a wrong method signatures

[jira] [Updated] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

2015-01-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5420: --- Summary: Cross-langauge load/store functions for creating and saving DataFrames (was: Create

[jira] [Created] (SPARK-5420) Create cross-langauge load/store functions for creating and saving DataFrames

2015-01-26 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-5420: -- Summary: Create cross-langauge load/store functions for creating and saving DataFrames Key: SPARK-5420 URL: https://issues.apache.org/jira/browse/SPARK-5420 Proje

[jira] [Resolved] (SPARK-5308) MD5 / SHA1 hash format doesn't match standard Maven output

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5308. Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Assignee: Sea

[jira] [Resolved] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5299. Resolution: Fixed Thanks I've fixed this. > Is http://www.apache.org/dist/spark/KEYS out of

[jira] [Reopened] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-5299: Actually I need to deal with past releases as well, so re-opening. > Is http://www.apache.org/d

[jira] [Resolved] (SPARK-5299) Is http://www.apache.org/dist/spark/KEYS out of date?

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5299. Resolution: Fixed Okay I've now added every key ever used to publish a Spark release (I thin

[jira] [Resolved] (SPARK-5199) Input metrics should show up for InputFormats that return CombineFileSplits

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5199. Resolution: Fixed Fix Version/s: 1.3.0 > Input metrics should show up for InputFormat

[jira] [Updated] (SPARK-5441) SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5441: --- Assignee: Michael Nazario > SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

[jira] [Updated] (SPARK-5441) SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs

2015-01-27 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5441: --- Target Version/s: 1.3.0 > SerDeUtil Pair RDD to python conversion doesn't accept empty RDDs >

[jira] [Resolved] (SPARK-4809) Improve Guava shading in Spark

2015-01-28 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-4809. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Marcelo Vanzin > Improve Gu

[jira] [Commented] (SPARK-5420) Cross-langauge load/store functions for creating and saving DataFrames

2015-01-28 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294928#comment-14294928 ] Patrick Wendell commented on SPARK-5420: How about just load and store then? > Cr

[jira] [Commented] (SPARK-5428) Declare the 'assembly' module at the bottom of the element in the parent POM

2015-01-28 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294935#comment-14294935 ] Patrick Wendell commented on SPARK-5428: [~tzolov] Do you mind explaining a bit mo

[jira] [Updated] (SPARK-5341) Support maven coordinates in spark-shell and spark-submit

2015-01-28 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-5341: --- Priority: Critical (was: Major) > Support maven coordinates in spark-shell and spark-submit >

[jira] [Resolved] (SPARK-5415) Upgrade sbt to 0.13.7

2015-01-28 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-5415. Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Ryan Williams > Upgrade sbt

<    11   12   13   14   15   16   17   18   19   20   >