[jira] [Commented] (SPARK-11714) Make Spark on Mesos honor port restrictions

[ https://issues.apache.org/jira/browse/SPARK-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136747#comment-15136747 ] Stavros Kontopoulos commented on SPARK-11714: - [~andrewor14] Would it be mean

[jira] [Commented] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136757#comment-15136757 ] Herman Schistad commented on SPARK-13198: - Hi [~srowen], thanks for your reply. I

[jira] [Updated] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman Schistad updated SPARK-13198: Attachment: Screen Shot 2016-02-08 at 09.30.59.png Screen Shot 2016-02-08 at

[jira] [Comment Edited] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136757#comment-15136757 ] Herman Schistad edited comment on SPARK-13198 at 2/8/16 9:55 AM: --

[jira] [Commented] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136773#comment-15136773 ] Sean Owen commented on SPARK-13198: --- I don't think stop() is relevant here. There's not

[jira] [Updated] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman Schistad updated SPARK-13198: Attachment: Screen Shot 2016-02-08 at 10.03.04.png > sc.stop() does not clean up on driver,

[jira] [Commented] (SPARK-13198) sc.stop() does not clean up on driver, causes Java heap OOM.

[ https://issues.apache.org/jira/browse/SPARK-13198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136774#comment-15136774 ] Herman Schistad commented on SPARK-13198: - Digging more into the dumped heap and

[jira] [Assigned] (SPARK-13177) Update ActorWordCount example to not directly use low level linked list as it is deprecated.

[ https://issues.apache.org/jira/browse/SPARK-13177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13177: Assignee: (was: Apache Spark) > Update ActorWordCount example to not directly use low

[jira] [Commented] (SPARK-13177) Update ActorWordCount example to not directly use low level linked list as it is deprecated.

[ https://issues.apache.org/jira/browse/SPARK-13177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136791#comment-15136791 ] Apache Spark commented on SPARK-13177: -- User 'agsachin' has created a pull request f

[jira] [Assigned] (SPARK-13177) Update ActorWordCount example to not directly use low level linked list as it is deprecated.

[ https://issues.apache.org/jira/browse/SPARK-13177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13177: Assignee: Apache Spark > Update ActorWordCount example to not directly use low level linke

[jira] [Created] (SPARK-13231) Rename Accumulable.countFailedValues to Accumulable.includeValuesOfFailedTasks and make it a user facing API.

Prashant Sharma created SPARK-13231: --- Summary: Rename Accumulable.countFailedValues to Accumulable.includeValuesOfFailedTasks and make it a user facing API. Key: SPARK-13231 URL: https://issues.apache.org/jira/b

[jira] [Commented] (SPARK-13156) JDBC using multiple partitions creates additional tasks but only executes on one

[ https://issues.apache.org/jira/browse/SPARK-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136834#comment-15136834 ] Charles Drotar commented on SPARK-13156: Thanks Sean. The driver inhibiting the c

[jira] [Closed] (SPARK-13156) JDBC using multiple partitions creates additional tasks but only executes on one

[ https://issues.apache.org/jira/browse/SPARK-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Drotar closed SPARK-13156. -- Resolution: Not A Problem The driver class was inhibiting concurrent connections. This was unre

[jira] [Comment Edited] (SPARK-13156) JDBC using multiple partitions creates additional tasks but only executes on one

[ https://issues.apache.org/jira/browse/SPARK-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136835#comment-15136835 ] Charles Drotar edited comment on SPARK-13156 at 2/8/16 11:28 AM: --

[jira] [Issue Comment Deleted] (SPARK-13156) JDBC using multiple partitions creates additional tasks but only executes on one

[ https://issues.apache.org/jira/browse/SPARK-13156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Drotar updated SPARK-13156: --- Comment: was deleted (was: Thanks Sean. The driver inhibiting the concurrent connections was

[jira] [Assigned] (SPARK-7848) Update SparkStreaming docs to incorporate FAQ and/or bullets w/ "knobs" information.

[ https://issues.apache.org/jira/browse/SPARK-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7848: --- Assignee: Apache Spark > Update SparkStreaming docs to incorporate FAQ and/or bullets w/ "kno

[jira] [Assigned] (SPARK-7848) Update SparkStreaming docs to incorporate FAQ and/or bullets w/ "knobs" information.

[ https://issues.apache.org/jira/browse/SPARK-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7848: --- Assignee: (was: Apache Spark) > Update SparkStreaming docs to incorporate FAQ and/or bull

[jira] [Commented] (SPARK-7848) Update SparkStreaming docs to incorporate FAQ and/or bullets w/ "knobs" information.

[ https://issues.apache.org/jira/browse/SPARK-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136849#comment-15136849 ] Apache Spark commented on SPARK-7848: - User 'nirmannarang' has created a pull request

[jira] [Assigned] (SPARK-13231) Rename Accumulable.countFailedValues to Accumulable.includeValuesOfFailedTasks and make it a user facing API.

[ https://issues.apache.org/jira/browse/SPARK-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13231: Assignee: (was: Apache Spark) > Rename Accumulable.countFailedValues to > Accumulable

[jira] [Assigned] (SPARK-13231) Rename Accumulable.countFailedValues to Accumulable.includeValuesOfFailedTasks and make it a user facing API.

[ https://issues.apache.org/jira/browse/SPARK-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13231: Assignee: Apache Spark > Rename Accumulable.countFailedValues to > Accumulable.includeVal

[jira] [Commented] (SPARK-13231) Rename Accumulable.countFailedValues to Accumulable.includeValuesOfFailedTasks and make it a user facing API.

[ https://issues.apache.org/jira/browse/SPARK-13231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136944#comment-15136944 ] Apache Spark commented on SPARK-13231: -- User 'ScrapCodes' has created a pull request

[jira] [Updated] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-12316: -- Assignee: SaintBacchus > Stack overflow with endless call of `Delegation token thread` when >

[jira] [Assigned] (SPARK-13013) Replace example code in mllib-clustering.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13013: Assignee: Apache Spark > Replace example code in mllib-clustering.md using include_example

[jira] [Assigned] (SPARK-13013) Replace example code in mllib-clustering.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13013: Assignee: (was: Apache Spark) > Replace example code in mllib-clustering.md using incl

[jira] [Commented] (SPARK-13013) Replace example code in mllib-clustering.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137004#comment-15137004 ] Apache Spark commented on SPARK-13013: -- User 'keypointt' has created a pull request

[jira] [Commented] (SPARK-13014) Replace example code in mllib-collaborative-filtering.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137007#comment-15137007 ] Xin Ren commented on SPARK-13014: - I'm working on this one, thanks :) > Replace example

[jira] [Commented] (SPARK-12316) Stack overflow with endless call of `Delegation token thread` when application end.

[ https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137000#comment-15137000 ] Thomas Graves commented on SPARK-12316: --- you say "endless cycle call" do you mean t

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137029#comment-15137029 ] Rama Mullapudi commented on SPARK-12177: Does the update include kerberos support

[jira] [Created] (SPARK-13232) YARN executor node label expressions bug

Atkins created SPARK-13232: -- Summary: YARN executor node label expressions bug Key: SPARK-13232 URL: https://issues.apache.org/jira/browse/SPARK-13232 Project: Spark Issue Type: Bug Compon

[jira] [Updated] (SPARK-13232) YARN executor node label expressions

[ https://issues.apache.org/jira/browse/SPARK-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13232: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) Summary: YARN executor no

[jira] [Commented] (SPARK-13172) Stop using RichException.getStackTrace it is deprecated

[ https://issues.apache.org/jira/browse/SPARK-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137136#comment-15137136 ] sachin aggarwal commented on SPARK-13172: - instead of getStackTraceString should

[jira] [Created] (SPARK-13233) Python Dataset

Wenchen Fan created SPARK-13233: --- Summary: Python Dataset Key: SPARK-13233 URL: https://issues.apache.org/jira/browse/SPARK-13233 Project: Spark Issue Type: New Feature Components: SQ

[jira] [Updated] (SPARK-13233) Python Dataset

[ https://issues.apache.org/jira/browse/SPARK-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-13233: Attachment: DesignDocPythonDataset.pdf > Python Dataset > -- > > Key: S

[jira] [Updated] (SPARK-13233) Python Dataset

[ https://issues.apache.org/jira/browse/SPARK-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-13233: Description: add Python Dataset w.r.t. the scala version > Python Dataset > -- > >

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137168#comment-15137168 ] Sangeet Chourey commented on SPARK-10528: - RESOLVED : Downloaded the correct Win

[jira] [Comment Edited] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137168#comment-15137168 ] Sangeet Chourey edited comment on SPARK-10528 at 2/8/16 4:26 PM: --

[jira] [Assigned] (SPARK-13233) Python Dataset

[ https://issues.apache.org/jira/browse/SPARK-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13233: Assignee: Apache Spark > Python Dataset > -- > > Key: SPARK-13

[jira] [Commented] (SPARK-13233) Python Dataset

[ https://issues.apache.org/jira/browse/SPARK-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137180#comment-15137180 ] Apache Spark commented on SPARK-13233: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-13233) Python Dataset

[ https://issues.apache.org/jira/browse/SPARK-13233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13233: Assignee: (was: Apache Spark) > Python Dataset > -- > > Ke

[jira] [Commented] (SPARK-10066) Can't create HiveContext with spark-shell or spark-sql on snapshot

[ https://issues.apache.org/jira/browse/SPARK-10066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137201#comment-15137201 ] Sangeet Chourey commented on SPARK-10066: - RESOLVED : Downloaded the correct Winu

[jira] [Commented] (SPARK-13232) YARN executor node label expressions

[ https://issues.apache.org/jira/browse/SPARK-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137222#comment-15137222 ] Atkins commented on SPARK-13232: If spark config "spark.yarn.executor.nodeLabelExpression

[jira] [Updated] (SPARK-13104) Spark Metrics currently does not return executors hostname

[ https://issues.apache.org/jira/browse/SPARK-13104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik updated SPARK-13104: Description: We been using Spark Metrics and porting the data to InfluxDB using the Graphite sink that is a

[jira] [Comment Edited] (SPARK-13232) YARN executor node label expressions

[ https://issues.apache.org/jira/browse/SPARK-13232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137222#comment-15137222 ] Atkins edited comment on SPARK-13232 at 2/8/16 4:59 PM: I am tell

[jira] [Updated] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13219: Component/s: (was: Spark Core) SQL > Pushdown predicate propagation in SparkSQL with j

[jira] [Commented] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137282#comment-15137282 ] Xiao Li commented on SPARK-13219: - See this PR: https://github.com/apache/spark/pull/1049

[jira] [Commented] (SPARK-13016) Replace example code in mllib-dimensionality-reduction.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137310#comment-15137310 ] Devaraj K commented on SPARK-13016: --- I am working on this, I will provide PR for this.

[jira] [Commented] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137320#comment-15137320 ] Devaraj K commented on SPARK-13117: --- Thanks [~jjordan] for reporting. I would like to p

[jira] [Commented] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137330#comment-15137330 ] Abhinav Chawade commented on SPARK-13219: - Thanks Xiao. I will pull in the reques

[jira] [Commented] (SPARK-7889) Jobs progress of apps on complete page of HistoryServer shows uncompleted

[ https://issues.apache.org/jira/browse/SPARK-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137336#comment-15137336 ] Apache Spark commented on SPARK-7889: - User 'squito' has created a pull request for th

[jira] [Commented] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137333#comment-15137333 ] Xiao Li commented on SPARK-13219: - Welcome > Pushdown predicate propagation in SparkSQL

[jira] [Commented] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137353#comment-15137353 ] Jeremiah Jordan commented on SPARK-13117: - go for it. > WebUI should use the loc

[jira] [Resolved] (SPARK-12455) Add ExpressionDescription to window functions

[ https://issues.apache.org/jira/browse/SPARK-12455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-12455. --- Resolution: Resolved Fix Version/s: 2.0.0 > Add ExpressionDescription to windo

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137369#comment-15137369 ] Sean Owen commented on SPARK-6305: -- I've started working on this, and it's as awful a dep

[jira] [Resolved] (SPARK-12986) Fix pydoc warnings in mllib/regression.py

[ https://issues.apache.org/jira/browse/SPARK-12986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-12986. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11025 [https://g

[jira] [Created] (SPARK-13234) Remove duplicated SQL metrics

Davies Liu created SPARK-13234: -- Summary: Remove duplicated SQL metrics Key: SPARK-13234 URL: https://issues.apache.org/jira/browse/SPARK-13234 Project: Spark Issue Type: Improvement C

[jira] [Resolved] (SPARK-8964) Use Exchange in limit operations (per partition limit -> exchange to one partition -> per partition limit)

[ https://issues.apache.org/jira/browse/SPARK-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8964. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 7334 [https://github.com/

[jira] [Commented] (SPARK-13213) BroadcastNestedLoopJoin is very slow

[ https://issues.apache.org/jira/browse/SPARK-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137551#comment-15137551 ] Davies Liu commented on SPARK-13213: [~sowen] Thanks very much for update these, I tr

[jira] [Updated] (SPARK-12585) The numFields of UnsafeRow should not changed by pointTo()

[ https://issues.apache.org/jira/browse/SPARK-12585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12585: --- Component/s: SQL > The numFields of UnsafeRow should not changed by pointTo() > -

[jira] [Updated] (SPARK-12840) Support passing arbitrary objects (not just expressions) into code generated classes

[ https://issues.apache.org/jira/browse/SPARK-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12840: --- Component/s: SQL > Support passing arbitrary objects (not just expressions) into code generated > cl

[jira] [Updated] (SPARK-13215) Remove fallback in codegen

[ https://issues.apache.org/jira/browse/SPARK-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13215: --- Component/s: SQL > Remove fallback in codegen > -- > > Key: S

[jira] [Comment Edited] (SPARK-13172) Stop using RichException.getStackTrace it is deprecated

[ https://issues.apache.org/jira/browse/SPARK-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137575#comment-15137575 ] Jakob Odersky edited comment on SPARK-13172 at 2/8/16 8:03 PM:

[jira] [Commented] (SPARK-13172) Stop using RichException.getStackTrace it is deprecated

[ https://issues.apache.org/jira/browse/SPARK-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137575#comment-15137575 ] Jakob Odersky commented on SPARK-13172: --- I would suggest taking similar approach to

[jira] [Updated] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-13101: - Fix Version/s: 1.6.1 > Dataset complex types mapping to DataFrame (element nullability)

[jira] [Resolved] (SPARK-13101) Dataset complex types mapping to DataFrame (element nullability) mismatch

[ https://issues.apache.org/jira/browse/SPARK-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-13101. -- Resolution: Fixed Fix Version/s: (was: 1.6.1) 2.0.0 Issue

[jira] [Resolved] (SPARK-13210) NPE in Sort

[ https://issues.apache.org/jira/browse/SPARK-13210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-13210. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11095 [https://github.

[jira] [Commented] (SPARK-13210) NPE in Sort

[ https://issues.apache.org/jira/browse/SPARK-13210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137598#comment-15137598 ] Josh Rosen commented on SPARK-13210: I'm also going to cherry-pick this for 1.6.1. >

[jira] [Commented] (SPARK-10780) Set initialModel in KMeans in Pipelines API

[ https://issues.apache.org/jira/browse/SPARK-10780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137607#comment-15137607 ] Apache Spark commented on SPARK-10780: -- User 'yinxusen' has created a pull request f

[jira] [Created] (SPARK-13235) Remove extra Distinct in Union Distinct

Xiao Li created SPARK-13235: --- Summary: Remove extra Distinct in Union Distinct Key: SPARK-13235 URL: https://issues.apache.org/jira/browse/SPARK-13235 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-10561) Provide tooling for auto-generating Spark SQL reference manual

[ https://issues.apache.org/jira/browse/SPARK-10561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-10561: --- Description: Here is the discussion thread: http://search-hadoop.com/m/q3RTtcD20F1o62xE Richard Hillegas mad

[jira] [Commented] (SPARK-13180) Protect against SessionState being null when accessing HiveClientImpl#conf

[ https://issues.apache.org/jira/browse/SPARK-13180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137674#comment-15137674 ] Ted Yu commented on SPARK-13180: I wonder if we should provide better error message when

[jira] [Assigned] (SPARK-13235) Remove extra Distinct in Union Distinct

[ https://issues.apache.org/jira/browse/SPARK-13235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13235: Assignee: (was: Apache Spark) > Remove extra Distinct in Union Distinct >

[jira] [Commented] (SPARK-13235) Remove extra Distinct in Union Distinct

[ https://issues.apache.org/jira/browse/SPARK-13235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137682#comment-15137682 ] Apache Spark commented on SPARK-13235: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-13235) Remove extra Distinct in Union Distinct

[ https://issues.apache.org/jira/browse/SPARK-13235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13235: Assignee: Apache Spark > Remove extra Distinct in Union Distinct > ---

[jira] [Updated] (SPARK-13235) Remove extra Distinct in Union

[ https://issues.apache.org/jira/browse/SPARK-13235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13235: Summary: Remove extra Distinct in Union (was: Remove extra Distinct in Union Distinct) > Remove extra Dis

[jira] [Updated] (SPARK-13235) Remove an extra Distinct in Union

[ https://issues.apache.org/jira/browse/SPARK-13235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13235: Summary: Remove an extra Distinct in Union (was: Remove extra Distinct in Union) > Remove an extra Distin

[jira] [Updated] (SPARK-13235) Remove an Extra Distinct in Union

[ https://issues.apache.org/jira/browse/SPARK-13235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13235: Summary: Remove an Extra Distinct in Union (was: Remove an extra Distinct in Union) > Remove an Extra Dis

[jira] [Updated] (SPARK-13235) Remove an Extra Distinct in Union

[ https://issues.apache.org/jira/browse/SPARK-13235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-13235: Description: Union Distinct has two Distinct that generate two Aggregation in the plan. {code} sql("select

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137702#comment-15137702 ] Shixiong Zhu commented on SPARK-13171: -- Looks something wrong happened between: [in

[jira] [Commented] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137701#comment-15137701 ] Abhinav Chawade commented on SPARK-13219: - I created a build of Spark 1.4.1 which

[jira] [Commented] (SPARK-12505) Pushdown a Limit on top of an Outer-Join

[ https://issues.apache.org/jira/browse/SPARK-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137710#comment-15137710 ] Apache Spark commented on SPARK-12505: -- User 'JoshRosen' has created a pull request

[jira] [Commented] (SPARK-12503) Pushdown a Limit on top of a Union

[ https://issues.apache.org/jira/browse/SPARK-12503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137709#comment-15137709 ] Apache Spark commented on SPARK-12503: -- User 'JoshRosen' has created a pull request

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137714#comment-15137714 ] holdenk commented on SPARK-13171: - I've been seeing that for intermittently for awhile in

[jira] [Commented] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137721#comment-15137721 ] Xiao Li commented on SPARK-13219: - Let me try your SQL query in Spark 1.6.1. > Pushdown

[jira] [Created] (SPARK-13236) SQL generation support for union

Xiao Li created SPARK-13236: --- Summary: SQL generation support for union Key: SPARK-13236 URL: https://issues.apache.org/jira/browse/SPARK-13236 Project: Spark Issue Type: Sub-task Compone

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137771#comment-15137771 ] Mark Grover commented on SPARK-12177: - Hi Rama, This particular PR adds support for t

[jira] [Commented] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137770#comment-15137770 ] Abhinav Chawade commented on SPARK-13219: - Here is my branch on github if you'd l

[jira] [Commented] (SPARK-12194) Add Sink for reporting Spark Metrics to OpenTSDB

[ https://issues.apache.org/jira/browse/SPARK-12194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137774#comment-15137774 ] Matt Kapilevich commented on SPARK-12194: - I am also looking to capture Spark met

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137795#comment-15137795 ] Jakob Odersky commented on SPARK-13171: --- This is very strange, are you sure it has

[jira] [Commented] (SPARK-13216) Spark streaming application not honoring --num-executors in restarting of an application from a checkpoint

[ https://issues.apache.org/jira/browse/SPARK-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137797#comment-15137797 ] Hari Shreedharan commented on SPARK-13216: -- I disagree that checkpointing is onl

[jira] [Resolved] (SPARK-13095) improve performance of hash join with dimension table

[ https://issues.apache.org/jira/browse/SPARK-13095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13095. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11065 [https://github.

[jira] [Commented] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137858#comment-15137858 ] Apache Spark commented on SPARK-13027: -- User 'aramesh117' has created a pull request

[jira] [Assigned] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13027: Assignee: (was: Apache Spark) > Add API for updateStateByKey to provide batch time as

[jira] [Assigned] (SPARK-13027) Add API for updateStateByKey to provide batch time as input

[ https://issues.apache.org/jira/browse/SPARK-13027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13027: Assignee: Apache Spark > Add API for updateStateByKey to provide batch time as input > ---

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137924#comment-15137924 ] Amir Gur commented on SPARK-10528: -- Thanks, confirming it worked on win8.1 64 bit. > s

[jira] [Assigned] (SPARK-13014) Replace example code in mllib-collaborative-filtering.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13014: Assignee: (was: Apache Spark) > Replace example code in mllib-collaborative-filtering.

[jira] [Commented] (SPARK-13014) Replace example code in mllib-collaborative-filtering.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137982#comment-15137982 ] Apache Spark commented on SPARK-13014: -- User 'keypointt' has created a pull request

[jira] [Assigned] (SPARK-13014) Replace example code in mllib-collaborative-filtering.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13014: Assignee: Apache Spark > Replace example code in mllib-collaborative-filtering.md using in

[jira] [Commented] (SPARK-13236) SQL generation support for union

[ https://issues.apache.org/jira/browse/SPARK-13236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137988#comment-15137988 ] Xiao Li commented on SPARK-13236: - After the merge of Spark-13235, I will upload a PR for

[jira] [Commented] (SPARK-13018) Replace example code in mllib-pmml-model-export.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-13018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137996#comment-15137996 ] Xin Ren commented on SPARK-13018: - I'm working on this one, thanks :) > Replace example

[jira] [Commented] (SPARK-13219) Pushdown predicate propagation in SparkSQL with join

[ https://issues.apache.org/jira/browse/SPARK-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15138024#comment-15138024 ] Evan Chan commented on SPARK-13219: --- [~smilegator] does your PR take care of the case

  1   2   >