[jira] [Closed] (SPARK-20337) Support upgrade a jar dependency and don't restart SparkContext

[ https://issues.apache.org/jira/browse/SPARK-20337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang closed SPARK-20337. --- Resolution: Won't Fix > Support upgrade a jar dependency and don't restart SparkContext >

[jira] [Assigned] (SPARK-20343) SBT master build for Hadoop 2.6 in Jenkins fails due to Avro version resolution

[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20343: Assignee: (was: Apache Spark) > SBT master build for Hadoop 2.6 in Jenkins fails due

[jira] [Assigned] (SPARK-20343) SBT master build for Hadoop 2.6 in Jenkins fails due to Avro version resolution

[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20343: Assignee: Apache Spark > SBT master build for Hadoop 2.6 in Jenkins fails due to Avro

[jira] [Commented] (SPARK-20343) SBT master build for Hadoop 2.6 in Jenkins fails due to Avro version resolution

[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969738#comment-15969738 ] Apache Spark commented on SPARK-20343: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-20343) SBT master build for Hadoop 2.6 in Jenkins due to Avro version resolution

[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-20343: - Summary: SBT master build for Hadoop 2.6 in Jenkins due to Avro version resolution (was:

[jira] [Updated] (SPARK-20343) SBT master build for Hadoop 2.6 in Jenkins fails due to Avro version resolution

[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-20343: - Summary: SBT master build for Hadoop 2.6 in Jenkins fails due to Avro version resolution (was:

[jira] [Updated] (SPARK-20343) Resolve SBT master build for Hadoop 2.6 in Jenkins due to Avro version resolution

[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-20343: - Summary: Resolve SBT master build for Hadoop 2.6 in Jenkins due to Avro version resolution

[jira] [Commented] (SPARK-20343) Resolve SBT master build for Hadoop 2.6 due to Avro version resolution

[ https://issues.apache.org/jira/browse/SPARK-20343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969709#comment-15969709 ] Hyukjin Kwon commented on SPARK-20343: -- I don't know the {{Priority}} in this case. Please correct

[jira] [Created] (SPARK-20343) Resolve SBT master build for Hadoop 2.6 due to Avro version resolution

Hyukjin Kwon created SPARK-20343: Summary: Resolve SBT master build for Hadoop 2.6 due to Avro version resolution Key: SPARK-20343 URL: https://issues.apache.org/jira/browse/SPARK-20343 Project:

[jira] [Commented] (SPARK-20340) Size estimate very wrong in ExternalAppendOnlyMap from CoGroupedRDD, cause OOM

[ https://issues.apache.org/jira/browse/SPARK-20340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969680#comment-15969680 ] Shixiong Zhu commented on SPARK-20340: -- I think it's just a trade off between accuracy and

[jira] [Updated] (SPARK-20341) Support BigIngeger values > 19 precision

[ https://issues.apache.org/jira/browse/SPARK-20341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20341: - Component/s: (was: Spark Core) SQL > Support BigIngeger values > 19

[jira] [Commented] (SPARK-20199) GradientBoostedTreesModel doesn't have Column Sampling Rate Paramenter

[ https://issues.apache.org/jira/browse/SPARK-20199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969622#comment-15969622 ] Yan Facai (颜发才) commented on SPARK-20199: - ping [~jkbreuer] [~sethah] [~mengxr]. Which one is

[jira] [Created] (SPARK-20342) DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators

Marcelo Vanzin created SPARK-20342: -- Summary: DAGScheduler sends SparkListenerTaskEnd before updating task's accumulators Key: SPARK-20342 URL: https://issues.apache.org/jira/browse/SPARK-20342

[jira] [Resolved] (SPARK-16899) Structured Streaming Checkpointing Example invalid

[ https://issues.apache.org/jira/browse/SPARK-16899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-16899. -- Resolution: Not A Problem This has been fixed. I believe you are using an old version

[jira] [Updated] (SPARK-16899) Structured Streaming Checkpointing Example invalid

[ https://issues.apache.org/jira/browse/SPARK-16899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-16899: - Component/s: Structured Streaming > Structured Streaming Checkpointing Example invalid >

[jira] [Commented] (SPARK-20329) Resolution error when HAVING clause uses GROUP BY expression that involves implicit type coercion

[ https://issues.apache.org/jira/browse/SPARK-20329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969483#comment-15969483 ] Apache Spark commented on SPARK-20329: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20329) Resolution error when HAVING clause uses GROUP BY expression that involves implicit type coercion

[ https://issues.apache.org/jira/browse/SPARK-20329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20329: Assignee: Apache Spark > Resolution error when HAVING clause uses GROUP BY expression

[jira] [Assigned] (SPARK-20329) Resolution error when HAVING clause uses GROUP BY expression that involves implicit type coercion

[ https://issues.apache.org/jira/browse/SPARK-20329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20329: Assignee: (was: Apache Spark) > Resolution error when HAVING clause uses GROUP BY

[jira] [Created] (SPARK-20341) Support BigIngeger values > 19 precision

Paul Zaczkieiwcz created SPARK-20341: Summary: Support BigIngeger values > 19 precision Key: SPARK-20341 URL: https://issues.apache.org/jira/browse/SPARK-20341 Project: Spark Issue Type:

[jira] [Created] (SPARK-20340) Size estimate very wrong in ExternalAppendOnlyMap from CoGroupedRDD, cause OOM

Thomas Graves created SPARK-20340: - Summary: Size estimate very wrong in ExternalAppendOnlyMap from CoGroupedRDD, cause OOM Key: SPARK-20340 URL: https://issues.apache.org/jira/browse/SPARK-20340

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969422#comment-15969422 ] Ritesh Tijoriwala commented on SPARK-650: - [~Skamandros] - I would also like to know about hooking

[jira] [Updated] (SPARK-20339) Issue in regex_replace in Apache Spark Java

[ https://issues.apache.org/jira/browse/SPARK-20339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nischay updated SPARK-20339: Description: We are currently facing couple of issues 1.

[jira] [Created] (SPARK-20339) Issue in regex_replace in Apache Spark Java

Nischay created SPARK-20339: --- Summary: Issue in regex_replace in Apache Spark Java Key: SPARK-20339 URL: https://issues.apache.org/jira/browse/SPARK-20339 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-20178) Improve Scheduler fetch failures

[ https://issues.apache.org/jira/browse/SPARK-20178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969388#comment-15969388 ] Thomas Graves commented on SPARK-20178: --- One thing I ran into today which is somewhat related to

[jira] [Commented] (SPARK-18406) Race between end-of-task and completion iterator read lock release

[ https://issues.apache.org/jira/browse/SPARK-18406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969384#comment-15969384 ] Yongqin Xiao commented on SPARK-18406: -- The same JIRA is found under

[jira] [Commented] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969379#comment-15969379 ] Michael Gummelt commented on SPARK-20328: - Ah, yes, of course. Thanks. > HadoopRDDs create a

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969341#comment-15969341 ] Michael Gummelt commented on SPARK-16742: - [~jerryshao] No, but you can look at our solution

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969341#comment-15969341 ] Michael Gummelt edited comment on SPARK-16742 at 4/14/17 6:01 PM: --

[jira] [Assigned] (SPARK-17608) Long type has incorrect serialization/deserialization

[ https://issues.apache.org/jira/browse/SPARK-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17608: Assignee: (was: Apache Spark) > Long type has incorrect serialization/deserialization

[jira] [Assigned] (SPARK-17608) Long type has incorrect serialization/deserialization

[ https://issues.apache.org/jira/browse/SPARK-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17608: Assignee: Apache Spark > Long type has incorrect serialization/deserialization >

[jira] [Commented] (SPARK-17608) Long type has incorrect serialization/deserialization

[ https://issues.apache.org/jira/browse/SPARK-17608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969328#comment-15969328 ] Apache Spark commented on SPARK-17608: -- User 'wangmiao1981' has created a pull request for this

[jira] [Updated] (SPARK-20243) DebugFilesystem.assertNoOpenStreams thread race

[ https://issues.apache.org/jira/browse/SPARK-20243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-20243: -- Fix Version/s: 2.1.1 > DebugFilesystem.assertNoOpenStreams thread race >

[jira] [Commented] (SPARK-19716) Dataset should allow by-name resolution for struct type elements in array

[ https://issues.apache.org/jira/browse/SPARK-19716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969051#comment-15969051 ] Apache Spark commented on SPARK-19716: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-20323) Calling stop in a transform stage causes the app to hang

[ https://issues.apache.org/jira/browse/SPARK-20323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969042#comment-15969042 ] Andrei Taleanu commented on SPARK-20323: Ok, thank you. Unfortunately I can't even get at-least

[jira] [Commented] (SPARK-20323) Calling stop in a transform stage causes the app to hang

[ https://issues.apache.org/jira/browse/SPARK-20323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969030#comment-15969030 ] Sean Owen commented on SPARK-20323: --- It's normal to continue processing if one batch fails. Achieving

[jira] [Commented] (SPARK-20323) Calling stop in a transform stage causes the app to hang

[ https://issues.apache.org/jira/browse/SPARK-20323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968969#comment-15968969 ] Andrei Taleanu commented on SPARK-20323: [~srowen] I see. Let me describe you better the problem.

[jira] [Comment Edited] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968945#comment-15968945 ] Maciej Bryński edited comment on SPARK-12717 at 4/14/17 12:12 PM: -- Same

[jira] [Comment Edited] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968945#comment-15968945 ] Maciej Bryński edited comment on SPARK-12717 at 4/14/17 12:10 PM: -- Same

[jira] [Commented] (SPARK-20323) Calling stop in a transform stage causes the app to hang

[ https://issues.apache.org/jira/browse/SPARK-20323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968943#comment-15968943 ] Sean Owen commented on SPARK-20323: --- Not being able to use a context in a remote operation? I think

[jira] [Commented] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968945#comment-15968945 ] Maciej Bryński commented on SPARK-12717: Same here. > pyspark broadcast fails when using

[jira] [Commented] (SPARK-20323) Calling stop in a transform stage causes the app to hang

[ https://issues.apache.org/jira/browse/SPARK-20323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968940#comment-15968940 ] Andrei Taleanu commented on SPARK-20323: [~srowen] could you please give me a link that documents

[jira] [Commented] (SPARK-10109) NPE when saving Parquet To HDFS

[ https://issues.apache.org/jira/browse/SPARK-10109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968934#comment-15968934 ] Steve Loughran commented on SPARK-10109: SPARK-20038 should stop the failure being so dramatic >

[jira] [Assigned] (SPARK-20318) Use Catalyst type for min/max in ColumnStat for ease of estimation

[ https://issues.apache.org/jira/browse/SPARK-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20318: --- Assignee: Zhenhua Wang > Use Catalyst type for min/max in ColumnStat for ease of estimation

[jira] [Resolved] (SPARK-20318) Use Catalyst type for min/max in ColumnStat for ease of estimation

[ https://issues.apache.org/jira/browse/SPARK-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20318. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17630

[jira] [Commented] (SPARK-20338) Spaces in spark.eventLog.dir are not correctly handled

[ https://issues.apache.org/jira/browse/SPARK-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968882#comment-15968882 ] Apache Spark commented on SPARK-20338: -- User 'zuotingbing' has created a pull request for this

[jira] [Assigned] (SPARK-20338) Spaces in spark.eventLog.dir are not correctly handled

[ https://issues.apache.org/jira/browse/SPARK-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20338: Assignee: (was: Apache Spark) > Spaces in spark.eventLog.dir are not correctly

[jira] [Assigned] (SPARK-20338) Spaces in spark.eventLog.dir are not correctly handled

[ https://issues.apache.org/jira/browse/SPARK-20338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20338: Assignee: Apache Spark > Spaces in spark.eventLog.dir are not correctly handled >

[jira] [Created] (SPARK-20338) Spaces in spark.eventLog.dir are not correctly handled

zuotingbing created SPARK-20338: --- Summary: Spaces in spark.eventLog.dir are not correctly handled Key: SPARK-20338 URL: https://issues.apache.org/jira/browse/SPARK-20338 Project: Spark Issue

[jira] [Assigned] (SPARK-20337) Support upgrade a jar dependency and don't restart SparkContext

[ https://issues.apache.org/jira/browse/SPARK-20337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20337: Assignee: (was: Apache Spark) > Support upgrade a jar dependency and don't restart

[jira] [Assigned] (SPARK-20337) Support upgrade a jar dependency and don't restart SparkContext

[ https://issues.apache.org/jira/browse/SPARK-20337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20337: Assignee: Apache Spark > Support upgrade a jar dependency and don't restart SparkContext

[jira] [Commented] (SPARK-20337) Support upgrade a jar dependency and don't restart SparkContext

[ https://issues.apache.org/jira/browse/SPARK-20337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968868#comment-15968868 ] Apache Spark commented on SPARK-20337: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-20337) Support upgrade a jar dependency and don't restart SparkContext

[ https://issues.apache.org/jira/browse/SPARK-20337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968843#comment-15968843 ] Sean Owen commented on SPARK-20337: --- I don't think this is a reasonable use case to support. Spark jobs

[jira] [Created] (SPARK-20337) Support upgrade a jar dependency and don't restart SparkContext

Yuming Wang created SPARK-20337: --- Summary: Support upgrade a jar dependency and don't restart SparkContext Key: SPARK-20337 URL: https://issues.apache.org/jira/browse/SPARK-20337 Project: Spark

[jira] [Resolved] (SPARK-20321) Spark UI cannot be shutdown in spark streaming app

[ https://issues.apache.org/jira/browse/SPARK-20321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20321. --- Resolution: Not A Problem > Spark UI cannot be shutdown in spark streaming app >

[jira] [Resolved] (SPARK-20323) Calling stop in a transform stage causes the app to hang

[ https://issues.apache.org/jira/browse/SPARK-20323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20323. --- Resolution: Not A Problem > Calling stop in a transform stage causes the app to hang >

[jira] [Commented] (SPARK-20336) spark.read.csv() with wholeFile=True option fails to read non ASCII unicode characters

[ https://issues.apache.org/jira/browse/SPARK-20336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968753#comment-15968753 ] Hyukjin Kwon commented on SPARK-20336: -- Oh, probably, I would also appreciate if you are able to

[jira] [Commented] (SPARK-20336) spark.read.csv() with wholeFile=True option fails to read non ASCII unicode characters

[ https://issues.apache.org/jira/browse/SPARK-20336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968749#comment-15968749 ] Hyukjin Kwon commented on SPARK-20336: -- I just tried to reproduce it as below: Scala {code} scala>

[jira] [Issue Comment Deleted] (SPARK-20215) ReuseExchange is boken in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-20215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-20215: --- Comment: was deleted (was: Seems to be fixed in SPARK-20229) > ReuseExchange is boken in SparkSQL >

[jira] [Commented] (SPARK-20215) ReuseExchange is boken in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-20215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968734#comment-15968734 ] Zhan Zhang commented on SPARK-20215: Seems to be fixed in SPARK-20229 > ReuseExchange is boken in

[jira] [Updated] (SPARK-20336) spark.read.csv() with wholeFile=True option fails to read non ASCII unicode characters

[ https://issues.apache.org/jira/browse/SPARK-20336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HanCheol Cho updated SPARK-20336: - Description: I used spark.read.csv() method with wholeFile=True option to load data that has

[jira] [Created] (SPARK-20336) spark.read.csv() with wholeFile=True option fails to read non ASCII unicode characters

HanCheol Cho created SPARK-20336: Summary: spark.read.csv() with wholeFile=True option fails to read non ASCII unicode characters Key: SPARK-20336 URL: https://issues.apache.org/jira/browse/SPARK-20336