[jira] [Resolved] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22279. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21186

[jira] [Assigned] (SPARK-22279) Turn on spark.sql.hive.convertMetastoreOrc by default

[ https://issues.apache.org/jira/browse/SPARK-22279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22279: --- Assignee: Dongjoon Hyun > Turn on spark.sql.hive.convertMetastoreOrc by default >

[jira] [Resolved] (SPARK-24073) DataSourceV2: Rename DataReaderFactory back to ReadTask.

[ https://issues.apache.org/jira/browse/SPARK-24073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24073. - Resolution: Fixed Assignee: Ryan Blue > DataSourceV2: Rename DataReaderFactory back to ReadTask. >

[jira] [Commented] (SPARK-24206) Improve DataSource benchmark code for read and pushdown

[ https://issues.apache.org/jira/browse/SPARK-24206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469918#comment-16469918 ] Apache Spark commented on SPARK-24206: -- User 'maropu' has created a pull request for this issue:

[jira] [Updated] (SPARK-23390) Flaky Test Suite: FileBasedDataSourceSuite in Spark 2.3/hadoop 2.7

[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23390: -- Description: We're seeing multiple failures in {{FileBasedDataSourceSuite}} in

[jira] [Created] (SPARK-24239) Flaky test: KafkaContinuousSourceSuite.subscribing topic by name from earliest offsets

Dongjoon Hyun created SPARK-24239: - Summary: Flaky test: KafkaContinuousSourceSuite.subscribing topic by name from earliest offsets Key: SPARK-24239 URL: https://issues.apache.org/jira/browse/SPARK-24239

[jira] [Updated] (SPARK-11150) Dynamic partition pruning

[ https://issues.apache.org/jira/browse/SPARK-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-11150: -- Affects Version/s: 2.1.2 2.2.1 2.3.0 > Dynamic

[jira] [Resolved] (SPARK-23843) Deploy yarn meets incorrect LOCALIZED_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-23843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoutai.zt resolved SPARK-23843. Resolution: Invalid > Deploy yarn meets incorrect LOCALIZED_CONF_DIR >

[jira] [Closed] (SPARK-23843) Deploy yarn meets incorrect LOCALIZED_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-23843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoutai.zt closed SPARK-23843. -- Invalid bug > Deploy yarn meets incorrect LOCALIZED_CONF_DIR >

[jira] [Commented] (SPARK-23843) Deploy yarn meets incorrect LOCALIZED_CONF_DIR

[ https://issues.apache.org/jira/browse/SPARK-23843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469882#comment-16469882 ] zhoutai.zt commented on SPARK-23843: Thanks Shao. This is a bug in our own new Hadoop-compatible

[jira] [Commented] (SPARK-1849) sc.textFile does not support non UTF-8 encodings

[ https://issues.apache.org/jira/browse/SPARK-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469880#comment-16469880 ] Apache Spark commented on SPARK-1849: - User 'cqzlxl' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469858#comment-16469858 ] spark_user edited comment on SPARK-24217 at 5/10/18 3:11 AM: - Hi Joseph K

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469859#comment-16469859 ] spark_user edited comment on SPARK-24217 at 5/10/18 3:10 AM: - Behaviour

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469863#comment-16469863 ] Jose Torres commented on SPARK-24036: - The way I was envisioning it, there would be four kinds of

[jira] [Commented] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469859#comment-16469859 ] spark_user commented on SPARK-24217: Behaviour should be same for both spark.ml and spark.mllib

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469858#comment-16469858 ] spark_user edited comment on SPARK-24217 at 5/10/18 2:59 AM: - Hi Joseph K

[jira] [Commented] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469858#comment-16469858 ] spark_user commented on SPARK-24217: For the same input in spark.ml and spark.mllib, spark.mllib

[jira] [Resolved] (SPARK-23852) Parquet MR bug can lead to incorrect SQL results

[ https://issues.apache.org/jira/browse/SPARK-23852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23852. - Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.4.0 > Parquet MR bug can lead to

[jira] [Comment Edited] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469830#comment-16469830 ] Li Yuanjian edited comment on SPARK-24036 at 5/10/18 2:32 AM: -- Hi

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469830#comment-16469830 ] Li Yuanjian commented on SPARK-24036: - Hi [~joseph.torres] Thanks for cc me, looks great!  My doc

[jira] [Updated] (SPARK-24238) HadoopFsRelation can't append the same table with multi job at the same time.

[ https://issues.apache.org/jira/browse/SPARK-24238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangz updated SPARK-24238: -- Summary: HadoopFsRelation can't append the same table with multi job at the same time. (was: HadoopFsRelation

[jira] [Assigned] (SPARK-24238) HadoopFsRelation can't append the same table with multi job.

[ https://issues.apache.org/jira/browse/SPARK-24238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24238: Assignee: (was: Apache Spark) > HadoopFsRelation can't append the same table with

[jira] [Commented] (SPARK-24238) HadoopFsRelation can't append the same table with multi job.

[ https://issues.apache.org/jira/browse/SPARK-24238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469826#comment-16469826 ] Apache Spark commented on SPARK-24238: -- User 'zheh12' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24238) HadoopFsRelation can't append the same table with multi job.

[ https://issues.apache.org/jira/browse/SPARK-24238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24238: Assignee: Apache Spark > HadoopFsRelation can't append the same table with multi job. >

[jira] [Commented] (SPARK-24194) HadoopFsRelation cannot overwrite a path that is also being read from

[ https://issues.apache.org/jira/browse/SPARK-24194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469825#comment-16469825 ] Apache Spark commented on SPARK-24194: -- User 'zheh12' has created a pull request for this issue:

[jira] [Created] (SPARK-24238) HadoopFsRelation can't append the same table with multi job.

yangz created SPARK-24238: - Summary: HadoopFsRelation can't append the same table with multi job. Key: SPARK-24238 URL: https://issues.apache.org/jira/browse/SPARK-24238 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24036) Stateful operators in continuous processing

[ https://issues.apache.org/jira/browse/SPARK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469801#comment-16469801 ] Jose Torres commented on SPARK-24036: - ~[~XuanYuan] Since it seems we've reached broad consensus on

[jira] [Resolved] (SPARK-24041) add flag to remove whitelist of continuous processing operators

[ https://issues.apache.org/jira/browse/SPARK-24041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Torres resolved SPARK-24041. - Resolution: Not A Problem > add flag to remove whitelist of continuous processing operators >

[jira] [Commented] (SPARK-24041) add flag to remove whitelist of continuous processing operators

[ https://issues.apache.org/jira/browse/SPARK-24041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469800#comment-16469800 ] Jose Torres commented on SPARK-24041: - This isn't needed, we can just disable

[jira] [Created] (SPARK-24237) continuous shuffle dependency

Jose Torres created SPARK-24237: --- Summary: continuous shuffle dependency Key: SPARK-24237 URL: https://issues.apache.org/jira/browse/SPARK-24237 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-24236) continuous replacement for ShuffleExchangeExec

Jose Torres created SPARK-24236: --- Summary: continuous replacement for ShuffleExchangeExec Key: SPARK-24236 URL: https://issues.apache.org/jira/browse/SPARK-24236 Project: Spark Issue Type:

[jira] [Created] (SPARK-24235) create the top-of-task RDD sending rows to the remote buffer

Jose Torres created SPARK-24235: --- Summary: create the top-of-task RDD sending rows to the remote buffer Key: SPARK-24235 URL: https://issues.apache.org/jira/browse/SPARK-24235 Project: Spark

[jira] [Updated] (SPARK-24234) create the bottom-of-task RDD with row buffer

[ https://issues.apache.org/jira/browse/SPARK-24234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Torres updated SPARK-24234: Summary: create the bottom-of-task RDD with row buffer (was: Write RDD with row buffer) > create

[jira] [Created] (SPARK-24234) Write RDD with row buffer

Jose Torres created SPARK-24234: --- Summary: Write RDD with row buffer Key: SPARK-24234 URL: https://issues.apache.org/jira/browse/SPARK-24234 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-24233) union operation on read of dataframe does nor produce correct result

smohr003 created SPARK-24233: Summary: union operation on read of dataframe does nor produce correct result Key: SPARK-24233 URL: https://issues.apache.org/jira/browse/SPARK-24233 Project: Spark

[jira] [Created] (SPARK-24232) Allow referring to kubernetes secrets as env variable

Dharmesh Kakadia created SPARK-24232: Summary: Allow referring to kubernetes secrets as env variable Key: SPARK-24232 URL: https://issues.apache.org/jira/browse/SPARK-24232 Project: Spark

[jira] [Updated] (SPARK-24231) Python API: Provide evaluateEachIteration method or equivalent for spark.ml GBTs

[ https://issues.apache.org/jira/browse/SPARK-24231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-24231: --- Summary: Python API: Provide evaluateEachIteration method or equivalent for spark.ml GBTs (was:

[jira] [Created] (SPARK-24231) Provide evaluateEachIteration method or equivalent for spark.ml GBTs: Python API

Weichen Xu created SPARK-24231: -- Summary: Provide evaluateEachIteration method or equivalent for spark.ml GBTs: Python API Key: SPARK-24231 URL: https://issues.apache.org/jira/browse/SPARK-24231

[jira] [Commented] (SPARK-24230) With Parquet 1.10 upgrade has errors in the vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-24230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469658#comment-16469658 ] Ian O Connell commented on SPARK-24230: --- Great, thanks!      (I think its probably worth just

[jira] [Comment Edited] (SPARK-24230) With Parquet 1.10 upgrade has errors in the vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-24230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469658#comment-16469658 ] Ian O Connell edited comment on SPARK-24230 at 5/9/18 11:18 PM: Great,

[jira] [Commented] (SPARK-24230) With Parquet 1.10 upgrade has errors in the vectorized reader

[ https://issues.apache.org/jira/browse/SPARK-24230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469656#comment-16469656 ] Ryan Blue commented on SPARK-24230: --- Looks like I have a fix for this that I missed when submitting the

[jira] [Created] (SPARK-24230) With Parquet 1.10 upgrade has errors in the vectorized reader

Ian O Connell created SPARK-24230: - Summary: With Parquet 1.10 upgrade has errors in the vectorized reader Key: SPARK-24230 URL: https://issues.apache.org/jira/browse/SPARK-24230 Project: Spark

[jira] [Resolved] (SPARK-24141) Fix bug in CoarseGrainedSchedulerBackend.killExecutors

[ https://issues.apache.org/jira/browse/SPARK-24141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24141. Resolution: Fixed Fix Version/s: (was: 2.3.0) 2.4.0 Issue

[jira] [Assigned] (SPARK-24141) Fix bug in CoarseGrainedSchedulerBackend.killExecutors

[ https://issues.apache.org/jira/browse/SPARK-24141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24141: -- Assignee: wuyi > Fix bug in CoarseGrainedSchedulerBackend.killExecutors >

[jira] [Created] (SPARK-24229) Upgrade to the latest Apache Thrift 0.10.0 release

Ray Donnelly created SPARK-24229: Summary: Upgrade to the latest Apache Thrift 0.10.0 release Key: SPARK-24229 URL: https://issues.apache.org/jira/browse/SPARK-24229 Project: Spark Issue

[jira] [Created] (SPARK-24228) Fix the lint error

Xiao Li created SPARK-24228: --- Summary: Fix the lint error Key: SPARK-24228 URL: https://issues.apache.org/jira/browse/SPARK-24228 Project: Spark Issue Type: Bug Components: Build

[jira] [Commented] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469562#comment-16469562 ] Joseph K. Bradley commented on SPARK-24217: --- But the reason that the IDs are missing from the

[jira] [Commented] (SPARK-11150) Dynamic partition pruning

[ https://issues.apache.org/jira/browse/SPARK-11150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469551#comment-16469551 ] tim geary commented on SPARK-11150: --- Ice/nyse is asking on status of this, it has been open for a

[jira] [Assigned] (SPARK-24176) The hdfs file path with wildcard can not be identified when loading data

[ https://issues.apache.org/jira/browse/SPARK-24176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24176: Assignee: (was: Apache Spark) > The hdfs file path with wildcard can not be

[jira] [Assigned] (SPARK-24176) The hdfs file path with wildcard can not be identified when loading data

[ https://issues.apache.org/jira/browse/SPARK-24176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24176: Assignee: Apache Spark > The hdfs file path with wildcard can not be identified when

[jira] [Commented] (SPARK-24176) The hdfs file path with wildcard can not be identified when loading data

[ https://issues.apache.org/jira/browse/SPARK-24176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469493#comment-16469493 ] Apache Spark commented on SPARK-24176: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Commented] (SPARK-23681) Switch OrcFileFormat to newer hadoop.mapreduce output classes

[ https://issues.apache.org/jira/browse/SPARK-23681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469476#comment-16469476 ] Marcelo Vanzin commented on SPARK-23681: [~ste...@apache.org] are you planning on sending a PR

[jira] [Updated] (SPARK-1866) Closure cleaner does not null shadowed fields when outer scope is referenced

[ https://issues.apache.org/jira/browse/SPARK-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-1866: -- Priority: Major (was: Critical) > Closure cleaner does not null shadowed fields when outer

[jira] [Assigned] (SPARK-1866) Closure cleaner does not null shadowed fields when outer scope is referenced

[ https://issues.apache.org/jira/browse/SPARK-1866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-1866: - Assignee: (was: Kan Zhang) > Closure cleaner does not null shadowed fields when

[jira] [Resolved] (SPARK-3492) Clean up Yarn integration code

[ https://issues.apache.org/jira/browse/SPARK-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-3492. --- Resolution: Incomplete I'm going to close this since I don't see much value in keeping this

[jira] [Resolved] (SPARK-8294) Break down large methods in YARN code

[ https://issues.apache.org/jira/browse/SPARK-8294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-8294. --- Resolution: Won't Fix I don't think it's helpful to keep this as a task. We generally do

[jira] [Resolved] (SPARK-8293) Add high-level java docs to important YARN classes

[ https://issues.apache.org/jira/browse/SPARK-8293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-8293. --- Resolution: Won't Fix Not sure it's worth keeping this open unless someone is actually

[jira] [Updated] (SPARK-3492) Clean up Yarn integration code

[ https://issues.apache.org/jira/browse/SPARK-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-3492: -- Priority: Major (was: Critical) > Clean up Yarn integration code >

[jira] [Updated] (SPARK-4476) Use MapType for dict in json which has unique keys in each row.

[ https://issues.apache.org/jira/browse/SPARK-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-4476: -- Priority: Major (was: Critical) > Use MapType for dict in json which has unique keys in each

[jira] [Resolved] (SPARK-5098) Number of running tasks become negative after tasks lost

[ https://issues.apache.org/jira/browse/SPARK-5098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-5098. --- Resolution: Cannot Reproduce Pretty sure this has been fixed in one way or another since 1.2.

[jira] [Resolved] (SPARK-5517) Add input types for Java UDFs

[ https://issues.apache.org/jira/browse/SPARK-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-5517. - Resolution: Unresolved > Add input types for Java UDFs > - >

[jira] [Commented] (SPARK-5517) Add input types for Java UDFs

[ https://issues.apache.org/jira/browse/SPARK-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469456#comment-16469456 ] Marcelo Vanzin commented on SPARK-5517: --- [~marmbrus] can we close this? Looks very out of date. >

[jira] [Resolved] (SPARK-6031) Refactor --packages to work inside the DriverBootstrapper so that the jars can be added to the driver classpath

[ https://issues.apache.org/jira/browse/SPARK-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6031. --- Resolution: Duplicate As far as I understand this looks like a dupe of SPARK-12559. >

[jira] [Resolved] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

[ https://issues.apache.org/jira/browse/SPARK-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6484. --- Resolution: Fixed Marking this as fixed given Josh's comments. > Ganglia metrics xml

[jira] [Resolved] (SPARK-6069) Deserialization Error ClassNotFoundException with Kryo, Guava 14

[ https://issues.apache.org/jira/browse/SPARK-6069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6069. --- Resolution: Unresolved I believe this should have been fixed by SPARK-5470. Please try out

[jira] [Updated] (SPARK-6442) MLlib Local Linear Algebra Package

[ https://issues.apache.org/jira/browse/SPARK-6442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-6442: -- Priority: Major (was: Critical) > MLlib Local Linear Algebra Package >

[jira] [Updated] (SPARK-24227) Not able to submit spark job to kubernetes on 2.3

[ https://issues.apache.org/jira/browse/SPARK-24227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe Cavalcanti updated SPARK-24227: -- Labels: kubernetes spark (was: ) > Not able to submit spark job to kubernetes on 2.3

[jira] [Resolved] (SPARK-6270) Standalone Master hangs when streaming job completes and event logging is enabled

[ https://issues.apache.org/jira/browse/SPARK-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6270. --- Resolution: Duplicate This was fixed by the child bug spawned from it (SPARK-12299). >

[jira] [Updated] (SPARK-24227) Not able to submit spark job to kubernetes on 2.3

[ https://issues.apache.org/jira/browse/SPARK-24227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe Cavalcanti updated SPARK-24227: -- Description: Hi, I'm trying to submit a spark job to kubernetes with no success, I

[jira] [Updated] (SPARK-6810) Performance benchmarks for SparkR

[ https://issues.apache.org/jira/browse/SPARK-6810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-6810: -- Priority: Major (was: Critical) > Performance benchmarks for SparkR >

[jira] [Created] (SPARK-24227) Not able to submit spark job to kubernetes on 2.3

Felipe Cavalcanti created SPARK-24227: - Summary: Not able to submit spark job to kubernetes on 2.3 Key: SPARK-24227 URL: https://issues.apache.org/jira/browse/SPARK-24227 Project: Spark

[jira] [Resolved] (SPARK-7354) Flaky test: o.a.s.deploy.SparkSubmitSuite --jars

[ https://issues.apache.org/jira/browse/SPARK-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-7354. --- Resolution: Cannot Reproduce > Flaky test: o.a.s.deploy.SparkSubmitSuite --jars >

[jira] [Commented] (SPARK-7354) Flaky test: o.a.s.deploy.SparkSubmitSuite --jars

[ https://issues.apache.org/jira/browse/SPARK-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469435#comment-16469435 ] Marcelo Vanzin commented on SPARK-7354: --- Closest thing here might be SPARK-19964, but this is so old

[jira] [Updated] (SPARK-7839) Augment build environment to support native libraries with SparkR

[ https://issues.apache.org/jira/browse/SPARK-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-7839: -- Priority: Major (was: Critical) > Augment build environment to support native libraries with

[jira] [Resolved] (SPARK-8447) Test external shuffle service with all shuffle managers

[ https://issues.apache.org/jira/browse/SPARK-8447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-8447. --- Resolution: Won't Do There's only one shuffle manager left. > Test external shuffle service

[jira] [Resolved] (SPARK-10486) Spark intermittently fails to recover from a worker failure (in standalone mode)

[ https://issues.apache.org/jira/browse/SPARK-10486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-10486. Resolution: Cannot Reproduce I'm going to close this given that since 1.4 this whole code

[jira] [Resolved] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11278. Resolution: Cannot Reproduce I'm going to close this since it's way out of date at this

[jira] [Resolved] (SPARK-11851) Unable to start spark thrift server against secured hive metastore(GSS initiate failed)

[ https://issues.apache.org/jira/browse/SPARK-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11851. Resolution: Duplicate > Unable to start spark thrift server against secured hive

[jira] [Resolved] (SPARK-18468) Flaky test: org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-9757 Persist Parquet relation with decimal column

[ https://issues.apache.org/jira/browse/SPARK-18468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-18468. Resolution: Cannot Reproduce Doesn't seem that flaky, so closing for now. > Flaky test:

[jira] [Updated] (SPARK-18468) Flaky test: org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-9757 Persist Parquet relation with decimal column

[ https://issues.apache.org/jira/browse/SPARK-18468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18468: --- Component/s: (was: Spark Core) SQL > Flaky test:

[jira] [Updated] (SPARK-18468) Flaky test: org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-9757 Persist Parquet relation with decimal column

[ https://issues.apache.org/jira/browse/SPARK-18468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18468: --- Priority: Major (was: Critical) > Flaky test:

[jira] [Updated] (SPARK-23346) Failed tasks reported as success if the failure reason is not ExceptionFailure

[ https://issues.apache.org/jira/browse/SPARK-23346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-23346: --- Priority: Major (was: Critical) > Failed tasks reported as success if the failure reason is

[jira] [Updated] (SPARK-23519) Create View Commands Fails with The view output (col1,col1) contains duplicate column name

[ https://issues.apache.org/jira/browse/SPARK-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-23519: --- Priority: Major (was: Critical) > Create View Commands Fails with The view output

[jira] [Resolved] (SPARK-23527) Error with spark-submit and kerberos with TLS-enabled Hadoop cluster

[ https://issues.apache.org/jira/browse/SPARK-23527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23527. Resolution: Not A Problem Doesn't seem like a bug. Either a Spark or KMS config issue.

[jira] [Resolved] (SPARK-23709) BaggedPoint.convertToBaggedRDDSamplingWithReplacement does not guarantee the sum of weights

[ https://issues.apache.org/jira/browse/SPARK-23709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23709. Resolution: Information Provided Please use the mailing lists to ask questions.

[jira] [Updated] (SPARK-24226) while reading data from oracle 12c from spark and using the numofpartition more than 1 is not returning the exact count

[ https://issues.apache.org/jira/browse/SPARK-24226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-24226: --- Target Version/s: (was: 2.2.0) > while reading data from oracle 12c from spark and using

[jira] [Updated] (SPARK-24226) while reading data from oracle 12c from spark and using the numofpartition more than 1 is not returning the exact count

[ https://issues.apache.org/jira/browse/SPARK-24226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-24226: --- Priority: Major (was: Blocker) > while reading data from oracle 12c from spark and using

[jira] [Created] (SPARK-24226) while reading data from oracle 12c from spark and using the numofpartition more than 1 is not returning the exact count

Chandan created SPARK-24226: --- Summary: while reading data from oracle 12c from spark and using the numofpartition more than 1 is not returning the exact count Key: SPARK-24226 URL:

[jira] [Comment Edited] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469335#comment-16469335 ] Imran Rashid edited comment on SPARK-23206 at 5/9/18 7:16 PM: -- Hi, I think

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469335#comment-16469335 ] Imran Rashid commented on SPARK-23206: -- Hi, I think getting together to discuss the design is still

[jira] [Assigned] (SPARK-23852) Parquet MR bug can lead to incorrect SQL results

[ https://issues.apache.org/jira/browse/SPARK-23852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23852: Assignee: (was: Apache Spark) > Parquet MR bug can lead to incorrect SQL results >

[jira] [Commented] (SPARK-23852) Parquet MR bug can lead to incorrect SQL results

[ https://issues.apache.org/jira/browse/SPARK-23852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469330#comment-16469330 ] Apache Spark commented on SPARK-23852: -- User 'henryr' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23852) Parquet MR bug can lead to incorrect SQL results

[ https://issues.apache.org/jira/browse/SPARK-23852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23852: Assignee: Apache Spark > Parquet MR bug can lead to incorrect SQL results >

[jira] [Created] (SPARK-24225) Support closing AutoClosable objects in MemoryStore so Broadcast Variables can be released properly

Doug Rohrer created SPARK-24225: --- Summary: Support closing AutoClosable objects in MemoryStore so Broadcast Variables can be released properly Key: SPARK-24225 URL: https://issues.apache.org/jira/browse/SPARK-24225

[jira] [Resolved] (SPARK-24214) StreamingRelationV2/StreamingExecutionRelation/ContinuousExecutionRelation.toJSON should not fail

[ https://issues.apache.org/jira/browse/SPARK-24214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-24214. -- Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by pull

[jira] [Updated] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] spark_user updated SPARK-24217: --- Description: We should display prediction and id corresponding to all the nodes.  Currently PIC is

[jira] [Comment Edited] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469243#comment-16469243 ] spark_user edited comment on SPARK-24217 at 5/9/18 6:20 PM: Thanks for the

[jira] [Commented] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469245#comment-16469245 ] spark_user commented on SPARK-24217: PIC should return the cluster indices of each vertex of the

[jira] [Commented] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469243#comment-16469243 ] spark_user commented on SPARK-24217: Thanks for the comment Joseph K. Bradley. Actually the issue is

[jira] [Updated] (SPARK-24217) Power Iteration Clustering is not displaying cluster indices corresponding to some vertices.

[ https://issues.apache.org/jira/browse/SPARK-24217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] spark_user updated SPARK-24217: --- Description: We should display prediction and id corresponding to all the nodes. As per the

  1   2   >