[jira] [Created] (SPARK-23474) mapWithState + async operations = no checkpointing

Daniel Lanza García created SPARK-23474: --- Summary: mapWithState + async operations = no checkpointing Key: SPARK-23474 URL: https://issues.apache.org/jira/browse/SPARK-23474 Project: Spark

[jira] [Updated] (SPARK-23473) spark.catalog.listTables error when database name starts with a number

[ https://issues.apache.org/jira/browse/SPARK-23473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Goun Na updated SPARK-23473: Description: Errors when Hive database name starts with a number such as 11st.  Attached is full error

[jira] [Updated] (SPARK-23473) spark.catalog.listTables error when database name starts with a number

[ https://issues.apache.org/jira/browse/SPARK-23473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Goun Na updated SPARK-23473: Description: Errors when Hive database name starts with a number such as 11st. 

[jira] [Updated] (SPARK-23473) spark.catalog.listTables error when database name starts with a number

[ https://issues.apache.org/jira/browse/SPARK-23473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Goun Na updated SPARK-23473: Attachment: spark_catalog_err.txt > spark.catalog.listTables error when database name starts with a number

[jira] [Updated] (SPARK-23473) spark.catalog.listTables error when database name starts with a number

[ https://issues.apache.org/jira/browse/SPARK-23473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Goun Na updated SPARK-23473: Description: Errors when Hive database name starts with a number such as 11st. Full error message is

[jira] [Updated] (SPARK-23473) spark.catalog.listTables error when database name starts with a number

[ https://issues.apache.org/jira/browse/SPARK-23473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Goun Na updated SPARK-23473: Description:     scala> spark.catalog.setCurrentDatabase("11st") scala> spark.catalog.listTables

[jira] [Updated] (SPARK-23473) spark.catalog.listTables error when database name starts with a number

[ https://issues.apache.org/jira/browse/SPARK-23473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Goun Na updated SPARK-23473: Priority: Trivial (was: Minor) > spark.catalog.listTables error when database name starts with a number >

[jira] [Created] (SPARK-23473) spark.catalog.listTables error when database name starts with a number

Goun Na created SPARK-23473: --- Summary: spark.catalog.listTables error when database name starts with a number Key: SPARK-23473 URL: https://issues.apache.org/jira/browse/SPARK-23473 Project: Spark

[jira] [Commented] (SPARK-23303) improve the explain result for data source v2 relations

[ https://issues.apache.org/jira/browse/SPARK-23303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370992#comment-16370992 ] Apache Spark commented on SPARK-23303: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370924#comment-16370924 ] Cody Koeninger commented on SPARK-18057: Just doing the upgrade is probably a good starting point

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370917#comment-16370917 ] Manan Bakshi commented on SPARK-23463: -- Hi Marco, That makes sense. However, this same code used to

[jira] [Assigned] (SPARK-23424) Add codegenStageId in comment

[ https://issues.apache.org/jira/browse/SPARK-23424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23424: --- Assignee: Kazuaki Ishizaki > Add codegenStageId in comment > -

[jira] [Resolved] (SPARK-23424) Add codegenStageId in comment

[ https://issues.apache.org/jira/browse/SPARK-23424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23424. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20612

[jira] [Updated] (SPARK-23053) taskBinarySerialization and task partitions calculate in DagScheduler.submitMissingTasks should keep the same RDD checkpoint status

[ https://issues.apache.org/jira/browse/SPARK-23053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-23053: - Fix Version/s: 2.1.3 > taskBinarySerialization and task partitions calculate in >

[jira] [Resolved] (SPARK-23454) Add Trigger information to the Structured Streaming programming guide

[ https://issues.apache.org/jira/browse/SPARK-23454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23454. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by

[jira] [Resolved] (SPARK-23468) Failure to authenticate with old shuffle service

[ https://issues.apache.org/jira/browse/SPARK-23468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23468. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20643

[jira] [Assigned] (SPARK-23468) Failure to authenticate with old shuffle service

[ https://issues.apache.org/jira/browse/SPARK-23468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23468: -- Assignee: Marcelo Vanzin > Failure to authenticate with old shuffle service >

[jira] [Resolved] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal resolved SPARK-23470. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.3.0 Issue

[jira] [Commented] (SPARK-23369) HiveClientSuites fails with unresolved dependency

[ https://issues.apache.org/jira/browse/SPARK-23369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370832#comment-16370832 ] Jose Torres commented on SPARK-23369: - Still seeing this in e.g. 

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370780#comment-16370780 ] Michael Armbrust commented on SPARK-18057: -- +1 to upgrading and it would also be great to add

[jira] [Assigned] (SPARK-23434) Spark should not warn `metadata directory` for a HDFS file path

[ https://issues.apache.org/jira/browse/SPARK-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-23434: Assignee: Dongjoon Hyun > Spark should not warn `metadata directory` for a HDFS file path

[jira] [Resolved] (SPARK-23434) Spark should not warn `metadata directory` for a HDFS file path

[ https://issues.apache.org/jira/browse/SPARK-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-23434. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20616

[jira] [Assigned] (SPARK-23408) Flaky test: StreamingOuterJoinSuite.left outer early state exclusion on right

[ https://issues.apache.org/jira/browse/SPARK-23408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23408: Assignee: (was: Apache Spark) > Flaky test: StreamingOuterJoinSuite.left outer early

[jira] [Assigned] (SPARK-23408) Flaky test: StreamingOuterJoinSuite.left outer early state exclusion on right

[ https://issues.apache.org/jira/browse/SPARK-23408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23408: Assignee: Apache Spark > Flaky test: StreamingOuterJoinSuite.left outer early state

[jira] [Commented] (SPARK-23408) Flaky test: StreamingOuterJoinSuite.left outer early state exclusion on right

[ https://issues.apache.org/jira/browse/SPARK-23408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370761#comment-16370761 ] Apache Spark commented on SPARK-23408: -- User 'jose-torres' has created a pull request for this

[jira] [Updated] (SPARK-23434) Spark should not warn `metadata directory` for a HDFS file path

[ https://issues.apache.org/jira/browse/SPARK-23434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23434: -- Description: In a kerberized cluster, when Spark reads a file path (e.g. `people.json`), it

[jira] [Commented] (SPARK-23472) Add config properties for administrator JVM options

[ https://issues.apache.org/jira/browse/SPARK-23472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370659#comment-16370659 ] Apache Spark commented on SPARK-23472: -- User 'rdblue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23472) Add config properties for administrator JVM options

[ https://issues.apache.org/jira/browse/SPARK-23472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23472: Assignee: (was: Apache Spark) > Add config properties for administrator JVM options >

[jira] [Assigned] (SPARK-23472) Add config properties for administrator JVM options

[ https://issues.apache.org/jira/browse/SPARK-23472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23472: Assignee: Apache Spark > Add config properties for administrator JVM options >

[jira] [Created] (SPARK-23472) Add config properties for administrator JVM options

Ryan Blue created SPARK-23472: - Summary: Add config properties for administrator JVM options Key: SPARK-23472 URL: https://issues.apache.org/jira/browse/SPARK-23472 Project: Spark Issue Type:

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370642#comment-16370642 ] Sönke Liebau commented on SPARK-18057: -- I think in addition to the naming convention issue there was

[jira] [Commented] (SPARK-15060) Fix stack overflow when executing long lineage transform without checkpoint

[ https://issues.apache.org/jira/browse/SPARK-15060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370637#comment-16370637 ] Anil Villait commented on SPARK-15060: -- Is there a workaround to this problem by increasing the

[jira] [Assigned] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23470: Assignee: Apache Spark > org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription

[jira] [Assigned] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23470: Assignee: (was: Apache Spark) >

[jira] [Commented] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370632#comment-16370632 ] Apache Spark commented on SPARK-23470: -- User 'vanzin' has created a pull request for this issue:

[jira] [Created] (SPARK-23471) RandomForestClassificationModel save() - incorrect metadata

Keepun created SPARK-23471: -- Summary: RandomForestClassificationModel save() - incorrect metadata Key: SPARK-23471 URL: https://issues.apache.org/jira/browse/SPARK-23471 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23377) Bucketizer with multiple columns persistence bug

[ https://issues.apache.org/jira/browse/SPARK-23377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23377: -- Fix Version/s: (was: 2.3.1) 2.3.0 > Bucketizer with multiple columns

[jira] [Updated] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23470: - Description: I was testing 2.3.0 RC3 and found that it's easy to hit "read timeout" when

[jira] [Resolved] (SPARK-23316) AnalysisException after max iteration reached for IN query

[ https://issues.apache.org/jira/browse/SPARK-23316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-23316. --- Resolution: Fixed Hi, [~smilegator]. Could you update the assignee with [~bograd]? >

[jira] [Updated] (SPARK-23316) AnalysisException after max iteration reached for IN query

[ https://issues.apache.org/jira/browse/SPARK-23316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23316: -- Fix Version/s: 2.3.0 > AnalysisException after max iteration reached for IN query >

[jira] [Commented] (SPARK-23468) Failure to authenticate with old shuffle service

[ https://issues.apache.org/jira/browse/SPARK-23468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370582#comment-16370582 ] Apache Spark commented on SPARK-23468: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23468) Failure to authenticate with old shuffle service

[ https://issues.apache.org/jira/browse/SPARK-23468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23468: Assignee: Apache Spark > Failure to authenticate with old shuffle service >

[jira] [Assigned] (SPARK-23468) Failure to authenticate with old shuffle service

[ https://issues.apache.org/jira/browse/SPARK-23468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23468: Assignee: (was: Apache Spark) > Failure to authenticate with old shuffle service >

[jira] [Commented] (SPARK-23410) Unable to read jsons in charset different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370580#comment-16370580 ] Dongjoon Hyun commented on SPARK-23410: --- I removed the target version, 2.3.0, from here. > Unable

[jira] [Updated] (SPARK-23410) Unable to read jsons in charset different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23410: -- Target Version/s: (was: 2.3.0) > Unable to read jsons in charset different from UTF-8 >

[jira] [Commented] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370532#comment-16370532 ] Marcelo Vanzin commented on SPARK-23470: Sure, I'll take a look in the PM. >

[jira] [Updated] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23470: - Target Version/s: 2.3.0 > org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too

[jira] [Commented] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370531#comment-16370531 ] Shixiong Zhu commented on SPARK-23470: -- [~vanzin] could you make a PR to fix it? I can help you test

[jira] [Updated] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-23470: - Priority: Blocker (was: Major) > org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription

[jira] [Commented] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370508#comment-16370508 ] Marcelo Vanzin commented on SPARK-23470: It should be pretty easy to make

[jira] [Commented] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

[ https://issues.apache.org/jira/browse/SPARK-23470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370496#comment-16370496 ] Shixiong Zhu commented on SPARK-23470: -- [~vanzin] [~cloud_fan] >

[jira] [Created] (SPARK-23470) org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow

Shixiong Zhu created SPARK-23470: Summary: org.apache.spark.ui.jobs.ApiHelper.lastStageNameAndDescription is too slow Key: SPARK-23470 URL: https://issues.apache.org/jira/browse/SPARK-23470 Project:

[jira] [Updated] (SPARK-23469) HashingTF should use corrected MurmurHash3 implementation

[ https://issues.apache.org/jira/browse/SPARK-23469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-23469: -- Description: [SPARK-23381] added a corrected MurmurHash3 implementation but left the

[jira] [Created] (SPARK-23469) HashingTF should use corrected MurmurHash3 implementation

Joseph K. Bradley created SPARK-23469: - Summary: HashingTF should use corrected MurmurHash3 implementation Key: SPARK-23469 URL: https://issues.apache.org/jira/browse/SPARK-23469 Project: Spark

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370460#comment-16370460 ] Cody Koeninger commented on SPARK-18057: My guess is that DStream based integrations aren't

[jira] [Created] (SPARK-23468) Failure to authenticate with old shuffle service

Marcelo Vanzin created SPARK-23468: -- Summary: Failure to authenticate with old shuffle service Key: SPARK-23468 URL: https://issues.apache.org/jira/browse/SPARK-23468 Project: Spark Issue

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370434#comment-16370434 ] Sönke Liebau commented on SPARK-18057: -- The upcoming release of Kafka 1.1 will support delegation

[jira] [Resolved] (SPARK-23456) Turn on `native` ORC implementation by default

[ https://issues.apache.org/jira/browse/SPARK-23456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23456. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.4.0 > Turn on `native` ORC

[jira] [Resolved] (SPARK-21783) Turn on ORC filter push-down by default

[ https://issues.apache.org/jira/browse/SPARK-21783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21783. - Resolution: Fixed Fix Version/s: 2.4.0 > Turn on ORC filter push-down by default >

[jira] [Comment Edited] (SPARK-21529) Uniontype not supported when reading from Hive tables.

[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370142#comment-16370142 ] Tom Wadeson edited comment on SPARK-21529 at 2/20/18 3:32 PM: -- Thanks for

[jira] [Commented] (SPARK-21529) Uniontype not supported when reading from Hive tables.

[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370142#comment-16370142 ] Tom Wadeson commented on SPARK-21529: - Thanks for the update. I'll keep you posted if we come up with

[jira] [Commented] (SPARK-23464) MesosClusterScheduler double-escapes parameters to bash command

[ https://issues.apache.org/jira/browse/SPARK-23464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370132#comment-16370132 ] Apache Spark commented on SPARK-23464: -- User 'krcz' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23464) MesosClusterScheduler double-escapes parameters to bash command

[ https://issues.apache.org/jira/browse/SPARK-23464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23464: Assignee: Apache Spark > MesosClusterScheduler double-escapes parameters to bash command

[jira] [Assigned] (SPARK-23464) MesosClusterScheduler double-escapes parameters to bash command

[ https://issues.apache.org/jira/browse/SPARK-23464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23464: Assignee: (was: Apache Spark) > MesosClusterScheduler double-escapes parameters to

[jira] [Resolved] (SPARK-23423) Application declines any offers when killed+active executors rich spark.dynamicAllocation.maxExecutors

[ https://issues.apache.org/jira/browse/SPARK-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Berman resolved SPARK-23423. - Resolution: Duplicate > Application declines any offers when killed+active executors rich >

[jira] [Commented] (SPARK-23423) Application declines any offers when killed+active executors rich spark.dynamicAllocation.maxExecutors

[ https://issues.apache.org/jira/browse/SPARK-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370114#comment-16370114 ] Igor Berman commented on SPARK-23423: - [~skonto] I've investigated failures a bit more so found 2

[jira] [Comment Edited] (SPARK-21529) Uniontype not supported when reading from Hive tables.

[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370100#comment-16370100 ] Elliot West edited comment on SPARK-21529 at 2/20/18 2:44 PM: -- Yes 

[jira] [Comment Edited] (SPARK-21529) Uniontype not supported when reading from Hive tables.

[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370100#comment-16370100 ] Elliot West edited comment on SPARK-21529 at 2/20/18 2:44 PM: -- Yes 

[jira] [Comment Edited] (SPARK-21529) Uniontype not supported when reading from Hive tables.

[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370100#comment-16370100 ] Elliot West edited comment on SPARK-21529 at 2/20/18 2:43 PM: -- Yes 

[jira] [Comment Edited] (SPARK-21529) Uniontype not supported when reading from Hive tables.

[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370100#comment-16370100 ] Elliot West edited comment on SPARK-21529 at 2/20/18 2:34 PM: -- Yes 

[jira] [Commented] (SPARK-21529) Uniontype not supported when reading from Hive tables.

[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370100#comment-16370100 ] Elliot West commented on SPARK-21529: - Yes [~tomwadeson], but not in a nice way: * We implemented a

[jira] [Updated] (SPARK-23448) Dataframe returns wrong result when column don't respect datatype

[ https://issues.apache.org/jira/browse/SPARK-23448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-23448: Component/s: (was: Spark Core) SQL > Dataframe returns wrong result

[jira] [Commented] (SPARK-10912) Improve Spark metrics executor.filesystem

[ https://issues.apache.org/jira/browse/SPARK-10912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370089#comment-16370089 ] Gil Vernik commented on SPARK-10912: This patch need to be generic and doesn't include "s3a" in the

[jira] [Commented] (SPARK-21529) Uniontype not supported when reading from Hive tables.

[ https://issues.apache.org/jira/browse/SPARK-21529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370088#comment-16370088 ] Tom Wadeson commented on SPARK-21529: - I've just bumped into this exact issue. Did you manage to find

[jira] [Comment Edited] (SPARK-23423) Application declines any offers when killed+active executors rich spark.dynamicAllocation.maxExecutors

[ https://issues.apache.org/jira/browse/SPARK-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370074#comment-16370074 ] Stavros Kontopoulos edited comment on SPARK-23423 at 2/20/18 2:04 PM:

[jira] [Comment Edited] (SPARK-23423) Application declines any offers when killed+active executors rich spark.dynamicAllocation.maxExecutors

[ https://issues.apache.org/jira/browse/SPARK-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370074#comment-16370074 ] Stavros Kontopoulos edited comment on SPARK-23423 at 2/20/18 2:04 PM:

[jira] [Commented] (SPARK-23423) Application declines any offers when killed+active executors rich spark.dynamicAllocation.maxExecutors

[ https://issues.apache.org/jira/browse/SPARK-23423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370074#comment-16370074 ] Stavros Kontopoulos commented on SPARK-23423: - [~igor.berman] Thnx for supplying the info. We

[jira] [Resolved] (SPARK-23383) Make a distribution should exit with usage while detecting wrong options

[ https://issues.apache.org/jira/browse/SPARK-23383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23383. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20571

[jira] [Assigned] (SPARK-23383) Make a distribution should exit with usage while detecting wrong options

[ https://issues.apache.org/jira/browse/SPARK-23383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23383: - Assignee: Kent Yao > Make a distribution should exit with usage while detecting wrong options >

[jira] [Commented] (SPARK-20845) Support specification of column names in INSERT INTO

[ https://issues.apache.org/jira/browse/SPARK-20845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370056#comment-16370056 ] Mihaly Toth commented on SPARK-20845: - As shown in the links this duplicates SPARK-21548. However, I

[jira] [Commented] (SPARK-23463) Filter operation fails to handle blank values and evicts rows that even satisfy the filtering condition

[ https://issues.apache.org/jira/browse/SPARK-23463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370046#comment-16370046 ] Marco Gaido commented on SPARK-23463: - Hi [~m.bakshi11]. The problem is very easy. The column `val`

[jira] [Commented] (SPARK-23443) Spark with Glue as external catalog

[ https://issues.apache.org/jira/browse/SPARK-23443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370015#comment-16370015 ] Ameen Tayyebi commented on SPARK-23443: --- Hello, I have this locally working (just limited read

[jira] [Commented] (SPARK-19755) Blacklist is always active for MesosCoarseGrainedSchedulerBackend. As result - scheduler cannot create an executor after some time.

[ https://issues.apache.org/jira/browse/SPARK-19755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370002#comment-16370002 ] Apache Spark commented on SPARK-19755: -- User 'IgorBerman' has created a pull request for this issue:

[jira] [Resolved] (SPARK-23240) PythonWorkerFactory issues unhelpful message when pyspark.daemon produces bogus stdout

[ https://issues.apache.org/jira/browse/SPARK-23240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23240. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20424

[jira] [Assigned] (SPARK-23240) PythonWorkerFactory issues unhelpful message when pyspark.daemon produces bogus stdout

[ https://issues.apache.org/jira/browse/SPARK-23240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23240: Assignee: Bruce Robbins > PythonWorkerFactory issues unhelpful message when

[jira] [Commented] (SPARK-11182) HDFS Delegation Token will be expired when calling "UserGroupInformation.getCurrentUser.addCredentials" in HA mode

[ https://issues.apache.org/jira/browse/SPARK-11182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369914#comment-16369914 ] Steve Loughran commented on SPARK-11182: bug is in HDFS; been fixed in 2.8.2+ with cherry

[jira] [Assigned] (SPARK-23288) Incorrect number of written records in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23288: Assignee: (was: Apache Spark) > Incorrect number of written records in structured

[jira] [Commented] (SPARK-23288) Incorrect number of written records in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369898#comment-16369898 ] Apache Spark commented on SPARK-23288: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-23288) Incorrect number of written records in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-23288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23288: Assignee: Apache Spark > Incorrect number of written records in structured streaming >

[jira] [Commented] (SPARK-11182) HDFS Delegation Token will be expired when calling "UserGroupInformation.getCurrentUser.addCredentials" in HA mode

[ https://issues.apache.org/jira/browse/SPARK-11182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369897#comment-16369897 ] Sumit Nigam commented on SPARK-11182: - Also, is adding --conf

[jira] [Commented] (SPARK-11182) HDFS Delegation Token will be expired when calling "UserGroupInformation.getCurrentUser.addCredentials" in HA mode

[ https://issues.apache.org/jira/browse/SPARK-11182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369896#comment-16369896 ] Sumit Nigam commented on SPARK-11182: - Is this issue fixed in later versions of spark? > HDFS

[jira] [Resolved] (SPARK-23203) DataSourceV2 should use immutable trees.

[ https://issues.apache.org/jira/browse/SPARK-23203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23203. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20387

[jira] [Assigned] (SPARK-23203) DataSourceV2 should use immutable trees.

[ https://issues.apache.org/jira/browse/SPARK-23203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23203: --- Assignee: Ryan Blue > DataSourceV2 should use immutable trees. >