[jira] [Created] (SPARK-32615) AQE aggregateMetrics java.util.NoSuchElementException

2020-08-13 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-32615: --- Summary: AQE aggregateMetrics java.util.NoSuchElementException Key: SPARK-32615 URL: https://issues.apache.org/jira/browse/SPARK-32615 Project: Spark Issue Typ

[jira] [Assigned] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32609: Assignee: Apache Spark > Incorrect exchange reuse with DataSourceV2 > ---

[jira] [Commented] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177528#comment-17177528 ] Apache Spark commented on SPARK-32609: -- User 'mingjialiu' has created a pull reques

[jira] [Assigned] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32609: Assignee: (was: Apache Spark) > Incorrect exchange reuse with DataSourceV2 >

[jira] [Resolved] (SPARK-32345) SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create Spark client for Spark session

2020-08-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-32345. - Resolution: Invalid > SemanticException Failed to get a spark session: > org.apache.hadoop.hive

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every lin

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every lin

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every lin

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Issue Type: Bug (was: New Feature) > Support for treating the line as valid record if it starts with \u00

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Affects Version/s: 2.4.5 > Support for treating the line as valid record if it starts with \u or > nu

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Issue Type: New Feature (was: Improvement) > Support for treating the line as valid record if it starts w

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every lin

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Attachment: screenshot-1.png > Support for treating the line as valid record if it starts with \u or

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every lin

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Fix Version/s: (was: 3.0.0) > Support for treating the line as valid record if it starts with \u o

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Affects Version/s: (was: 2.3.1) 3.0.0 > Support for treating the line as valid

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Priority: Major (was: Minor) > Support for treating the line as valid record if it starts with \u or

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Component/s: Spark Core > Support for treating the line as valid record if it starts with \u or > nul

[jira] [Created] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
Chandan created SPARK-32614: --- Summary: Support for treating the line as valid record if it starts with \u or null character, or starts with any character mentioned as comment Key: SPARK-32614 URL: https://issues.ap

[jira] [Commented] (SPARK-32345) SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create Spark client for Spark session

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177468#comment-17177468 ] ZhouDaHong commented on SPARK-32345: If the cause of the version conflict is exclude

[jira] [Updated] (SPARK-32357) Publish failed and succeeded test reports in GitHub Actions

2020-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32357: -- Summary: Publish failed and succeeded test reports in GitHub Actions (was: Investigate test r

[jira] [Assigned] (SPARK-32357) Investigate test result reporter integration

2020-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32357: - Assignee: Hyukjin Kwon > Investigate test result reporter integration > ---

[jira] [Resolved] (SPARK-32357) Investigate test result reporter integration

2020-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32357. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29333 [https://

[jira] [Comment Edited] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177465#comment-17177465 ] ZhouDaHong edited comment on SPARK-32587 at 8/14/20, 3:54 AM:

[jira] [Comment Edited] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177465#comment-17177465 ] ZhouDaHong edited comment on SPARK-32587 at 8/14/20, 3:54 AM:

[jira] [Commented] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177465#comment-17177465 ] ZhouDaHong commented on SPARK-32587: 抱歉,我不是特别明白你的问题。你是不是说数据存在空值列的时候,无法写入到sql server

[jira] [Commented] (SPARK-21774) The rule PromoteStrings cast string to a wrong data type

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177461#comment-17177461 ] ZhouDaHong commented on SPARK-21774: Hello, you compare the value of a field of stri

[jira] [Commented] (SPARK-9182) filter and groupBy on DataFrames are not passed through to jdbc source

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177455#comment-17177455 ] ZhouDaHong commented on SPARK-9182: --- Hello, it seems that the problem is that the "Sal"

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32609: - Shepherd: (was: Reynold Xin) > Incorrect exchange reuse with DataSourceV2 > --

[jira] [Commented] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177439#comment-17177439 ] Hyukjin Kwon commented on SPARK-32604: -- Oh right. So I guess it will be fixed in Sp

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177437#comment-17177437 ] Hyukjin Kwon commented on SPARK-32053: -- Seems like Hadoop and Windows issue assumin

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32053: - Affects Version/s: 3.0.0 > pyspark save of serialized model is failing for windows. > --

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32053: - Component/s: Windows > pyspark save of serialized model is failing for windows. > --

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32053: - Priority: Minor (was: Blocker) > pyspark save of serialized model is failing for windows. > ---

[jira] [Commented] (SPARK-31947) Solve string value error about Date/Timestamp in ScriptTransform

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177436#comment-17177436 ] Hyukjin Kwon commented on SPARK-31947: -- [~angerszhuuu] can you update which PR fixe

[jira] [Assigned] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-12312: Assignee: Gabor Somogyi > JDBC connection to Kerberos secured databases fails on remote e

[jira] [Commented] (SPARK-32598) Not able to see driver logs in spark history server in standalone mode

2020-08-13 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177429#comment-17177429 ] Lantao Jin commented on SPARK-32598: [~sriramgr] PullRequest is welcome. Please comm

[jira] [Assigned] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32608: Assignee: (was: Apache Spark) > Script Transform DELIMIT value error > -

[jira] [Assigned] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32608: Assignee: Apache Spark > Script Transform DELIMIT value error >

[jira] [Commented] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177420#comment-17177420 ] Apache Spark commented on SPARK-32608: -- User 'AngersZh' has created a pull requ

[jira] [Comment Edited] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-08-13 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177401#comment-17177401 ] Dustin Smith edited comment on SPARK-32046 at 8/14/20, 12:51 AM: -

[jira] [Comment Edited] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-08-13 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177401#comment-17177401 ] Dustin Smith edited comment on SPARK-32046 at 8/14/20, 12:49 AM: -

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-08-13 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177401#comment-17177401 ] Dustin Smith commented on SPARK-32046: -- [~maropu] the definition of current timesta

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-08-13 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177398#comment-17177398 ] Takeshi Yamamuro commented on SPARK-32046: -- This issue depends on ZP or Jupyter

[jira] [Assigned] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32613: Assignee: Apache Spark > DecommissionWorkerSuite has started failing sporadically again >

[jira] [Assigned] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32613: Assignee: (was: Apache Spark) > DecommissionWorkerSuite has started failing sporadica

[jira] [Commented] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177368#comment-17177368 ] Apache Spark commented on SPARK-32613: -- User 'agrawaldevesh' has created a pull req

[jira] [Created] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

2020-08-13 Thread Devesh Agrawal (Jira)
Devesh Agrawal created SPARK-32613: -- Summary: DecommissionWorkerSuite has started failing sporadically again Key: SPARK-32613 URL: https://issues.apache.org/jira/browse/SPARK-32613 Project: Spark

[jira] [Updated] (SPARK-32611) Querying ORC table in Spark3 using spark.sql.orc.impl=hive produces incorrect when timestamp is present in predicate

2020-08-13 Thread Sumeet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumeet updated SPARK-32611: --- Description: *How to reproduce this behavior?* * TZ="America/Los_Angeles" ./bin/spark-shell * sql("set spa

[jira] [Commented] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Zach Cahoone (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177315#comment-17177315 ] Zach Cahoone commented on SPARK-32604: -- The code here ([https://github.com/apache/

[jira] [Created] (SPARK-32612) int columns produce inconsistent results on pandas UDFs

2020-08-13 Thread Robert Joseph Evans (Jira)
Robert Joseph Evans created SPARK-32612: --- Summary: int columns produce inconsistent results on pandas UDFs Key: SPARK-32612 URL: https://issues.apache.org/jira/browse/SPARK-32612 Project: Spark

[jira] [Commented] (SPARK-25557) ORC predicate pushdown for nested fields

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177301#comment-17177301 ] Apache Spark commented on SPARK-25557: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-25557) ORC predicate pushdown for nested fields

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177300#comment-17177300 ] Apache Spark commented on SPARK-25557: -- User 'viirya' has created a pull request fo

[jira] [Commented] (SPARK-31776) Literal lit() supports lists and numpy arrays

2020-08-13 Thread Zirui Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177281#comment-17177281 ] Zirui Xu commented on SPARK-31776: -- Hi [~mengxr], is this ticket suggesting adding supp

[jira] [Created] (SPARK-32611) Querying ORC table in Spark3 using spark.sql.orc.impl=hive produces incorrect when timestamp is present in predicate

2020-08-13 Thread Sumeet (Jira)
Sumeet created SPARK-32611: -- Summary: Querying ORC table in Spark3 using spark.sql.orc.impl=hive produces incorrect when timestamp is present in predicate Key: SPARK-32611 URL: https://issues.apache.org/jira/browse/SPAR

[jira] [Comment Edited] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177246#comment-17177246 ] Rohit Mishra edited comment on SPARK-32609 at 8/13/20, 6:28 PM: --

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32609: - Target Version/s: (was: 2.4.5, 2.4.7) > Incorrect exchange reuse with DataSourceV2 > -

[jira] [Commented] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177246#comment-17177246 ] Rohit Mishra commented on SPARK-32609: -- [~mingjial], Please refrain from adding Pri

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32609: - Priority: Major (was: Critical) > Incorrect exchange reuse with DataSourceV2 >

[jira] [Commented] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Mingjia Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177231#comment-17177231 ] Mingjia Liu commented on SPARK-32609: - I am currently working on a fix &  unit test

[jira] [Resolved] (SPARK-31198) Use graceful decommissioning as part of dynamic scaling

2020-08-13 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-31198. -- Fix Version/s: 3.1.0 Resolution: Fixed > Use graceful decommissioning as part of dynami

[jira] [Commented] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177204#comment-17177204 ] Apache Spark commented on SPARK-32610: -- User 'sarutak' has created a pull request f

[jira] [Assigned] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32610: Assignee: Kousuke Saruta (was: Apache Spark) > Fix the link to metrics.dropwizard.io in

[jira] [Assigned] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32610: Assignee: Apache Spark (was: Kousuke Saruta) > Fix the link to metrics.dropwizard.io in

[jira] [Commented] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177203#comment-17177203 ] Apache Spark commented on SPARK-32610: -- User 'sarutak' has created a pull request f

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Mingjia Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingjia Liu updated SPARK-32609: Target Version/s: 2.4.5, 2.4.7 (was: 2.4.5) > Incorrect exchange reuse with DataSourceV2 > --

[jira] [Commented] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-08-13 Thread Ruslan Dautkhanov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177202#comment-17177202 ] Ruslan Dautkhanov commented on SPARK-28367: --- [~gsomogyi] thanks! yep would be

[jira] [Commented] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Mingjia Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177199#comment-17177199 ] Mingjia Liu commented on SPARK-32609: - Mitigation: Turn off  spark.sql.exchange.reu

[jira] [Created] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-32610: -- Summary: Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version Key: SPARK-32610 URL: https://issues.apache.org/jira/browse/SPARK-32610

[jira] [Created] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Mingjia Liu (Jira)
Mingjia Liu created SPARK-32609: --- Summary: Incorrect exchange reuse with DataSourceV2 Key: SPARK-32609 URL: https://issues.apache.org/jira/browse/SPARK-32609 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-32350) Add batch write support on LevelDB to improve performance of HybridStore

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177112#comment-17177112 ] Apache Spark commented on SPARK-32350: -- User 'baohe-zhang' has created a pull reque

[jira] [Commented] (SPARK-32350) Add batch write support on LevelDB to improve performance of HybridStore

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177110#comment-17177110 ] Apache Spark commented on SPARK-32350: -- User 'baohe-zhang' has created a pull reque

[jira] [Updated] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32608: -- Description: For SQL   {code:java} SELECT TRANSFORM(a, b, c) ROW FORMAT DELIMITED FIELDS TERMINAT

[jira] [Created] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread angerszhu (Jira)
angerszhu created SPARK-32608: - Summary: Script Transform DELIMIT value error Key: SPARK-32608 URL: https://issues.apache.org/jira/browse/SPARK-32608 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-32607) Script Transformation no-serde read line should respect `TOK_TABLEROWFORMATLINES`

2020-08-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32607: -- Description: reader.readLine() should respect `TOK_TABLEROWFORMATLINES` {code:java} protected def crea

[jira] [Updated] (SPARK-32607) Script Transformation no-serde read line should respect `TOK_TABLEROWFORMATLINES`

2020-08-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32607: -- Description: reader.readLine() should respect `` {code:java} protected def createOutputIteratorWithout

[jira] [Created] (SPARK-32607) Script Transformation no-serde read line should respect `TOK_TABLEROWFORMATLINES`

2020-08-13 Thread angerszhu (Jira)
angerszhu created SPARK-32607: - Summary: Script Transformation no-serde read line should respect `TOK_TABLEROWFORMATLINES` Key: SPARK-32607 URL: https://issues.apache.org/jira/browse/SPARK-32607 Project:

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177084#comment-17177084 ] Kayal commented on SPARK-32053: --- ~\AppData\Local\IBMWS\miniconda3\envs\desktop\lib\site-pa

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177082#comment-17177082 ] Kayal commented on SPARK-32053: --- !image-2020-08-13-20-29-40-555.png!     > pyspark save

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: image-2020-08-13-20-29-40-555.png > pyspark save of serialized model is failing for windows. > ---

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: image-2020-08-13-20-28-28-779.png > pyspark save of serialized model is failing for windows. > ---

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: screenshot-1.png > pyspark save of serialized model is failing for windows. >

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177079#comment-17177079 ] Kayal commented on SPARK-32053: --- !image-2020-08-13-20-25-57-585.png!   > pyspark save of

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: image-2020-08-13-20-25-57-585.png > pyspark save of serialized model is failing for windows. > ---

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: image-2020-08-13-20-24-57-309.png > pyspark save of serialized model is failing for windows. > ---

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177077#comment-17177077 ] Kayal commented on SPARK-32053: --- !image-2020-08-13-20-24-57-309.png! > pyspark save of se

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177073#comment-17177073 ] Kayal commented on SPARK-32053: --- The code to reproduce the issue on windows jupyter notebo

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Priority: Blocker (was: Major) > pyspark save of serialized model is failing for windows. > -

[jira] [Reopened] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal reopened SPARK-32053: --- Hi, I have verified the issue in spark latest version 3.0.0 , the issue seems to be still there on windows.  

[jira] [Commented] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176996#comment-17176996 ] Hyukjin Kwon commented on SPARK-32587: -- Could you share the reproducible codes and

[jira] [Commented] (SPARK-32591) Add better api docs for stage level scheduling Resources

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176994#comment-17176994 ] Hyukjin Kwon commented on SPARK-32591: -- +100!! > Add better api docs for stage lev

[jira] [Commented] (SPARK-32601) Issue in converting an RDD of Arrow RecordBatches in v3.0.0

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176993#comment-17176993 ] Hyukjin Kwon commented on SPARK-32601: -- {{ArrowSerializer}} isn't supposed to be an

[jira] [Commented] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176992#comment-17176992 ] Hyukjin Kwon commented on SPARK-32604: -- Are you interested in submitting a PR to fi

[jira] [Updated] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32604: - Component/s: PySpark > Bug in ALSModel Python Documentation > --

[jira] [Updated] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32604: - Target Version/s: (was: 3.0.0) > Bug in ALSModel Python Documentation > --

[jira] [Commented] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176990#comment-17176990 ] Hyukjin Kwon commented on SPARK-32604: -- Please avoid setting target version which i

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

2020-08-13 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176966#comment-17176966 ] JinxinTang commented on SPARK-32500: Kindly ping [~hyukjin.kwon] It's my pleasure : 

[jira] [Commented] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-08-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176955#comment-17176955 ] Wenchen Fan commented on SPARK-32018: - [~Gengliang.Wang] we should create a new JIRA

[jira] [Assigned] (SPARK-32244) Build and run the Spark with test cases in Github Actions

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32244: Assignee: Hyukjin Kwon > Build and run the Spark with test cases in Github Actions >

[jira] [Created] (SPARK-32606) Remove the fork of action-download-artifact in test_report.yml

2020-08-13 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32606: Summary: Remove the fork of action-download-artifact in test_report.yml Key: SPARK-32606 URL: https://issues.apache.org/jira/browse/SPARK-32606 Project: Spark

  1   2   >