[jira] [Resolved] (SPARK-32345) SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create Spark client for Spark session

2020-08-13 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-32345. - Resolution: Invalid > SemanticException Failed to get a spark session: >

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Issue Type: Bug (was: New Feature) > Support for treating the line as valid record if it starts with

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Affects Version/s: 2.4.5 > Support for treating the line as valid record if it starts with \u or >

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Issue Type: New Feature (was: Improvement) > Support for treating the line as valid record if it starts

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Attachment: screenshot-1.png > Support for treating the line as valid record if it starts with \u or

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Description: In most of the data ware housing scenarios files does not have comment records and every

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Fix Version/s: (was: 3.0.0) > Support for treating the line as valid record if it starts with \u

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Affects Version/s: (was: 2.3.1) 3.0.0 > Support for treating the line as valid

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Priority: Major (was: Minor) > Support for treating the line as valid record if it starts with \u or

[jira] [Updated] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-32614: Component/s: Spark Core > Support for treating the line as valid record if it starts with \u or >

[jira] [Created] (SPARK-32614) Support for treating the line as valid record if it starts with \u0000 or null character, or starts with any character mentioned as comment

2020-08-13 Thread Chandan (Jira)
Chandan created SPARK-32614: --- Summary: Support for treating the line as valid record if it starts with \u or null character, or starts with any character mentioned as comment Key: SPARK-32614 URL:

[jira] [Commented] (SPARK-32345) SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create Spark client for Spark session

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177468#comment-17177468 ] ZhouDaHong commented on SPARK-32345: If the cause of the version conflict is excluded. You can look

[jira] [Updated] (SPARK-32357) Publish failed and succeeded test reports in GitHub Actions

2020-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32357: -- Summary: Publish failed and succeeded test reports in GitHub Actions (was: Investigate test

[jira] [Assigned] (SPARK-32357) Investigate test result reporter integration

2020-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32357: - Assignee: Hyukjin Kwon > Investigate test result reporter integration >

[jira] [Resolved] (SPARK-32357) Investigate test result reporter integration

2020-08-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32357. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29333

[jira] [Comment Edited] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177465#comment-17177465 ] ZhouDaHong edited comment on SPARK-32587 at 8/14/20, 3:54 AM: -- Sorry, I

[jira] [Comment Edited] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177465#comment-17177465 ] ZhouDaHong edited comment on SPARK-32587 at 8/14/20, 3:54 AM: -- Sorry, I

[jira] [Commented] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177465#comment-17177465 ] ZhouDaHong commented on SPARK-32587: 抱歉,我不是特别明白你的问题。你是不是说数据存在空值列的时候,无法写入到sql

[jira] [Commented] (SPARK-21774) The rule PromoteStrings cast string to a wrong data type

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177461#comment-17177461 ] ZhouDaHong commented on SPARK-21774: Hello, you compare the value of a field of string type with the

[jira] [Commented] (SPARK-9182) filter and groupBy on DataFrames are not passed through to jdbc source

2020-08-13 Thread ZhouDaHong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177455#comment-17177455 ] ZhouDaHong commented on SPARK-9182: --- Hello, it seems that the problem is that the "Sal" field is of

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32609: - Shepherd: (was: Reynold Xin) > Incorrect exchange reuse with DataSourceV2 >

[jira] [Commented] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177439#comment-17177439 ] Hyukjin Kwon commented on SPARK-32604: -- Oh right. So I guess it will be fixed in Spark 3.1

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177437#comment-17177437 ] Hyukjin Kwon commented on SPARK-32053: -- Seems like Hadoop and Windows issue assuming from the log:

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32053: - Affects Version/s: 3.0.0 > pyspark save of serialized model is failing for windows. >

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32053: - Component/s: Windows > pyspark save of serialized model is failing for windows. >

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32053: - Priority: Minor (was: Blocker) > pyspark save of serialized model is failing for windows. >

[jira] [Commented] (SPARK-31947) Solve string value error about Date/Timestamp in ScriptTransform

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177436#comment-17177436 ] Hyukjin Kwon commented on SPARK-31947: -- [~angerszhuuu] can you update which PR fixed this JIRA? >

[jira] [Assigned] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-12312: Assignee: Gabor Somogyi > JDBC connection to Kerberos secured databases fails on remote

[jira] [Commented] (SPARK-32598) Not able to see driver logs in spark history server in standalone mode

2020-08-13 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177429#comment-17177429 ] Lantao Jin commented on SPARK-32598: [~sriramgr] PullRequest is welcome. Please commit on the master

[jira] [Assigned] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32608: Assignee: (was: Apache Spark) > Script Transform DELIMIT value error >

[jira] [Assigned] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32608: Assignee: Apache Spark > Script Transform DELIMIT value error >

[jira] [Commented] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177420#comment-17177420 ] Apache Spark commented on SPARK-32608: -- User 'AngersZh' has created a pull request for this

[jira] [Comment Edited] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-08-13 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177401#comment-17177401 ] Dustin Smith edited comment on SPARK-32046 at 8/14/20, 12:51 AM: -

[jira] [Comment Edited] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-08-13 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177401#comment-17177401 ] Dustin Smith edited comment on SPARK-32046 at 8/14/20, 12:49 AM: -

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-08-13 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177401#comment-17177401 ] Dustin Smith commented on SPARK-32046: -- [~maropu] the definition of current timestamp is as

[jira] [Commented] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-08-13 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177398#comment-17177398 ] Takeshi Yamamuro commented on SPARK-32046: -- This issue depends on ZP or Jupyter? If so, I think

[jira] [Assigned] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32613: Assignee: Apache Spark > DecommissionWorkerSuite has started failing sporadically again

[jira] [Assigned] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32613: Assignee: (was: Apache Spark) > DecommissionWorkerSuite has started failing

[jira] [Commented] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177368#comment-17177368 ] Apache Spark commented on SPARK-32613: -- User 'agrawaldevesh' has created a pull request for this

[jira] [Created] (SPARK-32613) DecommissionWorkerSuite has started failing sporadically again

2020-08-13 Thread Devesh Agrawal (Jira)
Devesh Agrawal created SPARK-32613: -- Summary: DecommissionWorkerSuite has started failing sporadically again Key: SPARK-32613 URL: https://issues.apache.org/jira/browse/SPARK-32613 Project: Spark

[jira] [Updated] (SPARK-32611) Querying ORC table in Spark3 using spark.sql.orc.impl=hive produces incorrect when timestamp is present in predicate

2020-08-13 Thread Sumeet (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumeet updated SPARK-32611: --- Description: *How to reproduce this behavior?* * TZ="America/Los_Angeles" ./bin/spark-shell * sql("set

[jira] [Commented] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Zach Cahoone (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177315#comment-17177315 ] Zach Cahoone commented on SPARK-32604: -- The code here

[jira] [Created] (SPARK-32612) int columns produce inconsistent results on pandas UDFs

2020-08-13 Thread Robert Joseph Evans (Jira)
Robert Joseph Evans created SPARK-32612: --- Summary: int columns produce inconsistent results on pandas UDFs Key: SPARK-32612 URL: https://issues.apache.org/jira/browse/SPARK-32612 Project: Spark

[jira] [Commented] (SPARK-25557) ORC predicate pushdown for nested fields

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177301#comment-17177301 ] Apache Spark commented on SPARK-25557: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-25557) ORC predicate pushdown for nested fields

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177300#comment-17177300 ] Apache Spark commented on SPARK-25557: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-31776) Literal lit() supports lists and numpy arrays

2020-08-13 Thread Zirui Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177281#comment-17177281 ] Zirui Xu commented on SPARK-31776: -- Hi [~mengxr], is this ticket suggesting adding support for Python

[jira] [Created] (SPARK-32611) Querying ORC table in Spark3 using spark.sql.orc.impl=hive produces incorrect when timestamp is present in predicate

2020-08-13 Thread Sumeet (Jira)
Sumeet created SPARK-32611: -- Summary: Querying ORC table in Spark3 using spark.sql.orc.impl=hive produces incorrect when timestamp is present in predicate Key: SPARK-32611 URL:

[jira] [Comment Edited] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177246#comment-17177246 ] Rohit Mishra edited comment on SPARK-32609 at 8/13/20, 6:28 PM:

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32609: - Target Version/s: (was: 2.4.5, 2.4.7) > Incorrect exchange reuse with DataSourceV2 >

[jira] [Commented] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177246#comment-17177246 ] Rohit Mishra commented on SPARK-32609: -- [~mingjial], Please refrain from adding Priority as

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Rohit Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Mishra updated SPARK-32609: - Priority: Major (was: Critical) > Incorrect exchange reuse with DataSourceV2 >

[jira] [Commented] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Mingjia Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177231#comment-17177231 ] Mingjia Liu commented on SPARK-32609: - I am currently working on a fix &  unit test > Incorrect

[jira] [Resolved] (SPARK-31198) Use graceful decommissioning as part of dynamic scaling

2020-08-13 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau resolved SPARK-31198. -- Fix Version/s: 3.1.0 Resolution: Fixed > Use graceful decommissioning as part of

[jira] [Commented] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177204#comment-17177204 ] Apache Spark commented on SPARK-32610: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32610: Assignee: Kousuke Saruta (was: Apache Spark) > Fix the link to metrics.dropwizard.io in

[jira] [Assigned] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32610: Assignee: Apache Spark (was: Kousuke Saruta) > Fix the link to metrics.dropwizard.io in

[jira] [Commented] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177203#comment-17177203 ] Apache Spark commented on SPARK-32610: -- User 'sarutak' has created a pull request for this issue:

[jira] [Updated] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Mingjia Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingjia Liu updated SPARK-32609: Target Version/s: 2.4.5, 2.4.7 (was: 2.4.5) > Incorrect exchange reuse with DataSourceV2 >

[jira] [Commented] (SPARK-28367) Kafka connector infinite wait because metadata never updated

2020-08-13 Thread Ruslan Dautkhanov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177202#comment-17177202 ] Ruslan Dautkhanov commented on SPARK-28367: --- [~gsomogyi] thanks! yep would be great to learn

[jira] [Commented] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Mingjia Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177199#comment-17177199 ] Mingjia Liu commented on SPARK-32609: - Mitigation: Turn off  spark.sql.exchange.reuse. eg: 

[jira] [Created] (SPARK-32610) Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version

2020-08-13 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-32610: -- Summary: Fix the link to metrics.dropwizard.io in monitoring.md to refer the proper version Key: SPARK-32610 URL: https://issues.apache.org/jira/browse/SPARK-32610

[jira] [Created] (SPARK-32609) Incorrect exchange reuse with DataSourceV2

2020-08-13 Thread Mingjia Liu (Jira)
Mingjia Liu created SPARK-32609: --- Summary: Incorrect exchange reuse with DataSourceV2 Key: SPARK-32609 URL: https://issues.apache.org/jira/browse/SPARK-32609 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-32350) Add batch write support on LevelDB to improve performance of HybridStore

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177112#comment-17177112 ] Apache Spark commented on SPARK-32350: -- User 'baohe-zhang' has created a pull request for this

[jira] [Commented] (SPARK-32350) Add batch write support on LevelDB to improve performance of HybridStore

2020-08-13 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177110#comment-17177110 ] Apache Spark commented on SPARK-32350: -- User 'baohe-zhang' has created a pull request for this

[jira] [Updated] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32608: -- Description: For SQL   {code:java} SELECT TRANSFORM(a, b, c) ROW FORMAT DELIMITED FIELDS

[jira] [Created] (SPARK-32608) Script Transform DELIMIT value error

2020-08-13 Thread angerszhu (Jira)
angerszhu created SPARK-32608: - Summary: Script Transform DELIMIT value error Key: SPARK-32608 URL: https://issues.apache.org/jira/browse/SPARK-32608 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-32607) Script Transformation no-serde read line should respect `TOK_TABLEROWFORMATLINES`

2020-08-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32607: -- Description: reader.readLine() should respect `TOK_TABLEROWFORMATLINES` {code:java} protected def

[jira] [Updated] (SPARK-32607) Script Transformation no-serde read line should respect `TOK_TABLEROWFORMATLINES`

2020-08-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu updated SPARK-32607: -- Description: reader.readLine() should respect `` {code:java} protected def

[jira] [Created] (SPARK-32607) Script Transformation no-serde read line should respect `TOK_TABLEROWFORMATLINES`

2020-08-13 Thread angerszhu (Jira)
angerszhu created SPARK-32607: - Summary: Script Transformation no-serde read line should respect `TOK_TABLEROWFORMATLINES` Key: SPARK-32607 URL: https://issues.apache.org/jira/browse/SPARK-32607 Project:

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177084#comment-17177084 ] Kayal commented on SPARK-32053: ---

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177082#comment-17177082 ] Kayal commented on SPARK-32053: --- !image-2020-08-13-20-29-40-555.png!     > pyspark save of serialized

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: image-2020-08-13-20-29-40-555.png > pyspark save of serialized model is failing for windows. >

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: image-2020-08-13-20-28-28-779.png > pyspark save of serialized model is failing for windows. >

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: screenshot-1.png > pyspark save of serialized model is failing for windows. >

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177079#comment-17177079 ] Kayal commented on SPARK-32053: --- !image-2020-08-13-20-25-57-585.png!   > pyspark save of serialized

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: image-2020-08-13-20-25-57-585.png > pyspark save of serialized model is failing for windows. >

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Attachment: image-2020-08-13-20-24-57-309.png > pyspark save of serialized model is failing for windows. >

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177077#comment-17177077 ] Kayal commented on SPARK-32053: --- !image-2020-08-13-20-24-57-309.png! > pyspark save of serialized model

[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177073#comment-17177073 ] Kayal commented on SPARK-32053: --- The code to reproduce the issue on windows jupyter notebook: import

[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal updated SPARK-32053: -- Priority: Blocker (was: Major) > pyspark save of serialized model is failing for windows. >

[jira] [Reopened] (SPARK-32053) pyspark save of serialized model is failing for windows.

2020-08-13 Thread Kayal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kayal reopened SPARK-32053: --- Hi, I have verified the issue in spark latest version 3.0.0 , the issue seems to be still there on windows.  

[jira] [Commented] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176996#comment-17176996 ] Hyukjin Kwon commented on SPARK-32587: -- Could you share the reproducible codes and the actual

[jira] [Commented] (SPARK-32591) Add better api docs for stage level scheduling Resources

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176994#comment-17176994 ] Hyukjin Kwon commented on SPARK-32591: -- +100!! > Add better api docs for stage level scheduling

[jira] [Commented] (SPARK-32601) Issue in converting an RDD of Arrow RecordBatches in v3.0.0

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176993#comment-17176993 ] Hyukjin Kwon commented on SPARK-32601: -- {{ArrowSerializer}} isn't supposed to be an API. Can this

[jira] [Commented] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176992#comment-17176992 ] Hyukjin Kwon commented on SPARK-32604: -- Are you interested in submitting a PR to fix? > Bug in

[jira] [Updated] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32604: - Component/s: PySpark > Bug in ALSModel Python Documentation >

[jira] [Updated] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32604: - Target Version/s: (was: 3.0.0) > Bug in ALSModel Python Documentation >

[jira] [Commented] (SPARK-32604) Bug in ALSModel Python Documentation

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176990#comment-17176990 ] Hyukjin Kwon commented on SPARK-32604: -- Please avoid setting target version which is reserved for

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

2020-08-13 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176966#comment-17176966 ] JinxinTang commented on SPARK-32500: Kindly ping [~hyukjin.kwon] It's my pleasure :  ) > Query and

[jira] [Commented] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-08-13 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176955#comment-17176955 ] Wenchen Fan commented on SPARK-32018: - [~Gengliang.Wang] we should create a new JIRA ticket for the

[jira] [Assigned] (SPARK-32244) Build and run the Spark with test cases in Github Actions

2020-08-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32244: Assignee: Hyukjin Kwon > Build and run the Spark with test cases in Github Actions >

[jira] [Created] (SPARK-32606) Remove the fork of action-download-artifact in test_report.yml

2020-08-13 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32606: Summary: Remove the fork of action-download-artifact in test_report.yml Key: SPARK-32606 URL: https://issues.apache.org/jira/browse/SPARK-32606 Project: Spark

[jira] [Created] (SPARK-32605) Remove the fork of action-surefire-report in test_report.yml

2020-08-13 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-32605: Summary: Remove the fork of action-surefire-report in test_report.yml Key: SPARK-32605 URL: https://issues.apache.org/jira/browse/SPARK-32605 Project: Spark

[jira] [Commented] (SPARK-32018) Fix UnsafeRow set overflowed decimal

2020-08-13 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176906#comment-17176906 ] Prashant Sharma commented on SPARK-32018: - This issue is resolved as fixed with version 2.4.7.

[jira] [Updated] (SPARK-32556) Fix release script to uri encode the user provided passwords.

2020-08-13 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-32556: Fix Version/s: 2.4.7 > Fix release script to uri encode the user provided passwords. >

[jira] [Resolved] (SPARK-31947) Solve string value error about Date/Timestamp in ScriptTransform

2020-08-13 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-31947. --- Resolution: Fixed > Solve string value error about Date/Timestamp in ScriptTransform >

  1   2   >