[jira] [Updated] (SPARK-28880) ANSI SQL: Nested bracketed comments

2020-11-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28880: - Fix Version/s: 3.1.0 > ANSI SQL: Nested bracketed comments >

[jira] [Commented] (SPARK-33441) Add "unused-import" compile arg to scalac and remove all unused imports in Scala code

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235929#comment-17235929 ] Apache Spark commented on SPARK-33441: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-33441) Add "unused-import" compile arg to scalac and remove all unused imports in Scala code

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235927#comment-17235927 ] Apache Spark commented on SPARK-33441: -- User 'LuciferYang' has created a pull request for this

[jira] [Created] (SPARK-33498) Datetime parsing should fail if the input string can't be parsed, or he pattern string is invalid

2020-11-19 Thread Leanken.Lin (Jira)
Leanken.Lin created SPARK-33498: --- Summary: Datetime parsing should fail if the input string can't be parsed, or he pattern string is invalid Key: SPARK-33498 URL: https://issues.apache.org/jira/browse/SPARK-33498

[jira] [Commented] (SPARK-33496) Improve error message of ANSI explicit cast

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235896#comment-17235896 ] Apache Spark commented on SPARK-33496: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-33496) Improve error message of ANSI explicit cast

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235897#comment-17235897 ] Apache Spark commented on SPARK-33496: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-33496) Improve error message of ANSI explicit cast

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33496: Assignee: Apache Spark (was: Gengliang Wang) > Improve error message of ANSI explicit

[jira] [Assigned] (SPARK-33496) Improve error message of ANSI explicit cast

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33496: Assignee: Gengliang Wang (was: Apache Spark) > Improve error message of ANSI explicit

[jira] [Commented] (SPARK-33223) Expose state information on SS UI

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235887#comment-17235887 ] Apache Spark commented on SPARK-33223: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Commented] (SPARK-33223) Expose state information on SS UI

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235888#comment-17235888 ] Apache Spark commented on SPARK-33223: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Updated] (SPARK-33497) Override maxRows in some LogicalPlan

2020-11-19 Thread ulysses you (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ulysses you updated SPARK-33497: Summary: Override maxRows in some LogicalPlan (was: Override max row in some LogicalPlan) >

[jira] [Created] (SPARK-33497) Override max row in some LogicalPlan

2020-11-19 Thread ulysses you (Jira)
ulysses you created SPARK-33497: --- Summary: Override max row in some LogicalPlan Key: SPARK-33497 URL: https://issues.apache.org/jira/browse/SPARK-33497 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-33496) Improve error message of ANSI explicit cast

2020-11-19 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-33496: -- Summary: Improve error message of ANSI explicit cast Key: SPARK-33496 URL: https://issues.apache.org/jira/browse/SPARK-33496 Project: Spark Issue Type:

[jira] [Commented] (SPARK-33472) IllegalArgumentException when applying RemoveRedundantSorts before EnsureRequirements

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235870#comment-17235870 ] Apache Spark commented on SPARK-33472: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-33472) IllegalArgumentException when applying RemoveRedundantSorts before EnsureRequirements

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235869#comment-17235869 ] Apache Spark commented on SPARK-33472: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-33472) IllegalArgumentException when applying RemoveRedundantSorts before EnsureRequirements

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235868#comment-17235868 ] Apache Spark commented on SPARK-33472: -- User 'allisonwang-db' has created a pull request for this

[jira] [Created] (SPARK-33495) jcl-over-slf4j conflicts with commons-logging.jar

2020-11-19 Thread lrz (Jira)
lrz created SPARK-33495: --- Summary: jcl-over-slf4j conflicts with commons-logging.jar Key: SPARK-33495 URL: https://issues.apache.org/jira/browse/SPARK-33495 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-31018) Deprecate support of multiple workers on the same host in Standalone

2020-11-19 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-31018. -- Resolution: Fixed resolved by https://github.com/apache/spark/pull/27768 > Deprecate support of multiple

[jira] [Commented] (SPARK-32185) User Guide - Monitoring

2020-11-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235806#comment-17235806 ] Hyukjin Kwon commented on SPARK-32185: -- Hey [~a7prasad], have you had a chance to take a look? The

[jira] [Commented] (SPARK-33486) Collapse Partial and Final Aggregation into Complete Aggregation mode

2020-11-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235782#comment-17235782 ] Dongjoon Hyun commented on SPARK-33486: --- Hi, [~prakharjain09]. Could you use `Improvement` or

[jira] [Resolved] (SPARK-33493) Scala 2.13 build with SBT failure

2020-11-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33493. --- Resolution: Duplicate This is resolved as a follow-up. > Scala 2.13 build with SBT failure

[jira] [Updated] (SPARK-33493) Scala 2.13 build with SBT failure

2020-11-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33493: -- Affects Version/s: (was: 3.0.1) 3.1.0 > Scala 2.13 build with SBT

[jira] [Commented] (SPARK-26645) CSV infer schema bug infers decimal(9,-1)

2020-11-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235757#comment-17235757 ] Dongjoon Hyun commented on SPARK-26645: --- Do you need this, [~bullsoverbears]? > CSV infer schema

[jira] [Commented] (SPARK-33445) Can't parse decimal type from csv file

2020-11-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235755#comment-17235755 ] Dongjoon Hyun commented on SPARK-33445: --- Thank you for updating, [~hyukjin.kwon]. > Can't parse

[jira] [Commented] (SPARK-32916) Add support for external shuffle service in YARN deployment mode to leverage push-based shuffle

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235726#comment-17235726 ] Apache Spark commented on SPARK-32916: -- User 'otterc' has created a pull request for this issue:

[jira] [Commented] (SPARK-32916) Add support for external shuffle service in YARN deployment mode to leverage push-based shuffle

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235728#comment-17235728 ] Apache Spark commented on SPARK-32916: -- User 'otterc' has created a pull request for this issue:

[jira] [Updated] (SPARK-33463) Spark Thrift Server, keep Job Id when using incremental collect

2020-11-19 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-33463: - Fix Version/s: (was: 3.1.0) Target Version/s: (was: 3.1.0) > Spark Thrift Server,

[jira] [Assigned] (SPARK-33494) Do not use local shuffle reader for repartition

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33494: Assignee: Apache Spark (was: Wenchen Fan) > Do not use local shuffle reader for

[jira] [Commented] (SPARK-33494) Do not use local shuffle reader for repartition

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235685#comment-17235685 ] Apache Spark commented on SPARK-33494: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33494) Do not use local shuffle reader for repartition

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33494: Assignee: Wenchen Fan (was: Apache Spark) > Do not use local shuffle reader for

[jira] [Updated] (SPARK-33494) Do not use local shuffle reader for repartition

2020-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-33494: Summary: Do not use local shuffle reader for repartition (was: Do not use local shuffle reader

[jira] [Created] (SPARK-33494) Do not use local shuffle reader for user-given repartition

2020-11-19 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-33494: --- Summary: Do not use local shuffle reader for user-given repartition Key: SPARK-33494 URL: https://issues.apache.org/jira/browse/SPARK-33494 Project: Spark

[jira] [Updated] (SPARK-33482) V2 Datasources that extend FileScan preclude exchange reuse

2020-11-19 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-33482: -- Description: Sample query: {noformat}

[jira] [Commented] (SPARK-33045) Implement built-in LIKE ANY and LIKE ALL UDF

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235632#comment-17235632 ] Apache Spark commented on SPARK-33045: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-33400) Normalize sameOrderExpressions in SortOrder

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235633#comment-17235633 ] Apache Spark commented on SPARK-33400: -- User 'prakharjain09' has created a pull request for this

[jira] [Updated] (SPARK-33493) Scala 2.13 build with SBT failure

2020-11-19 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-33493: - Component/s: SQL > Scala 2.13 build with SBT failure > - > >

[jira] [Created] (SPARK-33493) Scala 2.13 build with SBT failure

2020-11-19 Thread Chao Sun (Jira)
Chao Sun created SPARK-33493: Summary: Scala 2.13 build with SBT failure Key: SPARK-33493 URL: https://issues.apache.org/jira/browse/SPARK-33493 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-33492) DSv2: Append/Overwrite/ReplaceTable should invalidate cache

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235611#comment-17235611 ] Apache Spark commented on SPARK-33492: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33492) DSv2: Append/Overwrite/ReplaceTable should invalidate cache

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33492: Assignee: (was: Apache Spark) > DSv2: Append/Overwrite/ReplaceTable should

[jira] [Assigned] (SPARK-33492) DSv2: Append/Overwrite/ReplaceTable should invalidate cache

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33492: Assignee: Apache Spark > DSv2: Append/Overwrite/ReplaceTable should invalidate cache >

[jira] [Resolved] (SPARK-33045) Implement built-in LIKE ANY and LIKE ALL UDF

2020-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33045. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 2

[jira] [Created] (SPARK-33492) DSv2: Append/Overwrite/ReplaceTable should invalidate cache

2020-11-19 Thread Chao Sun (Jira)
Chao Sun created SPARK-33492: Summary: DSv2: Append/Overwrite/ReplaceTable should invalidate cache Key: SPARK-33492 URL: https://issues.apache.org/jira/browse/SPARK-33492 Project: Spark Issue

[jira] [Assigned] (SPARK-33045) Implement built-in LIKE ANY and LIKE ALL UDF

2020-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33045: --- Assignee: jiaan.geng > Implement built-in LIKE ANY and LIKE ALL UDF >

[jira] [Created] (SPARK-33491) Update Structured Streaming UI documentation page

2020-11-19 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-33491: - Summary: Update Structured Streaming UI documentation page Key: SPARK-33491 URL: https://issues.apache.org/jira/browse/SPARK-33491 Project: Spark Issue

[jira] [Assigned] (SPARK-28704) Test backward compatibility on JDK9+ once we have a version supports JDK9+

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28704: Assignee: Apache Spark > Test backward compatibility on JDK9+ once we have a version

[jira] [Assigned] (SPARK-28704) Test backward compatibility on JDK9+ once we have a version supports JDK9+

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28704: Assignee: (was: Apache Spark) > Test backward compatibility on JDK9+ once we have a

[jira] [Commented] (SPARK-28704) Test backward compatibility on JDK9+ once we have a version supports JDK9+

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235514#comment-17235514 ] Apache Spark commented on SPARK-28704: -- User 'AngersZh' has created a pull request for this

[jira] [Updated] (SPARK-33490) calling system.exit(-1) from a future does not result in the driver reflecting the fact that the spark app failed.

2020-11-19 Thread Srimantha Rajapaksa (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srimantha Rajapaksa updated SPARK-33490: Summary: calling system.exit(-1) from a future does not result in the driver

[jira] [Created] (SPARK-33490) calling system.exit(-1) from a future does not result in the driver reflecting the fact that spark app failed.

2020-11-19 Thread Srimantha Rajapaksa (Jira)
Srimantha Rajapaksa created SPARK-33490: --- Summary: calling system.exit(-1) from a future does not result in the driver reflecting the fact that spark app failed. Key: SPARK-33490 URL:

[jira] [Assigned] (SPARK-33442) Change Combine Limit to Eliminate limit using max row

2020-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33442: --- Assignee: ulysses you > Change Combine Limit to Eliminate limit using max row >

[jira] [Resolved] (SPARK-33442) Change Combine Limit to Eliminate limit using max row

2020-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33442. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30368

[jira] [Commented] (SPARK-33348) Use scala.jdk.CollectionConverters replace scala.collection.JavaConverters

2020-11-19 Thread Darcy Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235462#comment-17235462 ] Darcy Shen commented on SPARK-33348: Would better keep it until Apache Spark switches to Scala 2.13

[jira] [Updated] (SPARK-27421) RuntimeException when querying a view on a partitioned parquet table

2020-11-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27421: - Fix Version/s: 2.4.8 > RuntimeException when querying a view on a partitioned parquet table >

[jira] [Assigned] (SPARK-33224) Expose watermark information on SS UI

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33224: Assignee: (was: Apache Spark) > Expose watermark information on SS UI >

[jira] [Commented] (SPARK-33224) Expose watermark information on SS UI

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235448#comment-17235448 ] Apache Spark commented on SPARK-33224: -- User 'HeartSaVioR' has created a pull request for this

[jira] [Assigned] (SPARK-33224) Expose watermark information on SS UI

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33224: Assignee: Apache Spark > Expose watermark information on SS UI >

[jira] [Updated] (SPARK-28103) Cannot infer filters from union table with empty local relation table properly

2020-11-19 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-28103: - Affects Version/s: 3.1.0 3.0.2 2.4.8 >

[jira] [Updated] (SPARK-28103) Cannot infer filters from union table with empty local relation table properly

2020-11-19 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-28103: - Issue Type: Improvement (was: Bug) > Cannot infer filters from union table with empty

[jira] [Commented] (SPARK-28103) Cannot infer filters from union table with empty local relation table properly

2020-11-19 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235428#comment-17235428 ] Takeshi Yamamuro commented on SPARK-28103: -- It seem this is related to

[jira] [Updated] (SPARK-33489) Support null for conversion from and to Arrow type

2020-11-19 Thread Yuya Kanai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuya Kanai updated SPARK-33489: --- Description: I got below error when using from_arrow_type() in pyspark.sql.pandas.types

[jira] [Updated] (SPARK-33488) Re SPARK-21820. Creating Spark dataframe with carriage return/line feed leaves cr in multiline

2020-11-19 Thread GREGORY WERNER (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GREGORY WERNER updated SPARK-33488: --- Description: In SPARK-21820 I see what seems to be the same issue reported, but marked as

[jira] [Updated] (SPARK-33488) Re SPARK-21820. Creating Spark dataframe with carriage return/line feed leaves cr in multiline

2020-11-19 Thread GREGORY WERNER (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GREGORY WERNER updated SPARK-33488: --- Description: In SPARK-21820 I see what seems to be the same issue reported, but marked as

[jira] [Updated] (SPARK-33489) Support null for conversion from and to Arrow type

2020-11-19 Thread Yuya Kanai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuya Kanai updated SPARK-33489: --- Issue Type: Improvement (was: Bug) > Support null for conversion from and to Arrow type >

[jira] [Updated] (SPARK-33488) Re SPARK-21820. Creating Spark dataframe with carriage return/line feed leaves cr in multiline

2020-11-19 Thread GREGORY WERNER (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GREGORY WERNER updated SPARK-33488: --- Description: In SPARK-21820 I see what seems to be the same issue reported, but marked as

[jira] [Updated] (SPARK-33488) Re SPARK-21820. Creating Spark dataframe with carriage return/line feed leaves cr in multiline

2020-11-19 Thread GREGORY WERNER (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GREGORY WERNER updated SPARK-33488: --- Description: In SPARK-21820 I see what seems to be the same issue reported, but mark as

[jira] [Created] (SPARK-33489) Support null for conversion from and to Arrow type

2020-11-19 Thread Yuya Kanai (Jira)
Yuya Kanai created SPARK-33489: -- Summary: Support null for conversion from and to Arrow type Key: SPARK-33489 URL: https://issues.apache.org/jira/browse/SPARK-33489 Project: Spark Issue Type:

[jira] [Updated] (SPARK-33488) Re SPARK-21820. Creating Spark dataframe with carriage return/line feed leaves cr in multiline

2020-11-19 Thread GREGORY WERNER (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GREGORY WERNER updated SPARK-33488: --- Description: In [SPARK-21820] I see what seems to be the same issue reported, but resolved. 

[jira] [Created] (SPARK-33488) Re SPARK-21820. Creating Spark dataframe with carriage return/line feed leaves cr in multiline

2020-11-19 Thread GREGORY WERNER (Jira)
GREGORY WERNER created SPARK-33488: -- Summary: Re SPARK-21820. Creating Spark dataframe with carriage return/line feed leaves cr in multiline Key: SPARK-33488 URL:

[jira] [Updated] (SPARK-33483) Fix rat exclusion patterns and add a LICENSE

2020-11-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-33483: - Fix Version/s: 3.0.2 2.4.8 > Fix rat exclusion patterns and add a LICENSE >

[jira] [Updated] (SPARK-33487) Let ML ALS recommend for BOTH subsets - users and items

2020-11-19 Thread Rose Aysina (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rose Aysina updated SPARK-33487: Description: Currently ALS in Spark ML supports next methods for getting recommendations: *

[jira] [Updated] (SPARK-33487) Let ML ALS recommend for BOTH subsets - users and items

2020-11-19 Thread Rose Aysina (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rose Aysina updated SPARK-33487: Summary: Let ML ALS recommend for BOTH subsets - users and items (was: Let ML ALS recommend for

[jira] [Created] (SPARK-33487) Let ML ALS recommend for BOTH subsets - users nd items

2020-11-19 Thread Rose Aysina (Jira)
Rose Aysina created SPARK-33487: --- Summary: Let ML ALS recommend for BOTH subsets - users nd items Key: SPARK-33487 URL: https://issues.apache.org/jira/browse/SPARK-33487 Project: Spark Issue

[jira] [Commented] (SPARK-33486) Collapse Partial and Final Aggregation into Complete Aggregation mode

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235292#comment-17235292 ] Apache Spark commented on SPARK-33486: -- User 'prakharjain09' has created a pull request for this

[jira] [Assigned] (SPARK-33486) Collapse Partial and Final Aggregation into Complete Aggregation mode

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33486: Assignee: (was: Apache Spark) > Collapse Partial and Final Aggregation into Complete

[jira] [Commented] (SPARK-33486) Collapse Partial and Final Aggregation into Complete Aggregation mode

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235290#comment-17235290 ] Apache Spark commented on SPARK-33486: -- User 'prakharjain09' has created a pull request for this

[jira] [Assigned] (SPARK-33486) Collapse Partial and Final Aggregation into Complete Aggregation mode

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33486: Assignee: Apache Spark > Collapse Partial and Final Aggregation into Complete

[jira] [Created] (SPARK-33486) Collapse Partial and Final Aggregation into Complete Aggregation mode

2020-11-19 Thread Prakhar Jain (Jira)
Prakhar Jain created SPARK-33486: Summary: Collapse Partial and Final Aggregation into Complete Aggregation mode Key: SPARK-33486 URL: https://issues.apache.org/jira/browse/SPARK-33486 Project: Spark

[jira] [Commented] (SPARK-33483) Fix rat exclusion patterns and add a LICENSE

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235255#comment-17235255 ] Apache Spark commented on SPARK-33483: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-33483) Fix rat exclusion patterns and add a LICENSE

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235252#comment-17235252 ] Apache Spark commented on SPARK-33483: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-33483) Fix rat exclusion patterns and add a LICENSE

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235250#comment-17235250 ] Apache Spark commented on SPARK-33483: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-33483) Fix rat exclusion patterns and add a LICENSE

2020-11-19 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235249#comment-17235249 ] Apache Spark commented on SPARK-33483: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-33485) running spark application in kerbernetes,bug the application log shows yarn authentications

2020-11-19 Thread Yuan Jiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Jiao updated SPARK-33485: -- Description: My spark application accessing kerberized HDFS is running in kubernetes cluster, but

[jira] [Updated] (SPARK-33485) running spark application in kerbernetes,bug the application log shows yarn authentications

2020-11-19 Thread Yuan Jiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Jiao updated SPARK-33485: -- Description: My spark application accessing kerberized HDFS is running in kubernetes cluster, but

[jira] [Resolved] (SPARK-33483) Fix rat exclusion patterns and add a LICENSE

2020-11-19 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33483. --- Fix Version/s: 3.1.0 Assignee: Dongjoon Hyun Resolution: Fixed This is