[jira] [Commented] (SPARK-28990) SparkSQL invalid call to toAttribute on unresolved object, tree: *

[ https://issues.apache.org/jira/browse/SPARK-28990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002698#comment-17002698 ] Wenchao Wu commented on SPARK-28990: [~lucusguo] [~xiaozhang] me too > SparkSQL invalid call to

[jira] [Commented] (SPARK-28990) SparkSQL invalid call to toAttribute on unresolved object, tree: *

[ https://issues.apache.org/jira/browse/SPARK-28990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002696#comment-17002696 ] Xiao Zhang commented on SPARK-28990: [~fengchaoge] me too > SparkSQL invalid call to toAttribute on

[jira] [Commented] (SPARK-28990) SparkSQL invalid call to toAttribute on unresolved object, tree: *

[ https://issues.apache.org/jira/browse/SPARK-28990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002695#comment-17002695 ] lucusguo commented on SPARK-28990: -- but, I cannot reproduce  it in spark2.4.3 > SparkSQL invalid call

[jira] [Created] (SPARK-30342) Update LIST JAR/FILE command

Rakesh Raushan created SPARK-30342: -- Summary: Update LIST JAR/FILE command Key: SPARK-30342 URL: https://issues.apache.org/jira/browse/SPARK-30342 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-30333) Bump jackson-databind to 2.6.7.3

[ https://issues.apache.org/jira/browse/SPARK-30333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-30333. -- Fix Version/s: 2.4.5 Assignee: Sandeep Katta Resolution: Fixed

[jira] [Created] (SPARK-30341) check overflow for interval arithmetic operations

Kent Yao created SPARK-30341: Summary: check overflow for interval arithmetic operations Key: SPARK-30341 URL: https://issues.apache.org/jira/browse/SPARK-30341 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-30340) Python tests failed on arm64/x86

[ https://issues.apache.org/jira/browse/SPARK-30340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtianhua updated SPARK-30340: - Summary: Python tests failed on arm64/x86 (was: Python tests failed on arm64 ) > Python tests

[jira] [Updated] (SPARK-30340) Python tests failed on arm64

[ https://issues.apache.org/jira/browse/SPARK-30340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtianhua updated SPARK-30340: - Description: Jenkins job spark-master-test-python-arm failed after the commit 

[jira] [Commented] (SPARK-30328) Fail to write local files with RDD.saveTextFile when setting the incorrect Hadoop configuration files

[ https://issues.apache.org/jira/browse/SPARK-30328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002613#comment-17002613 ] Ankit Raj Boudh commented on SPARK-30328: - Thank you [~tobe], i will analyse this issue and will

[jira] [Updated] (SPARK-30328) Fail to write local files with RDD.saveTextFile when setting the incorrect Hadoop configuration files

[ https://issues.apache.org/jira/browse/SPARK-30328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chendihao updated SPARK-30328: -- Description: We find that the incorrect Hadoop configuration files cause the failure of saving RDD

[jira] [Commented] (SPARK-30328) Fail to write local files with RDD.saveTextFile when setting the incorrect Hadoop configuration files

[ https://issues.apache.org/jira/browse/SPARK-30328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002609#comment-17002609 ] chendihao commented on SPARK-30328: --- Of course and thanks [~Ankitraj] . We don't have time to dig into

[jira] [Comment Edited] (SPARK-30328) Fail to write local files with RDD.saveTextFile when setting the incorrect Hadoop configuration files

[ https://issues.apache.org/jira/browse/SPARK-30328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002609#comment-17002609 ] chendihao edited comment on SPARK-30328 at 12/24/19 2:57 AM: - Of course and

[jira] [Updated] (SPARK-30340) Python tests failed on arm64

[ https://issues.apache.org/jira/browse/SPARK-30340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtianhua updated SPARK-30340: - Description: Jenkins job spark-master-test-python-arm failed after the commit 

[jira] [Created] (SPARK-30340) Python tests failed on arm64

huangtianhua created SPARK-30340: Summary: Python tests failed on arm64 Key: SPARK-30340 URL: https://issues.apache.org/jira/browse/SPARK-30340 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-30339) Avoid to fail twice in function lookup

[ https://issues.apache.org/jira/browse/SPARK-30339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-30339: - Description: Currently if function lookup fails, spark will give it a second change by casting

[jira] [Created] (SPARK-30339) Avoid to fail twice in function lookup

Zhenhua Wang created SPARK-30339: Summary: Avoid to fail twice in function lookup Key: SPARK-30339 URL: https://issues.apache.org/jira/browse/SPARK-30339 Project: Spark Issue Type:

[jira] [Commented] (SPARK-30328) Fail to write local files with RDD.saveTextFile when setting the incorrect Hadoop configuration files

[ https://issues.apache.org/jira/browse/SPARK-30328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002585#comment-17002585 ] Ankit Raj Boudh commented on SPARK-30328: - @chendihao, can i check this issue ? > Fail to write

[jira] [Created] (SPARK-30338) Avoid unnecessary InternalRow copies in ParquetRowConverter

Josh Rosen created SPARK-30338: -- Summary: Avoid unnecessary InternalRow copies in ParquetRowConverter Key: SPARK-30338 URL: https://issues.apache.org/jira/browse/SPARK-30338 Project: Spark

[jira] [Commented] (SPARK-25603) Generalize Nested Column Pruning

[ https://issues.apache.org/jira/browse/SPARK-25603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002577#comment-17002577 ] Takeshi Yamamuro commented on SPARK-25603: -- Still WIP? Since we've finished implementing the

[jira] [Created] (SPARK-30337) Convert case class with var to normal class in spark-sql-kafka module

Jungtaek Lim created SPARK-30337: Summary: Convert case class with var to normal class in spark-sql-kafka module Key: SPARK-30337 URL: https://issues.apache.org/jira/browse/SPARK-30337 Project: Spark

[jira] [Created] (SPARK-30336) Move Kafka consumer related classes to its own package

Jungtaek Lim created SPARK-30336: Summary: Move Kafka consumer related classes to its own package Key: SPARK-30336 URL: https://issues.apache.org/jira/browse/SPARK-30336 Project: Spark Issue

[jira] [Resolved] (SPARK-30120) LSH approxNearestNeighbors should use BoundedPriorityQueue when numNearestNeighbors is small

[ https://issues.apache.org/jira/browse/SPARK-30120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-30120. -- Resolution: Not A Problem > LSH approxNearestNeighbors should use BoundedPriorityQueue when

[jira] [Commented] (SPARK-29245) CCE during creating HiveMetaStoreClient

[ https://issues.apache.org/jira/browse/SPARK-29245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002539#comment-17002539 ] Xiao Li commented on SPARK-29245: - Since JDK support is experimental, it is not a blocker of Spark 3.0.

[jira] [Updated] (SPARK-29245) CCE during creating HiveMetaStoreClient

[ https://issues.apache.org/jira/browse/SPARK-29245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-29245: Priority: Major (was: Blocker) > CCE during creating HiveMetaStoreClient >

[jira] [Commented] (SPARK-30316) data size boom after shuffle writing dataframe save as parquet

[ https://issues.apache.org/jira/browse/SPARK-30316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002529#comment-17002529 ] Xiao Li commented on SPARK-30316: - The compression ratio depends on your data layout, instead of number

[jira] [Updated] (SPARK-30316) data size boom after shuffle writing dataframe save as parquet

[ https://issues.apache.org/jira/browse/SPARK-30316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30316: Priority: Major (was: Blocker) > data size boom after shuffle writing dataframe save as parquet >

[jira] [Resolved] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-21869. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-21869) A cached Kafka producer should not be closed if any task is using it.

[ https://issues.apache.org/jira/browse/SPARK-21869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-21869: -- Assignee: Jungtaek Lim (was: Gabor Somogyi) > A cached Kafka

[jira] [Created] (SPARK-30335) Clarify behavior of FIRST and LAST without OVER caluse.

xqods9o5ekm3 created SPARK-30335: Summary: Clarify behavior of FIRST and LAST without OVER caluse. Key: SPARK-30335 URL: https://issues.apache.org/jira/browse/SPARK-30335 Project: Spark

[jira] [Commented] (SPARK-27838) Support user provided non-nullable avro schema for nullable catalyst schema without any null record

[ https://issues.apache.org/jira/browse/SPARK-27838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002440#comment-17002440 ] Frank Lee commented on SPARK-27838: --- Hello Is there a workaround for this before this is released?

[jira] [Commented] (SPARK-29224) Implement Factorization Machines as a ml-pipeline component

[ https://issues.apache.org/jira/browse/SPARK-29224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002402#comment-17002402 ] Ruslan Dautkhanov commented on SPARK-29224: --- E.g. would this work with 0.1m or 1m sparse

[jira] [Assigned] (SPARK-27762) Support user provided avro schema for writing fields with different ordering

[ https://issues.apache.org/jira/browse/SPARK-27762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-27762: --- Assignee: DB Tsai > Support user provided avro schema for writing fields with different ordering >

[jira] [Commented] (SPARK-29224) Implement Factorization Machines as a ml-pipeline component

[ https://issues.apache.org/jira/browse/SPARK-29224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002398#comment-17002398 ] Ruslan Dautkhanov commented on SPARK-29224: --- That's great. Out of curiosity - what's largest

[jira] [Updated] (SPARK-30334) Add metadata around semi-structured columns to Spark

[ https://issues.apache.org/jira/browse/SPARK-30334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-30334: Description: Semi-structured data is used widely in the data industry for reporting events in a

[jira] [Created] (SPARK-30334) Add metadata around semi-structured columns to Spark

Burak Yavuz created SPARK-30334: --- Summary: Add metadata around semi-structured columns to Spark Key: SPARK-30334 URL: https://issues.apache.org/jira/browse/SPARK-30334 Project: Spark Issue

[jira] [Commented] (SPARK-26663) Cannot query a Hive table with subdirectories

[ https://issues.apache.org/jira/browse/SPARK-26663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002367#comment-17002367 ] Xiaoguang Wang commented on SPARK-26663: I meet the same problem here.   How to debug? >

[jira] [Resolved] (SPARK-29224) Implement Factorization Machines as a ml-pipeline component

[ https://issues.apache.org/jira/browse/SPARK-29224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29224. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26124

[jira] [Assigned] (SPARK-29224) Implement Factorization Machines as a ml-pipeline component

[ https://issues.apache.org/jira/browse/SPARK-29224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29224: Assignee: mob-ai > Implement Factorization Machines as a ml-pipeline component >

[jira] [Created] (SPARK-30333) Bump jackson-databind to 2.6.7.3

Sandeep Katta created SPARK-30333: - Summary: Bump jackson-databind to 2.6.7.3 Key: SPARK-30333 URL: https://issues.apache.org/jira/browse/SPARK-30333 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-30332) When running sql query with limit catalyst throw StackOverFlow exception

Izek Greenfield created SPARK-30332: --- Summary: When running sql query with limit catalyst throw StackOverFlow exception Key: SPARK-30332 URL: https://issues.apache.org/jira/browse/SPARK-30332

[jira] [Created] (SPARK-30331) The final AdaptiveSparkPlan event is not marked with `isFinalPlan=true`

Manu Zhang created SPARK-30331: -- Summary: The final AdaptiveSparkPlan event is not marked with `isFinalPlan=true` Key: SPARK-30331 URL: https://issues.apache.org/jira/browse/SPARK-30331 Project: Spark

[jira] [Assigned] (SPARK-28332) SQLMetric wrong initValue

[ https://issues.apache.org/jira/browse/SPARK-28332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28332: --- Assignee: EdisonWang > SQLMetric wrong initValue > -- > >

[jira] [Commented] (SPARK-28332) SQLMetric wrong initValue

[ https://issues.apache.org/jira/browse/SPARK-28332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002148#comment-17002148 ] EdisonWang commented on SPARK-28332: I've taken it [~cloud_fan] > SQLMetric wrong initValue >

[jira] [Updated] (SPARK-26002) SQL date operators calculates with incorrect dayOfYears for dates before 1500-03-01

[ https://issues.apache.org/jira/browse/SPARK-26002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26002: Labels: correctness (was: ) > SQL date operators calculates with incorrect dayOfYears for dates before

[jira] [Updated] (SPARK-30330) Support single quotes json parsing for get_json_object and json_tuple

[ https://issues.apache.org/jira/browse/SPARK-30330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fang Wen updated SPARK-30330: - External issue URL: https://github.com/apache/spark/pull/26965 > Support single quotes json parsing for

[jira] [Updated] (SPARK-30330) Support single quotes json parsing for get_json_object and json_tuple

[ https://issues.apache.org/jira/browse/SPARK-30330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fang Wen updated SPARK-30330: - External issue URL: (was: https://github.com/apache/spark/pull/26965) > Support single quotes json

[jira] [Updated] (SPARK-30330) Support single quotes json parsing for get_json_object and json_tuple

[ https://issues.apache.org/jira/browse/SPARK-30330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fang Wen updated SPARK-30330: - Labels: release-notes (was: ) > Support single quotes json parsing for get_json_object and json_tuple

[jira] [Created] (SPARK-30330) Support single quotes json parsing for get_json_object and json_tuple

Fang Wen created SPARK-30330: Summary: Support single quotes json parsing for get_json_object and json_tuple Key: SPARK-30330 URL: https://issues.apache.org/jira/browse/SPARK-30330 Project: Spark

[jira] [Updated] (SPARK-30328) Fail to write local files with RDD.saveTextFile when setting the incorrect Hadoop configuration files

[ https://issues.apache.org/jira/browse/SPARK-30328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chendihao updated SPARK-30328: -- Description: We find that the incorrect Hadoop configuration files cause the failure of saving RDD