[jira] [Assigned] (SPARK-26747) Makes GetMapValue nullability more precise

[ https://issues.apache.org/jira/browse/SPARK-26747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26747: Assignee: Apache Spark > Makes GetMapValue nullability more precise > ---

[jira] [Commented] (SPARK-26741) Analyzer incorrectly resolves aggregate function outside of Aggregate operators

[ https://issues.apache.org/jira/browse/SPARK-26741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753748#comment-16753748 ] Xiao Li commented on SPARK-26741: - Could we re-enable the test by making the test case v

[jira] [Assigned] (SPARK-26747) Makes GetMapValue nullability more precise

[ https://issues.apache.org/jira/browse/SPARK-26747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26747: Assignee: (was: Apache Spark) > Makes GetMapValue nullability more precise >

[jira] [Created] (SPARK-26747) Makes GetMapValue nullability more precise

Takeshi Yamamuro created SPARK-26747: Summary: Makes GetMapValue nullability more precise Key: SPARK-26747 URL: https://issues.apache.org/jira/browse/SPARK-26747 Project: Spark Issue Type

[jira] [Commented] (SPARK-24959) Do not invoke the CSV/JSON parser for empty schema

[ https://issues.apache.org/jira/browse/SPARK-24959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753697#comment-16753697 ] Apache Spark commented on SPARK-24959: -- User 'HyukjinKwon' has created a pull reque

[jira] [Resolved] (SPARK-26725) Fix the input values of UnifiedMemoryManager constructor in test suites

[ https://issues.apache.org/jira/browse/SPARK-26725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26725. - Resolution: Fixed Fix Version/s: 3.0.0 > Fix the input values of UnifiedMemoryManager con

[jira] [Updated] (SPARK-26745) Non-parsing Dataset.count() optimization causes inconsistent results for JSON inputs with empty lines

[ https://issues.apache.org/jira/browse/SPARK-26745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26745: - Priority: Blocker (was: Major) > Non-parsing Dataset.count() optimization causes inconsistent r

[jira] [Resolved] (SPARK-26743) Add a test to check the actual resource limit set via 'spark.executor.pyspark.memory'

[ https://issues.apache.org/jira/browse/SPARK-26743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26743. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23663 [https://gi

[jira] [Created] (SPARK-26746) Adaptive causes non-action operations to trigger computation

eaton created SPARK-26746: - Summary: Adaptive causes non-action operations to trigger computation Key: SPARK-26746 URL: https://issues.apache.org/jira/browse/SPARK-26746 Project: Spark Issue Type: B

[jira] [Assigned] (SPARK-26743) Add a test to check the actual resource limit set via 'spark.executor.pyspark.memory'

[ https://issues.apache.org/jira/browse/SPARK-26743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26743: Assignee: Hyukjin Kwon > Add a test to check the actual resource limit set via > 'spark.

[jira] [Commented] (SPARK-26745) Non-parsing Dataset.count() optimization causes inconsistent results for JSON inputs with empty lines

[ https://issues.apache.org/jira/browse/SPARK-26745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753644#comment-16753644 ] Takeshi Yamamuro commented on SPARK-26745: -- I checked I could reproduce this in

[jira] [Updated] (SPARK-26745) Non-parsing Dataset.count() optimization causes inconsistent results for JSON inputs with empty lines

[ https://issues.apache.org/jira/browse/SPARK-26745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-26745: - Labels: correctness (was: ) > Non-parsing Dataset.count() optimization causes inconsist

[jira] [Commented] (SPARK-26745) Non-parsing Dataset.count() optimization causes inconsistent results for JSON inputs with empty lines

[ https://issues.apache.org/jira/browse/SPARK-26745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753635#comment-16753635 ] Branden Smith commented on SPARK-26745: --- opened PR with proposed resolution: https

[jira] [Assigned] (SPARK-26745) Non-parsing Dataset.count() optimization causes inconsistent results for JSON inputs with empty lines

[ https://issues.apache.org/jira/browse/SPARK-26745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26745: Assignee: (was: Apache Spark) > Non-parsing Dataset.count() optimization causes incon

[jira] [Assigned] (SPARK-26745) Non-parsing Dataset.count() optimization causes inconsistent results for JSON inputs with empty lines

[ https://issues.apache.org/jira/browse/SPARK-26745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26745: Assignee: Apache Spark > Non-parsing Dataset.count() optimization causes inconsistent res

[jira] [Created] (SPARK-26745) Non-parsing Dataset.count() optimization causes inconsistent results for JSON inputs with empty lines

Branden Smith created SPARK-26745: - Summary: Non-parsing Dataset.count() optimization causes inconsistent results for JSON inputs with empty lines Key: SPARK-26745 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Closed] (SPARK-26724) Non negative coefficients for LinearRegression

[ https://issues.apache.org/jira/browse/SPARK-26724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-26724. - > Non negative coefficients for LinearRegression > -- >

[jira] [Closed] (SPARK-26717) Support PodPriority for spark driver and executor on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-26717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-26717. - > Support PodPriority for spark driver and executor on kubernetes >

[jira] [Updated] (SPARK-26379) Use dummy TimeZoneId for CurrentTimestamp to avoid UnresolvedException in CurrentBatchTimestamp

[ https://issues.apache.org/jira/browse/SPARK-26379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26379: -- Summary: Use dummy TimeZoneId for CurrentTimestamp to avoid UnresolvedException in CurrentBatc

[jira] [Updated] (SPARK-26379) Use dummy TimeZoneId to avoid UnresolvedException in CurrentBatchTimestamp

[ https://issues.apache.org/jira/browse/SPARK-26379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26379: -- Summary: Use dummy TimeZoneId to avoid UnresolvedException in CurrentBatchTimestamp (was: Fix

[jira] [Updated] (SPARK-26379) Fix issue on adding current_timestamp to streaming query

[ https://issues.apache.org/jira/browse/SPARK-26379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26379: -- Summary: Fix issue on adding current_timestamp to streaming query (was: Fix issue on adding c

[jira] [Resolved] (SPARK-26379) Fix issue on adding current_timestamp/current_date to streaming query

[ https://issues.apache.org/jira/browse/SPARK-26379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26379. --- Resolution: Fixed > Fix issue on adding current_timestamp/current_date to streaming query >

[jira] [Updated] (SPARK-26739) Standardized Join Types for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-26739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-26739: - Fix Version/s: (was: 2.4.1) > Standardized Join Types for DataFrames > -

[jira] [Commented] (SPARK-19254) Support Seq, Map, and Struct in functions.lit

[ https://issues.apache.org/jira/browse/SPARK-19254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753582#comment-16753582 ] Artem Rukavytsia commented on SPARK-19254: -- Looks like Pyspark doesn't support

[jira] [Resolved] (SPARK-26716) Refactor supportDataType API: the supported types of read/write should be consistent

[ https://issues.apache.org/jira/browse/SPARK-26716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26716. --- Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 This is reso

[jira] [Updated] (SPARK-26739) Standardized Join Types for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-26739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Skyler Lehan updated SPARK-26739: - Description: h3. *Q1.* What are you trying to do? Articulate your objectives using absolutely n

[jira] [Updated] (SPARK-26689) Single disk broken causing broadcast failure

[ https://issues.apache.org/jira/browse/SPARK-26689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26689: - Summary: Single disk broken causing broadcast failure (was: Disk broken causing broadcast failu

[jira] [Commented] (SPARK-26675) Error happened during creating avro files

[ https://issues.apache.org/jira/browse/SPARK-26675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753354#comment-16753354 ] Hyukjin Kwon commented on SPARK-26675: -- Also, please make the code as minimised as

[jira] [Updated] (SPARK-26675) Error happened during creating avro files

[ https://issues.apache.org/jira/browse/SPARK-26675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26675: - Component/s: SQL > Error happened during creating avro files > -

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753353#comment-16753353 ] Hyukjin Kwon commented on SPARK-26727: -- I can't reproduce too. Since this issue is

[jira] [Updated] (SPARK-26722) Set SPARK_TEST_KEY to pull request builder and spark-master-test-sbt-hadoop-2.7

[ https://issues.apache.org/jira/browse/SPARK-26722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26722: - Description: See https://github.com/apache/spark/pull/23117#issuecomment-456083489 (was: from

[jira] [Commented] (SPARK-26722) Set SPARK_TEST_KEY to pull request builder and spark-master-test-sbt-hadoop-2.7

[ https://issues.apache.org/jira/browse/SPARK-26722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753352#comment-16753352 ] Hyukjin Kwon commented on SPARK-26722: -- We're synced now and I think it's (mostly)

[jira] [Updated] (SPARK-26722) Set SPARK_TEST_KEY to pull request builder and spark-master-test-sbt-hadoop-2.7

[ https://issues.apache.org/jira/browse/SPARK-26722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26722: - Summary: Set SPARK_TEST_KEY to pull request builder and spark-master-test-sbt-hadoop-2.7 (was:

[jira] [Resolved] (SPARK-26722) add SPARK_TEST_KEY=1 to pull request builder and spark-master-test-sbt-hadoop-2.7

[ https://issues.apache.org/jira/browse/SPARK-26722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26722. -- Resolution: Fixed Thank you Shane so much for doing this! > add SPARK_TEST_KEY=1 to pull requ

[jira] [Commented] (SPARK-15368) Spark History Server does not pick up extraClasspath

[ https://issues.apache.org/jira/browse/SPARK-15368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753339#comment-16753339 ] Sandeep Nemuri commented on SPARK-15368: Just for the records, {{SPARK_DIST_CLAS

[jira] [Commented] (SPARK-26675) Error happened during creating avro files

[ https://issues.apache.org/jira/browse/SPARK-26675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753326#comment-16753326 ] Gengliang Wang commented on SPARK-26675: [~tony0918] Can you provide a sample in