[jira] [Resolved] (SPARK-24634) Add a new metric regarding number of rows later than watermark

[ https://issues.apache.org/jira/browse/SPARK-24634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24634. -- Fix Version/s: 3.1.0 Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-31968) write.partitionBy() creates duplicate subdirectories when user provide duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-31968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31968. --- Fix Version/s: 3.1.0 2.4.7 3.0.1 Resolution:

[jira] [Commented] (SPARK-31632) The ApplicationInfo in KVStore may be accessed before it's prepared

[ https://issues.apache.org/jira/browse/SPARK-31632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135038#comment-17135038 ] Hyukjin Kwon commented on SPARK-31632: -- Followup

[jira] [Commented] (SPARK-31968) write.partitionBy() creates duplicate subdirectories when user provide duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-31968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135036#comment-17135036 ] Dongjoon Hyun commented on SPARK-31968: --- Thanks. I also verified that the read-side exceptions

[jira] [Updated] (SPARK-31968) write.partitionBy() creates duplicate subdirectories when user provide duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-31968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31968: -- Affects Version/s: (was: 2.4.5) 2.0.2 2.1.3

[jira] [Commented] (SPARK-31585) Support Z-order curve

[ https://issues.apache.org/jira/browse/SPARK-31585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135004#comment-17135004 ] Yuming Wang commented on SPARK-31585: -

[jira] [Commented] (SPARK-29429) Support Prometheus monitoring natively

[ https://issues.apache.org/jira/browse/SPARK-29429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134999#comment-17134999 ] Dongjoon Hyun commented on SPARK-29429: --- Sure, [~smilegator]. > Support Prometheus monitoring

[jira] [Resolved] (SPARK-29429) Support Prometheus monitoring natively

[ https://issues.apache.org/jira/browse/SPARK-29429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29429. --- Assignee: Dongjoon Hyun Resolution: Done > Support Prometheus monitoring natively >

[jira] [Updated] (SPARK-29430) Document new metric endpoints for Prometheus

[ https://issues.apache.org/jira/browse/SPARK-29430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29430: -- Parent: (was: SPARK-29429) Issue Type: Task (was: Sub-task) > Document new

[jira] [Assigned] (SPARK-31984) Make micros rebasing functions via local timestamps pure

[ https://issues.apache.org/jira/browse/SPARK-31984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31984: Assignee: (was: Apache Spark) > Make micros rebasing functions via local timestamps

[jira] [Assigned] (SPARK-31984) Make micros rebasing functions via local timestamps pure

[ https://issues.apache.org/jira/browse/SPARK-31984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31984: Assignee: Apache Spark > Make micros rebasing functions via local timestamps pure >

[jira] [Commented] (SPARK-31984) Make micros rebasing functions via local timestamps pure

[ https://issues.apache.org/jira/browse/SPARK-31984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134929#comment-17134929 ] Apache Spark commented on SPARK-31984: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-31984) Make micros rebasing functions via local timestamps pure

Maxim Gekk created SPARK-31984: -- Summary: Make micros rebasing functions via local timestamps pure Key: SPARK-31984 URL: https://issues.apache.org/jira/browse/SPARK-31984 Project: Spark Issue

[jira] [Commented] (SPARK-29429) Support Prometheus monitoring natively

[ https://issues.apache.org/jira/browse/SPARK-29429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134911#comment-17134911 ] Xiao Li commented on SPARK-29429: - [~dongjoon] Can we resolve this? > Support Prometheus monitoring

[jira] [Resolved] (SPARK-31944) Add instance weight support in LinearRegressionSummary

[ https://issues.apache.org/jira/browse/SPARK-31944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31944. -- Fix Version/s: 3.1.0 Assignee: Huaxin Gao Resolution: Fixed Resolved by

[jira] [Comment Edited] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134862#comment-17134862 ] Oscar Cassetti edited comment on SPARK-26365 at 6/13/20, 4:36 PM: -- I

[jira] [Commented] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134862#comment-17134862 ] Oscar Cassetti commented on SPARK-26365: I can see the same issue and I think it is due to this 

[jira] [Commented] (SPARK-30968) Upgrade aws-java-sdk-sts to 1.11.655

[ https://issues.apache.org/jira/browse/SPARK-30968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134861#comment-17134861 ] Xiao Li commented on SPARK-30968: - Added the release note based on the issue we identified.  > Upgrade

[jira] [Updated] (SPARK-30968) Upgrade aws-java-sdk-sts to 1.11.655

[ https://issues.apache.org/jira/browse/SPARK-30968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30968: Docs Text: With AWS SDK upgrade to 1.11.655, we strongly encourage the users that use S3N file system

[jira] [Updated] (SPARK-30968) Upgrade aws-java-sdk-sts to 1.11.655

[ https://issues.apache.org/jira/browse/SPARK-30968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30968: Labels: release-notes (was: ) > Upgrade aws-java-sdk-sts to 1.11.655 >

[jira] [Assigned] (SPARK-31983) Tables of structured streaming tab show wrong result for duration column

[ https://issues.apache.org/jira/browse/SPARK-31983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31983: Assignee: Apache Spark > Tables of structured streaming tab show wrong result for

[jira] [Commented] (SPARK-31983) Tables of structured streaming tab show wrong result for duration column

[ https://issues.apache.org/jira/browse/SPARK-31983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134840#comment-17134840 ] Apache Spark commented on SPARK-31983: -- User 'iRakson' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31983) Tables of structured streaming tab show wrong result for duration column

[ https://issues.apache.org/jira/browse/SPARK-31983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31983: Assignee: (was: Apache Spark) > Tables of structured streaming tab show wrong result

[jira] [Created] (SPARK-31983) Tables of structured streaming tab show wrong result for duration column

Rakesh Raushan created SPARK-31983: -- Summary: Tables of structured streaming tab show wrong result for duration column Key: SPARK-31983 URL: https://issues.apache.org/jira/browse/SPARK-31983

[jira] [Commented] (SPARK-31980) Spark sequence() fails if start and end of range are identical dates

[ https://issues.apache.org/jira/browse/SPARK-31980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134784#comment-17134784 ] Dave DeCaprio commented on SPARK-31980: --- TJX2014 thanks for the quick turnaround there.  I was

[jira] [Created] (SPARK-31982) Spark sequence doesn't handle date increments that cross DST

Dave DeCaprio created SPARK-31982: - Summary: Spark sequence doesn't handle date increments that cross DST Key: SPARK-31982 URL: https://issues.apache.org/jira/browse/SPARK-31982 Project: Spark

[jira] [Commented] (SPARK-31644) Make Spark's guava version configurable from the maven command line.

[ https://issues.apache.org/jira/browse/SPARK-31644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134723#comment-17134723 ] Apache Spark commented on SPARK-31644: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-31644) Make Spark's guava version configurable from the maven command line.

[ https://issues.apache.org/jira/browse/SPARK-31644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134724#comment-17134724 ] Apache Spark commented on SPARK-31644: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-31981) Keep TimestampType when taking an average of a Timestamp

[ https://issues.apache.org/jira/browse/SPARK-31981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134717#comment-17134717 ] Apache Spark commented on SPARK-31981: -- User 'Fokko' has created a pull request for this issue:

[jira] [Commented] (SPARK-31981) Keep TimestampType when taking an average of a Timestamp

[ https://issues.apache.org/jira/browse/SPARK-31981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134716#comment-17134716 ] Apache Spark commented on SPARK-31981: -- User 'Fokko' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31981) Keep TimestampType when taking an average of a Timestamp

[ https://issues.apache.org/jira/browse/SPARK-31981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31981: Assignee: Apache Spark > Keep TimestampType when taking an average of a Timestamp >

[jira] [Assigned] (SPARK-31981) Keep TimestampType when taking an average of a Timestamp

[ https://issues.apache.org/jira/browse/SPARK-31981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31981: Assignee: (was: Apache Spark) > Keep TimestampType when taking an average of a

[jira] [Created] (SPARK-31981) Keep TimestampType when taking an average of a Timestamp

Fokko Driesprong created SPARK-31981: Summary: Keep TimestampType when taking an average of a Timestamp Key: SPARK-31981 URL: https://issues.apache.org/jira/browse/SPARK-31981 Project: Spark

[jira] [Commented] (SPARK-31632) The ApplicationInfo in KVStore may be accessed before it's prepared

[ https://issues.apache.org/jira/browse/SPARK-31632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134655#comment-17134655 ] Apache Spark commented on SPARK-31632: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-31632) The ApplicationInfo in KVStore may be accessed before it's prepared

[ https://issues.apache.org/jira/browse/SPARK-31632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134654#comment-17134654 ] Apache Spark commented on SPARK-31632: -- User 'sarutak' has created a pull request for this issue: