[jira] [Updated] (SPARK-28410) ANSI SQL: OCTET_LENGTH function

2019-07-15 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-28410: --- Description: ||Function||Return Type||Description||Example||Result||

[jira] [Updated] (SPARK-28410) ANSI SQL: OCTET_LENGTH function

2019-07-15 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-28410: --- Description: ||Function||Return Type||Description||Example||Result||

[jira] [Commented] (SPARK-28410) ANSI SQL: OCTET_LENGTH function

2019-07-15 Thread jiaan.geng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885838#comment-16885838 ] jiaan.geng commented on SPARK-28410: I'm working on. > ANSI SQL: OCTET_LENGTH function >

[jira] [Created] (SPARK-28410) ANSI SQL: OCTET_LENGTH function

2019-07-15 Thread jiaan.geng (JIRA)
jiaan.geng created SPARK-28410: -- Summary: ANSI SQL: OCTET_LENGTH function Key: SPARK-28410 URL: https://issues.apache.org/jira/browse/SPARK-28410 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-18299) Allow more aggregations on KeyValueGroupedDataset

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18299: -- Affects Version/s: (was: 2.0.1) 3.0.0 > Allow more aggregations on

[jira] [Updated] (SPARK-18299) Allow more aggregations on KeyValueGroupedDataset

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18299: -- Labels: (was: bulk-closed) > Allow more aggregations on KeyValueGroupedDataset >

[jira] [Updated] (SPARK-18299) Allow more aggregations on KeyValueGroupedDataset

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18299: -- Priority: Minor (was: Major) > Allow more aggregations on KeyValueGroupedDataset >

[jira] [Reopened] (SPARK-18299) Allow more aggregations on KeyValueGroupedDataset

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-18299: --- > Allow more aggregations on KeyValueGroupedDataset >

[jira] [Resolved] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28398. -- Resolution: Incomplete > Exception thrown while writing record:

[jira] [Commented] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885826#comment-16885826 ] Hyukjin Kwon commented on SPARK-28398: -- Also, 2.0.2 is an EOL release where we don't land a fix

[jira] [Updated] (SPARK-28389) Use Java 8 API to add months to dates

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28389: -- Issue Type: Sub-task (was: Improvement) Parent: SPARK-27764 > Use Java 8 API to add

[jira] [Updated] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28398: - Priority: Major (was: Critical) > Exception thrown while writing record:

[jira] [Commented] (SPARK-28389) Use Java 8 API to add months to dates

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885825#comment-16885825 ] Dongjoon Hyun commented on SPARK-28389: --- I moved this to SPARK-27764 to give more visibility to

[jira] [Commented] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885824#comment-16885824 ] Hyukjin Kwon commented on SPARK-28398: -- Please just don't copy and paste the error messages. No one

[jira] [Updated] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28398: - Description: {code} ReceivedBlockTracker - Exception thrown while writing record:

[jira] [Resolved] (SPARK-28408) Restrict tested values of DateType, TimestampType and CalendarIntervalType

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28408. --- Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 3.0.0 This is

[jira] [Commented] (SPARK-28389) Use Java 8 API to add months to dates

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885812#comment-16885812 ] Dongjoon Hyun commented on SPARK-28389: --- As a side note, this will be another instance which is

[jira] [Updated] (SPARK-28247) Flaky test: "query without test harness" in ContinuousSuite

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28247: -- Component/s: Tests > Flaky test: "query without test harness" in ContinuousSuite >

[jira] [Created] (SPARK-28409) SELECT FROM syntax

2019-07-15 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28409: --- Summary: SELECT FROM syntax Key: SPARK-28409 URL: https://issues.apache.org/jira/browse/SPARK-28409 Project: Spark Issue Type: Sub-task Components:

[jira] [Assigned] (SPARK-27485) Certain query plans fail to run when autoBroadcastJoinThreshold is set to -1

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27485: Assignee: (was: Apache Spark) > Certain query plans fail to run when 

[jira] [Assigned] (SPARK-27485) Certain query plans fail to run when autoBroadcastJoinThreshold is set to -1

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27485: Assignee: Apache Spark > Certain query plans fail to run when autoBroadcastJoinThreshold

[jira] [Resolved] (SPARK-28407) Support mapping spark.local.dir to hostPath volume

2019-07-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-28407. Resolution: Duplicate This had already been cloned elsewhere. > Support mapping

[jira] [Commented] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

2019-07-15 Thread Junjie Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885666#comment-16885666 ] Junjie Chen commented on SPARK-27499: - Hi [~vanzin] There is an opened Jira SPARK-28042 for this.

[jira] [Updated] (SPARK-28049) i want to a first ticket in zira

2019-07-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-28049: --- Fix Version/s: (was: 2.4.4) (was: 2.4.3) > i want to a first ticket in

[jira] [Assigned] (SPARK-28408) Restrict tested values of DateType, TimestampType and CalendarIntervalType

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28408: Assignee: (was: Apache Spark) > Restrict tested values of DateType, TimestampType

[jira] [Assigned] (SPARK-28408) Restrict tested values of DateType, TimestampType and CalendarIntervalType

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28408: Assignee: Apache Spark > Restrict tested values of DateType, TimestampType and

[jira] [Created] (SPARK-28408) Restrict tested values of DateType, TimestampType and CalendarIntervalType

2019-07-15 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-28408: -- Summary: Restrict tested values of DateType, TimestampType and CalendarIntervalType Key: SPARK-28408 URL: https://issues.apache.org/jira/browse/SPARK-28408 Project:

[jira] [Commented] (SPARK-28264) Revisiting Python / pandas UDF

2019-07-15 Thread Holden Karau (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885586#comment-16885586 ] Holden Karau commented on SPARK-28264: -- I'd love to know [~bryanc]'s thoughts on this proposal as

[jira] [Commented] (SPARK-28264) Revisiting Python / pandas UDF

2019-07-15 Thread Holden Karau (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885581#comment-16885581 ] Holden Karau commented on SPARK-28264: -- Maybe I missed it in the document, but I feel like I don't

[jira] [Commented] (SPARK-27499) Support mapping spark.local.dir to hostPath volume

2019-07-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885578#comment-16885578 ] Marcelo Vanzin commented on SPARK-27499: I can't see an option to reopen this, so I'll clone it

[jira] [Created] (SPARK-28407) Support mapping spark.local.dir to hostPath volume

2019-07-15 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-28407: -- Summary: Support mapping spark.local.dir to hostPath volume Key: SPARK-28407 URL: https://issues.apache.org/jira/browse/SPARK-28407 Project: Spark Issue

[jira] [Commented] (SPARK-28340) Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException

2019-07-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885550#comment-16885550 ] Josh Rosen commented on SPARK-28340: SPARK-23816 is a related issue about fetch failures caused by

[jira] [Commented] (SPARK-28340) Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException

2019-07-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885547#comment-16885547 ] Josh Rosen commented on SPARK-28340: Another variant of this issue, this time on the shuffle read

[jira] [Updated] (SPARK-28152) Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDialect

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28152: -- Summary: Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDialect (was:

[jira] [Resolved] (SPARK-28152) [JDBC Connector] ShortType and FloatTypes are not mapped correctly for read/write of SQLServer Tables

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28152. --- Resolution: Fixed Assignee: Shiv Prashant Sood Fix Version/s: 3.0.0 This is

[jira] [Commented] (SPARK-28375) Enforce idempotence on the PullupCorrelatedPredicates optimizer rule

2019-07-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885531#comment-16885531 ] Josh Rosen commented on SPARK-28375: I'm not sure; is it possible to trigger double-optimization

[jira] [Commented] (SPARK-28201) Revisit MakeDecimal behavior on overflow

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885514#comment-16885514 ] Apache Spark commented on SPARK-28201: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-28288) Convert and port 'window.sql' into UDF test base

2019-07-15 Thread YoungGyu Chun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885493#comment-16885493 ] YoungGyu Chun commented on SPARK-28288: --- Thank you for the heads up. I will try this once the PR

[jira] [Commented] (SPARK-28152) [JDBC Connector] ShortType and FloatTypes are not mapped correctly for read/write of SQLServer Tables

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885488#comment-16885488 ] Dongjoon Hyun commented on SPARK-28152: --- Hi, [~shivsood]. Could you check `2.3.3` please, too? >

[jira] [Resolved] (SPARK-28404) Fix negative timeout value in RateStreamContinuousPartitionReader

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28404. --- Resolution: Fixed Fix Version/s: 2.4.4 2.3.4

[jira] [Assigned] (SPARK-28404) Fix negative timeout value in RateStreamContinuousPartitionReader

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28404: - Assignee: Gabor Somogyi > Fix negative timeout value in

[jira] [Commented] (SPARK-28375) Enforce idempotence on the PullupCorrelatedPredicates optimizer rule

2019-07-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885450#comment-16885450 ] Xiao Li commented on SPARK-28375: - [~joshrosen] The issue is only triggered if we run the rule more than

[jira] [Commented] (SPARK-28376) Support to write sorted parquet files in each row group

2019-07-15 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885428#comment-16885428 ] Ryan Blue commented on SPARK-28376: --- I don't think this is a regression. The linked issue was to

[jira] [Assigned] (SPARK-28406) Port union.sql

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28406: Assignee: Apache Spark > Port union.sql > -- > > Key:

[jira] [Assigned] (SPARK-28406) Port union.sql

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28406: Assignee: (was: Apache Spark) > Port union.sql > -- > >

[jira] [Created] (SPARK-28406) Port union.sql

2019-07-15 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-28406: --- Summary: Port union.sql Key: SPARK-28406 URL: https://issues.apache.org/jira/browse/SPARK-28406 Project: Spark Issue Type: Sub-task Components: SQL,

[jira] [Updated] (SPARK-28404) Fix negative timeout value in RateStreamContinuousPartitionReader

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28404: -- Summary: Fix negative timeout value in RateStreamContinuousPartitionReader (was: Flaky test:

[jira] [Updated] (SPARK-28404) Fix negative timeout value in RateStreamContinuousPartitionReader

2019-07-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28404: -- Component/s: (was: Tests) > Fix negative timeout value in

[jira] [Updated] (SPARK-28405) Join with USING syntax doesn't hide original tables

2019-07-15 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-28405: --- Description: The following query doesn't work in SparkSQL as t1.* and t2.* can't be used after

[jira] [Updated] (SPARK-28405) Join with USING caluse doesn't hide original tables

2019-07-15 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-28405: --- Summary: Join with USING caluse doesn't hide original tables (was: Join with USING syntax doesn't

[jira] [Created] (SPARK-28405) Join with USING syntax doesn't hide original tables

2019-07-15 Thread Peter Toth (JIRA)
Peter Toth created SPARK-28405: -- Summary: Join with USING syntax doesn't hide original tables Key: SPARK-28405 URL: https://issues.apache.org/jira/browse/SPARK-28405 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-28404) Flaky test: ContinuousStressSuite.only one epoch

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28404: Assignee: Apache Spark > Flaky test: ContinuousStressSuite.only one epoch >

[jira] [Assigned] (SPARK-28404) Flaky test: ContinuousStressSuite.only one epoch

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28404: Assignee: (was: Apache Spark) > Flaky test: ContinuousStressSuite.only one epoch >

[jira] [Commented] (SPARK-24666) Word2Vec generate infinity vectors when numIterations are large

2019-07-15 Thread ZhongYu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885302#comment-16885302 ] ZhongYu commented on SPARK-24666: - Not any update for one year? > Word2Vec generate infinity vectors

[jira] [Updated] (SPARK-24666) Word2Vec generate infinity vectors when numIterations are large

2019-07-15 Thread ZhongYu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhongYu updated SPARK-24666: Priority: Critical (was: Major) > Word2Vec generate infinity vectors when numIterations are large >

[jira] [Created] (SPARK-28404) Flaky test: ContinuousStressSuite.only one epoch

2019-07-15 Thread Gabor Somogyi (JIRA)
Gabor Somogyi created SPARK-28404: - Summary: Flaky test: ContinuousStressSuite.only one epoch Key: SPARK-28404 URL: https://issues.apache.org/jira/browse/SPARK-28404 Project: Spark Issue

[jira] [Assigned] (SPARK-28311) Spark Thrift Server protocol version compatibility setup too late

2019-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28311: - Assignee: angerszhu > Spark Thrift Server protocol version compatibility setup too late >

[jira] [Resolved] (SPARK-28311) Spark Thrift Server protocol version compatibility setup too late

2019-07-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28311. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25083

[jira] [Commented] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Edwin Biemond (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885264#comment-16885264 ] Edwin Biemond commented on SPARK-27927: --- make sense , for us we also needed to backport jackson to

[jira] [Created] (SPARK-28403) Executor Allocation Manager can add an extra executor when speculative tasks

2019-07-15 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-28403: - Summary: Executor Allocation Manager can add an extra executor when speculative tasks Key: SPARK-28403 URL: https://issues.apache.org/jira/browse/SPARK-28403

[jira] [Comment Edited] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:43 PM:

[jira] [Comment Edited] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:42 PM:

[jira] [Comment Edited] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:41 PM:

[jira] [Comment Edited] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:40 PM:

[jira] [Comment Edited] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:39 PM:

[jira] [Commented] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Yu Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885238#comment-16885238 ] Yu Wang commented on SPARK-28398: - [~gsomogyi] Thank you for your answer. I try it again by using

[jira] [Comment Edited] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:38 PM:

[jira] [Comment Edited] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:37 PM:

[jira] [Comment Edited] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos edited comment on SPARK-27927 at 7/15/19 1:35 PM:

[jira] [Commented] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885234#comment-16885234 ] Stavros Kontopoulos commented on SPARK-27927: - np. There will be a 2.4.4 release so there is

[jira] [Updated] (SPARK-28402) Array indexing is 1-based

2019-07-15 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-28402: --- Description: Array indexing is 1-based in PostgreSQL: 

[jira] [Created] (SPARK-28402) Array indexing is 1-based

2019-07-15 Thread Peter Toth (JIRA)
Peter Toth created SPARK-28402: -- Summary: Array indexing is 1-based Key: SPARK-28402 URL: https://issues.apache.org/jira/browse/SPARK-28402 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-27812) kubernetes client import non-daemon thread which block jvm exit.

2019-07-15 Thread Edwin Biemond (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885219#comment-16885219 ] Edwin Biemond commented on SPARK-27812: --- for SPARK-27927 indeed all is fine on spark 2.4.3 when I

[jira] [Commented] (SPARK-27927) driver pod hangs with pyspark 2.4.3 and master on kubenetes

2019-07-15 Thread Edwin Biemond (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885217#comment-16885217 ] Edwin Biemond commented on SPARK-27927: --- Thanks you so much for your time and  indeed all is fine

[jira] [Commented] (SPARK-22989) sparkstreaming ui show 0 records when spark-streaming-kafka application restore from checkpoint

2019-07-15 Thread yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885209#comment-16885209 ] yang commented on SPARK-22989: -- i came across this phenomenon on spark 2.2.1, when can there be an official

[jira] [Commented] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885211#comment-16885211 ] Gabor Somogyi commented on SPARK-28398: --- Maybe you can try to increase

[jira] [Commented] (SPARK-28352) Spark UI shows RDD blocks as 0 in some executor metrics

2019-07-15 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-28352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885208#comment-16885208 ] Burak KÖSE commented on SPARK-28352: I don't think I can show the code because there are more than

[jira] [Commented] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885205#comment-16885205 ] Gabor Somogyi commented on SPARK-28398: --- I've had a look at this and not yet sure what's the exact

[jira] [Resolved] (SPARK-28389) Use Java 8 API to add months to dates

2019-07-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28389. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25153

[jira] [Assigned] (SPARK-28389) Use Java 8 API to add months to dates

2019-07-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28389: --- Assignee: Maxim Gekk > Use Java 8 API to add months to dates >

[jira] [Updated] (SPARK-28401) Add built-in Array Functions: element to array concatenation

2019-07-15 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-28401: --- Description: ||Function||Description||Example||Result|| |{{\|\|}}|element-to-array

[jira] [Assigned] (SPARK-28390) Convert and port 'pgSQL/select_having.sql' into UDF test base

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28390: Assignee: Apache Spark > Convert and port 'pgSQL/select_having.sql' into UDF test base >

[jira] [Assigned] (SPARK-28390) Convert and port 'pgSQL/select_having.sql' into UDF test base

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28390: Assignee: (was: Apache Spark) > Convert and port 'pgSQL/select_having.sql' into UDF

[jira] [Created] (SPARK-28401) Add built-in Array Functions: element to array concatenation

2019-07-15 Thread Peter Toth (JIRA)
Peter Toth created SPARK-28401: -- Summary: Add built-in Array Functions: element to array concatenation Key: SPARK-28401 URL: https://issues.apache.org/jira/browse/SPARK-28401 Project: Spark

[jira] [Commented] (SPARK-28086) Adds `random()` sql function

2019-07-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885177#comment-16885177 ] Yuming Wang commented on SPARK-28086: - Could we use {{rand()}}? > Adds `random()` sql function >

[jira] [Updated] (SPARK-28400) Add built-in Array Functions: array_upper

2019-07-15 Thread Peter Toth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-28400: --- Description: ||Function||Return Type||Description||Example||Result||

[jira] [Created] (SPARK-28400) Add built-in Array Functions: array_upper

2019-07-15 Thread Peter Toth (JIRA)
Peter Toth created SPARK-28400: -- Summary: Add built-in Array Functions: array_upper Key: SPARK-28400 URL: https://issues.apache.org/jira/browse/SPARK-28400 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-28399) Impl RobustScaler

2019-07-15 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-28399: - Issue Type: New Feature (was: Improvement) > Impl RobustScaler > - > >

[jira] [Assigned] (SPARK-28399) Impl RobustScaler

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28399: Assignee: Apache Spark > Impl RobustScaler > - > > Key:

[jira] [Assigned] (SPARK-28399) Impl RobustScaler

2019-07-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28399: Assignee: (was: Apache Spark) > Impl RobustScaler > - > >

[jira] [Created] (SPARK-28399) Impl RobustScaler

2019-07-15 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-28399: Summary: Impl RobustScaler Key: SPARK-28399 URL: https://issues.apache.org/jira/browse/SPARK-28399 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-28397) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Yu Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu Wang resolved SPARK-28397. - Resolution: Invalid > Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer()) >

[jira] [Created] (SPARK-28398) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Yu Wang (JIRA)
Yu Wang created SPARK-28398: --- Summary: Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer()) Key: SPARK-28398 URL: https://issues.apache.org/jira/browse/SPARK-28398 Project: Spark

[jira] [Commented] (SPARK-25411) Implement range partition in Spark

2019-07-15 Thread Wang, Gang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885028#comment-16885028 ] Wang, Gang commented on SPARK-25411: I referred to ORACLE DDLs, quite close to PG.   > Implement

[jira] [Created] (SPARK-28397) Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer())

2019-07-15 Thread Yu Wang (JIRA)
Yu Wang created SPARK-28397: --- Summary: Exception thrown while writing record: BatchCleanupEvent(ArrayBuffer()) Key: SPARK-28397 URL: https://issues.apache.org/jira/browse/SPARK-28397 Project: Spark

[jira] [Commented] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

2019-07-15 Thread Xianyin Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885019#comment-16885019 ] Xianyin Xin commented on SPARK-21067: - [~dricard],does the attached patch work in your case? >

[jira] [Comment Edited] (SPARK-28368) Row.getAs() return different values in scala and java

2019-07-15 Thread Kideok Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885001#comment-16885001 ] Kideok Kim edited comment on SPARK-28368 at 7/15/19 9:34 AM: - Got it. I

[jira] [Commented] (SPARK-28368) Row.getAs() return different values in scala and java

2019-07-15 Thread Kideok Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885001#comment-16885001 ] Kideok Kim commented on SPARK-28368: Got it. I thought maybe we can add more comments on that api

[jira] [Comment Edited] (SPARK-28368) Row.getAs() return different values in scala and java

2019-07-15 Thread Kideok Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884971#comment-16884971 ] Kideok Kim edited comment on SPARK-28368 at 7/15/19 9:23 AM: - Thank you for

[jira] [Commented] (SPARK-28368) Row.getAs() return different values in scala and java

2019-07-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-28368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16884980#comment-16884980 ] Hyukjin Kwon commented on SPARK-28368: -- Yea that's due to Java limitation. What do we need to fix

  1   2   >