[jira] [Created] (SPARK-28345) PythonUDF predicate should be able to pushdown to join

Liang-Chi Hsieh created SPARK-28345: --- Summary: PythonUDF predicate should be able to pushdown to join Key: SPARK-28345 URL: https://issues.apache.org/jira/browse/SPARK-28345 Project: Spark

[jira] [Comment Edited] (SPARK-28269) ArrowStreamPandasSerializer get stack

[ https://issues.apache.org/jira/browse/SPARK-28269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882639#comment-16882639 ] Hyukjin Kwon edited comment on SPARK-28269 at 7/11/19 4:52 AM: --- Workaround

[jira] [Comment Edited] (SPARK-28269) ArrowStreamPandasSerializer get stack

[ https://issues.apache.org/jira/browse/SPARK-28269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882639#comment-16882639 ] Hyukjin Kwon edited comment on SPARK-28269 at 7/11/19 4:51 AM: --- Workaround

[jira] [Created] (SPARK-28344) fail the query if detect ambiguous self join

Wenchen Fan created SPARK-28344: --- Summary: fail the query if detect ambiguous self join Key: SPARK-28344 URL: https://issues.apache.org/jira/browse/SPARK-28344 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28269) ArrowStreamPandasSerializer get stack

[ https://issues.apache.org/jira/browse/SPARK-28269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882641#comment-16882641 ] Hyukjin Kwon commented on SPARK-28269: -- Seems like Arrow stream batches are not properly created

[jira] [Commented] (SPARK-28269) ArrowStreamPandasSerializer get stack

[ https://issues.apache.org/jira/browse/SPARK-28269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882639#comment-16882639 ] Hyukjin Kwon commented on SPARK-28269: -- Workaround to me was call `copy()` on this line:

[jira] [Commented] (SPARK-28337) spark jars do not contain commons-jxpath jar, cause ClassNotFound exception

[ https://issues.apache.org/jira/browse/SPARK-28337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882619#comment-16882619 ] Wang Yanlin commented on SPARK-28337: - my pom configuration for shade commons-configuration and

[jira] [Updated] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28015: -- Fix Version/s: 2.4.4 > Check stringToDate() consumes entire input for the and -[m]m

[jira] [Updated] (SPARK-28269) ArrowStreamPandasSerializer get stack

[ https://issues.apache.org/jira/browse/SPARK-28269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28269: - Description: I'm working with Pyspark version 2.4.3. I have a big data frame: * ~15M rows *

[jira] [Assigned] (SPARK-28306) Once optimizer rule NormalizeFloatingNumbers is not idempotent

[ https://issues.apache.org/jira/browse/SPARK-28306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28306: --- Assignee: Yesheng Ma > Once optimizer rule NormalizeFloatingNumbers is not idempotent >

[jira] [Resolved] (SPARK-28306) Once optimizer rule NormalizeFloatingNumbers is not idempotent

[ https://issues.apache.org/jira/browse/SPARK-28306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28306. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25080

[jira] [Resolved] (SPARK-28300) Kmeans is failing when we run parallely passing an RDD

[ https://issues.apache.org/jira/browse/SPARK-28300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28300. -- Resolution: Invalid Looks like a question. Let's interact with mailing list first before

[jira] [Commented] (SPARK-28320) Spark job eventually fails after several "attempted to access non-existent accumulator" in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-28320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882613#comment-16882613 ] Hyukjin Kwon commented on SPARK-28320: -- Is it possible to provide a reproducer? Seems difficult to

[jira] [Commented] (SPARK-28343) PostgreSQL test should change some default config

[ https://issues.apache.org/jira/browse/SPARK-28343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882611#comment-16882611 ] Yuming Wang commented on SPARK-28343: - I'm working on. > PostgreSQL test should change some default

[jira] [Commented] (SPARK-28327) Spark SQL can't support union with left query have queryOrganization

[ https://issues.apache.org/jira/browse/SPARK-28327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882601#comment-16882601 ] Hyukjin Kwon commented on SPARK-28327: -- Currently the feature party is being matched against

[jira] [Resolved] (SPARK-28327) Spark SQL can't support union with left query have queryOrganization

[ https://issues.apache.org/jira/browse/SPARK-28327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28327. -- Resolution: Won't Fix > Spark SQL can't support union with left query have queryOrganization

[jira] [Resolved] (SPARK-28336) Tried running same code in local machine in IDE pycharm it running fine but issue arises when i setup all on EC2 my RDD has Json Value and convert it to data frame and

[ https://issues.apache.org/jira/browse/SPARK-28336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28336. -- Resolution: Invalid Looks like a question. Let's interact with mailing list first before

[jira] [Updated] (SPARK-28336) Tried running same code in local machine in IDE pycharm it running fine but issue arises when i setup all on EC2 my RDD has Json Value and convert it to data frame and s

[ https://issues.apache.org/jira/browse/SPARK-28336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28336: - Description: I am a beginner to pyspark and I am creating a pilot project in spark i used

[jira] [Updated] (SPARK-28336) Tried running same code in local machine in IDE pycharm it running fine but issue arises when i setup all on EC2 my RDD has Json Value and convert it to data frame and s

[ https://issues.apache.org/jira/browse/SPARK-28336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28336: - Labels: kafka (was: beginner kafka newbie) > Tried running same code in local machine in IDE

[jira] [Updated] (SPARK-28336) Tried running same code in local machine in IDE pycharm it running fine but issue arises when i setup all on EC2 my RDD has Json Value and convert it to data frame and s

[ https://issues.apache.org/jira/browse/SPARK-28336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28336: - Description: I am a beginner to pyspark and I am creating a pilot project in spark i used

[jira] [Created] (SPARK-28343) PostgreSQL test should change some default config

Yuming Wang created SPARK-28343: --- Summary: PostgreSQL test should change some default config Key: SPARK-28343 URL: https://issues.apache.org/jira/browse/SPARK-28343 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-28342) Replace REL_12_BETA1 to REL_12_BETA2 in PostgresSQL SQL tests

[ https://issues.apache.org/jira/browse/SPARK-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28342. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via

[jira] [Assigned] (SPARK-28342) Replace REL_12_BETA1 to REL_12_BETA2 in PostgresSQL SQL tests

[ https://issues.apache.org/jira/browse/SPARK-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28342: Assignee: Apache Spark (was: Hyukjin Kwon) > Replace REL_12_BETA1 to REL_12_BETA2 in

[jira] [Assigned] (SPARK-28342) Replace REL_12_BETA1 to REL_12_BETA2 in PostgresSQL SQL tests

[ https://issues.apache.org/jira/browse/SPARK-28342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28342: Assignee: Hyukjin Kwon (was: Apache Spark) > Replace REL_12_BETA1 to REL_12_BETA2 in

[jira] [Created] (SPARK-28342) Replace REL_12_BETA1 to REL_12_BETA2 in PostgresSQL SQL tests

Hyukjin Kwon created SPARK-28342: Summary: Replace REL_12_BETA1 to REL_12_BETA2 in PostgresSQL SQL tests Key: SPARK-28342 URL: https://issues.apache.org/jira/browse/SPARK-28342 Project: Spark

[jira] [Assigned] (SPARK-28341) remove session catalog config

[ https://issues.apache.org/jira/browse/SPARK-28341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28341: Assignee: Wenchen Fan (was: Apache Spark) > remove session catalog config >

[jira] [Assigned] (SPARK-28341) remove session catalog config

[ https://issues.apache.org/jira/browse/SPARK-28341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28341: Assignee: Apache Spark (was: Wenchen Fan) > remove session catalog config >

[jira] [Created] (SPARK-28341) remove session catalog config

Wenchen Fan created SPARK-28341: --- Summary: remove session catalog config Key: SPARK-28341 URL: https://issues.apache.org/jira/browse/SPARK-28341 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-28327) Spark SQL can't support union with left query have queryOrganization

[ https://issues.apache.org/jira/browse/SPARK-28327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882581#comment-16882581 ] angerszhu commented on SPARK-28327: --- [~yumwang]Thank you for you Seems current SparkSQL's SQL

[jira] [Resolved] (SPARK-28339) Rename Spark SQL adaptive execution configuration name

[ https://issues.apache.org/jira/browse/SPARK-28339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28339. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25102

[jira] [Assigned] (SPARK-28339) Rename Spark SQL adaptive execution configuration name

[ https://issues.apache.org/jira/browse/SPARK-28339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28339: --- Assignee: Carson Wang > Rename Spark SQL adaptive execution configuration name >

[jira] [Commented] (SPARK-28272) Convert and port 'pgSQL/aggregates_part3.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882568#comment-16882568 ] Hyukjin Kwon commented on SPARK-28272: -- Argh, sorry. It's blocked by SPARK-27988. > Convert and

[jira] [Resolved] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28015. --- Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 3.0.0 This is

[jira] [Resolved] (SPARK-27919) DataSourceV2: Add v2 session catalog

[ https://issues.apache.org/jira/browse/SPARK-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27919. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24768

[jira] [Assigned] (SPARK-28270) Convert and port 'pgSQL/aggregates_part1.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28270: Assignee: Hyukjin Kwon > Convert and port 'pgSQL/aggregates_part1.sql' into UDF test

[jira] [Resolved] (SPARK-28270) Convert and port 'pgSQL/aggregates_part1.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28270. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25069

[jira] [Assigned] (SPARK-27919) DataSourceV2: Add v2 session catalog

[ https://issues.apache.org/jira/browse/SPARK-27919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27919: --- Assignee: Ryan Blue > DataSourceV2: Add v2 session catalog >

[jira] [Assigned] (SPARK-28281) Convert and port 'having.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28281: Assignee: Huaxin Gao (was: Hyukjin Kwon) > Convert and port 'having.sql' into UDF test

[jira] [Assigned] (SPARK-28281) Convert and port 'having.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28281: Assignee: Hyukjin Kwon > Convert and port 'having.sql' into UDF test base >

[jira] [Resolved] (SPARK-28107) Interval type conversion syntax support

[ https://issues.apache.org/jira/browse/SPARK-28107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28107. --- Resolution: Fixed Assignee: Zhu, Lipeng Fix Version/s: 3.0.0 This is

[jira] [Resolved] (SPARK-28281) Convert and port 'having.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28281. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25093

[jira] [Assigned] (SPARK-28285) Convert and port 'outer-join.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28285: Assignee: Apache Spark > Convert and port 'outer-join.sql' into UDF test base >

[jira] [Resolved] (SPARK-28271) Convert and port 'pgSQL/aggregates_part2.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28271. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25086

[jira] [Assigned] (SPARK-28271) Convert and port 'pgSQL/aggregates_part2.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28271: Assignee: Terry Kim > Convert and port 'pgSQL/aggregates_part2.sql' into UDF test base >

[jira] [Resolved] (SPARK-28275) Convert and port 'count.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28275. -- Resolution: Fixed Assignee: Vinod KC Fix Version/s: 3.0.0 Fixed at 

[jira] [Resolved] (SPARK-28323) PythonUDF should be able to use in join condition

[ https://issues.apache.org/jira/browse/SPARK-28323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28323. -- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 3.0.0 Fixed at 

[jira] [Resolved] (SPARK-27922) Convert and port 'natural-join.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-27922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27922. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25088

[jira] [Assigned] (SPARK-27922) Convert and port 'natural-join.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-27922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-27922: Assignee: Manu Zhang > Convert and port 'natural-join.sql' into UDF test base >

[jira] [Resolved] (SPARK-28234) Spark Resources - add python support to get resources

[ https://issues.apache.org/jira/browse/SPARK-28234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28234. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25087

[jira] [Comment Edited] (SPARK-27991) ShuffleBlockFetcherIterator should take Netty constant-factor overheads into account when limiting number of simultaneous block fetches

[ https://issues.apache.org/jira/browse/SPARK-27991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882532#comment-16882532 ] Josh Rosen edited comment on SPARK-27991 at 7/11/19 12:28 AM: -- I've tried

[jira] [Commented] (SPARK-27991) ShuffleBlockFetcherIterator should take Netty constant-factor overheads into account when limiting number of simultaneous block fetches

[ https://issues.apache.org/jira/browse/SPARK-27991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882532#comment-16882532 ] Josh Rosen commented on SPARK-27991: I've tried to come up with a standalone reproduction of this

[jira] [Commented] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882524#comment-16882524 ] Yuming Wang commented on SPARK-28015: - Thank you [~dongjoon] > Check stringToDate() consumes entire

[jira] [Created] (SPARK-28340) Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException"

Josh Rosen created SPARK-28340: -- Summary: Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException" Key: SPARK-28340

[jira] [Updated] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't have quotes in data

[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-28338: Description: The csv input file +cat sample.csv+ Name,Lastname,Age abc,,32 pqr,xxx,30  

[jira] [Updated] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't have quotes in data

[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-28338: Summary: spark.read.format("csv") treat empty string as null if csv file don't have quotes in

[jira] [Assigned] (SPARK-28339) Rename Spark SQL adaptive execution configuration name

[ https://issues.apache.org/jira/browse/SPARK-28339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28339: Assignee: (was: Apache Spark) > Rename Spark SQL adaptive execution configuration

[jira] [Assigned] (SPARK-28339) Rename Spark SQL adaptive execution configuration name

[ https://issues.apache.org/jira/browse/SPARK-28339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28339: Assignee: Apache Spark > Rename Spark SQL adaptive execution configuration name >

[jira] [Created] (SPARK-28339) Rename Spark SQL adaptive execution configuration name

Carson Wang created SPARK-28339: --- Summary: Rename Spark SQL adaptive execution configuration name Key: SPARK-28339 URL: https://issues.apache.org/jira/browse/SPARK-28339 Project: Spark Issue

[jira] [Updated] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28015: -- Labels: correctness (was: ) > Check stringToDate() consumes entire input for the and

[jira] [Updated] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28015: -- Description: Invalid date formats should throw an exception: {code:sql} SELECT date '1999 08

[jira] [Commented] (SPARK-28266) data correctness issue: data duplication when `path` serde property is present

[ https://issues.apache.org/jira/browse/SPARK-28266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882381#comment-16882381 ] Ruslan Dautkhanov commented on SPARK-28266: --- This issue happens `spark.sql.sources.provider`

[jira] [Assigned] (SPARK-28277) Convert and port 'except.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28277: Assignee: Apache Spark > Convert and port 'except.sql' into UDF test base >

[jira] [Assigned] (SPARK-28277) Convert and port 'except.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28277: Assignee: (was: Apache Spark) > Convert and port 'except.sql' into UDF test base >

[jira] [Updated] (SPARK-28199) Move Trigger implementations to Triggers.scala and avoid exposing these to the end users

[ https://issues.apache.org/jira/browse/SPARK-28199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-28199: - Description: Even ProcessingTime is deprecated in 2.2.0, it's being used in Spark codebase,

[jira] [Updated] (SPARK-28199) Move Trigger implementations to Triggers.scala and avoid exposing these to the end users

[ https://issues.apache.org/jira/browse/SPARK-28199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-28199: - Summary: Move Trigger implementations to Triggers.scala and avoid exposing these to the end

[jira] [Commented] (SPARK-28324) The LOG function using 10 as the base, but Spark using E

[ https://issues.apache.org/jira/browse/SPARK-28324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882288#comment-16882288 ] Sean Owen commented on SPARK-28324: --- I don't think we should change this as it will break code and

[jira] [Commented] (SPARK-4591) Algorithm/model parity for spark.ml (Scala)

[ https://issues.apache.org/jira/browse/SPARK-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882286#comment-16882286 ] Sean Owen commented on SPARK-4591: -- What else would go under this umbrella? > Algorithm/model parity

[jira] [Updated] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28015: -- Affects Version/s: 1.6.3 > Check stringToDate() consumes entire input for the and

[jira] [Comment Edited] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882275#comment-16882275 ] Dongjoon Hyun edited comment on SPARK-28015 at 7/10/19 5:21 PM: I added

[jira] [Resolved] (SPARK-24462) Text socket micro-batch reader throws error when a query is restarted with saved state

[ https://issues.apache.org/jira/browse/SPARK-24462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24462. --- Resolution: Duplicate > Text socket micro-batch reader throws error when a query is restarted with

[jira] [Commented] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882280#comment-16882280 ] Dongjoon Hyun commented on SPARK-28015: --- Hi, [~yumwang]. I updated the JIRA title according to the

[jira] [Updated] (SPARK-28015) Check stringToDate() consumes entire input for the yyyy and yyyy-[m]m formats

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28015: -- Summary: Check stringToDate() consumes entire input for the and -[m]m formats (was:

[jira] [Comment Edited] (SPARK-28015) Invalid date formats should throw an exception

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882275#comment-16882275 ] Dongjoon Hyun edited comment on SPARK-28015 at 7/10/19 5:08 PM: I added

[jira] [Updated] (SPARK-28015) Invalid date formats should throw an exception

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28015: -- Affects Version/s: 2.0.2 2.1.3 2.2.3 > Invalid

[jira] [Commented] (SPARK-28015) Invalid date formats should throw an exception

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882275#comment-16882275 ] Dongjoon Hyun commented on SPARK-28015: --- I added `2.0~2.3`, too. {code} scala> sql("SELECT

[jira] [Updated] (SPARK-28015) Invalid date formats should throw an exception

[ https://issues.apache.org/jira/browse/SPARK-28015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28015: -- Affects Version/s: 2.3.3 > Invalid date formats should throw an exception >

[jira] [Resolved] (SPARK-28335) Flaky test: org.apache.spark.streaming.kafka010.DirectKafkaStreamSuite.offset recovery from kafka

[ https://issues.apache.org/jira/browse/SPARK-28335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28335. --- Resolution: Fixed Fix Version/s: 2.4.4 2.3.4

[jira] [Assigned] (SPARK-28335) Flaky test: org.apache.spark.streaming.kafka010.DirectKafkaStreamSuite.offset recovery from kafka

[ https://issues.apache.org/jira/browse/SPARK-28335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28335: - Assignee: Gabor Somogyi > Flaky test:

[jira] [Resolved] (SPARK-28290) Use `SslContextFactory.Server` instead of `SslContextFactory`

[ https://issues.apache.org/jira/browse/SPARK-28290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28290. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25067

[jira] [Assigned] (SPARK-28290) Use `SslContextFactory.Server` instead of `SslContextFactory`

[ https://issues.apache.org/jira/browse/SPARK-28290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28290: - Assignee: Dongjoon Hyun > Use `SslContextFactory.Server` instead of

[jira] [Commented] (SPARK-28266) data correctness issue: data duplication when `path` serde property is present

[ https://issues.apache.org/jira/browse/SPARK-28266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882200#comment-16882200 ] Ruslan Dautkhanov commented on SPARK-28266: --- Suspecting change in SPARK-22158 causes this  >

[jira] [Comment Edited] (SPARK-28280) Convert and port 'group-by.sql' into UDF test base

[ https://issues.apache.org/jira/browse/SPARK-28280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881970#comment-16881970 ] Stavros Kontopoulos edited comment on SPARK-28280 at 7/10/19 3:31 PM:

[jira] [Resolved] (SPARK-27560) HashPartitioner uses Object.hashCode which is not seeded

[ https://issues.apache.org/jira/browse/SPARK-27560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27560. --- Resolution: Not A Problem > HashPartitioner uses Object.hashCode which is not seeded >

[jira] [Resolved] (SPARK-26440) Show total CPU time across all tasks on stage pages

[ https://issues.apache.org/jira/browse/SPARK-26440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26440. --- Resolution: Won't Fix > Show total CPU time across all tasks on stage pages >

[jira] [Resolved] (SPARK-26497) Show users where the pre-packaged SparkR and PySpark Dockerfiles are in the image build script.

[ https://issues.apache.org/jira/browse/SPARK-26497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26497. --- Resolution: Later > Show users where the pre-packaged SparkR and PySpark Dockerfiles are in the >

[jira] [Resolved] (SPARK-26097) Show partitioning details in DAG UI

[ https://issues.apache.org/jira/browse/SPARK-26097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26097. --- Resolution: Later > Show partitioning details in DAG UI > --- > >

[jira] [Updated] (SPARK-26097) Show partitioning details in DAG UI

[ https://issues.apache.org/jira/browse/SPARK-26097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26097: -- Priority: Minor (was: Major) This can be reopened with a PR that would address the different

[jira] [Assigned] (SPARK-28310) ANSI SQL grammar support: first_value/last_value(expression, [RESPECT NULLS | IGNORE NULLS])

[ https://issues.apache.org/jira/browse/SPARK-28310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28310: - Assignee: Zhu, Lipeng > ANSI SQL grammar support: first_value/last_value(expression,

[jira] [Resolved] (SPARK-28310) ANSI SQL grammar support: first_value/last_value(expression, [RESPECT NULLS | IGNORE NULLS])

[ https://issues.apache.org/jira/browse/SPARK-28310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28310. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25082

[jira] [Commented] (SPARK-28327) Spark SQL can't support union with left query have queryOrganization

[ https://issues.apache.org/jira/browse/SPARK-28327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882109#comment-16882109 ] Yuming Wang commented on SPARK-28327: - PostgreSQL also does not support this: {code:sql} postgres=#

[jira] [Assigned] (SPARK-28335) Flaky test: org.apache.spark.streaming.kafka010.DirectKafkaStreamSuite.offset recovery from kafka

[ https://issues.apache.org/jira/browse/SPARK-28335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28335: Assignee: (was: Apache Spark) > Flaky test:

[jira] [Assigned] (SPARK-28335) Flaky test: org.apache.spark.streaming.kafka010.DirectKafkaStreamSuite.offset recovery from kafka

[ https://issues.apache.org/jira/browse/SPARK-28335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-28335: Assignee: Apache Spark > Flaky test:

[jira] [Resolved] (SPARK-28294) Support `spark.history.fs.cleaner.maxNum` configuration

[ https://issues.apache.org/jira/browse/SPARK-28294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28294. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25072

[jira] [Assigned] (SPARK-28294) Support `spark.history.fs.cleaner.maxNum` configuration

[ https://issues.apache.org/jira/browse/SPARK-28294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-28294: - Assignee: Dongjoon Hyun > Support `spark.history.fs.cleaner.maxNum` configuration >

[jira] [Updated] (SPARK-28199) Remove usage of ProcessingTime in Spark codebase

[ https://issues.apache.org/jira/browse/SPARK-28199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28199: -- Labels: release-notes (was: ) > Remove usage of ProcessingTime in Spark codebase >

[jira] [Commented] (SPARK-28234) Spark Resources - add python support to get resources

[ https://issues.apache.org/jira/browse/SPARK-28234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882092#comment-16882092 ] Thomas Graves commented on SPARK-28234: --- Testing driver side: {code:java} >>>

[jira] [Updated] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't quotes in data

[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayadevan M updated SPARK-28338: Summary: spark.read.format("csv") treat empty string as null if csv file don't quotes in data

[jira] [Created] (SPARK-28338) spark.read.format("csv") treat empty string as null if csv file don't quotes in columns

Jayadevan M created SPARK-28338: --- Summary: spark.read.format("csv") treat empty string as null if csv file don't quotes in columns Key: SPARK-28338 URL: https://issues.apache.org/jira/browse/SPARK-28338

[jira] [Assigned] (SPARK-28267) Update building-spark.md

[ https://issues.apache.org/jira/browse/SPARK-28267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28267: - Assignee: Yuming Wang > Update building-spark.md > > >

[jira] [Resolved] (SPARK-28267) Update building-spark.md

[ https://issues.apache.org/jira/browse/SPARK-28267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28267. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 25063

  1   2   >