[jira] [Updated] (SPARK-26576) Broadcast hint not applied to partitioned table

[ https://issues.apache.org/jira/browse/SPARK-26576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-26576: --- Summary: Broadcast hint not applied to partitioned table (was: Broadcast hint not applied to

[jira] [Commented] (SPARK-26491) Use ConfigEntry for hardcoded configs for test categories.

[ https://issues.apache.org/jira/browse/SPARK-26491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739167#comment-16739167 ] Dongjoon Hyun commented on SPARK-26491: --- The broken K8S integration compilation is fixed via

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739083#comment-16739083 ] deshanxiao commented on SPARK-26570: [~hyukjin.kwon] OK, I will try it. Thank you! > Out of memory

[jira] [Created] (SPARK-26586) Streaming queries should have isolated SparkSessions and confs

Mukul Murthy created SPARK-26586: Summary: Streaming queries should have isolated SparkSessions and confs Key: SPARK-26586 URL: https://issues.apache.org/jira/browse/SPARK-26586 Project: Spark

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738955#comment-16738955 ] Hyukjin Kwon commented on SPARK-26570: -- Would you be able to test this in upper version of Spark?

[jira] [Resolved] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26574. -- Resolution: Invalid > Cloud sql stronge > - > > Key:

[jira] [Updated] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26574: - Fix Version/s: (was: 0.8.2) > Cloud sql stronge > - > >

[jira] [Commented] (SPARK-26585) [K8S] Add additional integration tests for K8s Scheduler Backend

[ https://issues.apache.org/jira/browse/SPARK-26585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738956#comment-16738956 ] Nagaram Prasad Addepally commented on SPARK-26585: --

[jira] [Assigned] (SPARK-26585) [K8S] Add additional integration tests for K8s Scheduler Backend

[ https://issues.apache.org/jira/browse/SPARK-26585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26585: Assignee: (was: Apache Spark) > [K8S] Add additional integration tests for K8s

[jira] [Assigned] (SPARK-26585) [K8S] Add additional integration tests for K8s Scheduler Backend

[ https://issues.apache.org/jira/browse/SPARK-26585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26585: Assignee: Apache Spark > [K8S] Add additional integration tests for K8s Scheduler

[jira] [Commented] (SPARK-26579) SparkML DecisionTree, how does the algorithm identify categorical features?

[ https://issues.apache.org/jira/browse/SPARK-26579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738954#comment-16738954 ] Hyukjin Kwon commented on SPARK-26579: -- Let's ask question to mailing list rather then filing a

[jira] [Resolved] (SPARK-26579) SparkML DecisionTree, how does the algorithm identify categorical features?

[ https://issues.apache.org/jira/browse/SPARK-26579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26579. -- Resolution: Invalid > SparkML DecisionTree, how does the algorithm identify categorical

[jira] [Updated] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26574: - Flags: (was: Patch,Important) > Cloud sql stronge > - > >

[jira] [Updated] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26574: - External issue URL: (was: https://pakegecloud.atlassian.net) > Cloud sql stronge >

[jira] [Updated] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26574: - Labels: (was: PA) > Cloud sql stronge > - > > Key:

[jira] [Commented] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738952#comment-16738952 ] Hyukjin Kwon commented on SPARK-26574: -- Please fill the JIRA description, and reopen. > Cloud sql

[jira] [Updated] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26574: - External issue ID: (was: roufi...@rtat.net) > Cloud sql stronge > - > >

[jira] [Updated] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26574: - Component/s: (was: jenkins) > Cloud sql stronge > - > >

[jira] [Updated] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26574: - Shepherd: (was: pakegecloud.atlassian.net) > Cloud sql stronge > - > >

[jira] [Updated] (SPARK-26574) Cloud sql stronge

[ https://issues.apache.org/jira/browse/SPARK-26574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26574: - Target Version/s: (was: 2.4.0) > Cloud sql stronge > - > >

[jira] [Resolved] (SPARK-26581) Spark Dataset write JSON with Multiline

[ https://issues.apache.org/jira/browse/SPARK-26581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26581. -- Resolution: Invalid Also, multiline concept is not applicable to write side. Let's also ask

[jira] [Commented] (SPARK-26581) Spark Dataset write JSON with Multiline

[ https://issues.apache.org/jira/browse/SPARK-26581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738948#comment-16738948 ] Hyukjin Kwon commented on SPARK-26581: -- {{multiline}} is not supported in write option. You can

[jira] [Commented] (SPARK-26585) [K8S] Add additional integration tests for K8s Scheduler Backend

[ https://issues.apache.org/jira/browse/SPARK-26585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738944#comment-16738944 ] Nagaram Prasad Addepally commented on SPARK-26585: -- I am working on adding these tests.

[jira] [Created] (SPARK-26585) [K8S] Add additional integration tests for K8s Scheduler Backend

Nagaram Prasad Addepally created SPARK-26585: Summary: [K8S] Add additional integration tests for K8s Scheduler Backend Key: SPARK-26585 URL: https://issues.apache.org/jira/browse/SPARK-26585

[jira] [Comment Edited] (SPARK-10781) Allow certain number of failed tasks and allow job to succeed

[ https://issues.apache.org/jira/browse/SPARK-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738932#comment-16738932 ] nxet edited comment on SPARK-10781 at 1/10/19 2:52 AM: --- I met the same problem as

[jira] [Commented] (SPARK-10781) Allow certain number of failed tasks and allow job to succeed

[ https://issues.apache.org/jira/browse/SPARK-10781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738932#comment-16738932 ] nxet commented on SPARK-10781: -- I met the same problem as some empty sequence files cause the failure of

[jira] [Resolved] (SPARK-26546) Caching of DateTimeFormatter

[ https://issues.apache.org/jira/browse/SPARK-26546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26546. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23462

[jira] [Assigned] (SPARK-26546) Caching of DateTimeFormatter

[ https://issues.apache.org/jira/browse/SPARK-26546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26546: Assignee: Maxim Gekk > Caching of DateTimeFormatter > > >

[jira] [Resolved] (SPARK-26493) spark.sql.extensions should support multiple extensions

[ https://issues.apache.org/jira/browse/SPARK-26493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26493. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23398

[jira] [Assigned] (SPARK-26493) spark.sql.extensions should support multiple extensions

[ https://issues.apache.org/jira/browse/SPARK-26493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26493: Assignee: Jamison Bennett > spark.sql.extensions should support multiple extensions >

[jira] [Assigned] (SPARK-26584) Remove `spark.sql.orc.copyBatchToSpark` internal configuration

[ https://issues.apache.org/jira/browse/SPARK-26584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26584: Assignee: (was: Apache Spark) > Remove `spark.sql.orc.copyBatchToSpark` internal

[jira] [Assigned] (SPARK-26584) Remove `spark.sql.orc.copyBatchToSpark` internal configuration

[ https://issues.apache.org/jira/browse/SPARK-26584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26584: Assignee: Apache Spark > Remove `spark.sql.orc.copyBatchToSpark` internal configuration

[jira] [Created] (SPARK-26584) Remove `spark.sql.orc.copyBatchToSpark` internal configuration

Dongjoon Hyun created SPARK-26584: - Summary: Remove `spark.sql.orc.copyBatchToSpark` internal configuration Key: SPARK-26584 URL: https://issues.apache.org/jira/browse/SPARK-26584 Project: Spark

[jira] [Updated] (SPARK-26583) Add `paranamer` dependency to `core` module

[ https://issues.apache.org/jira/browse/SPARK-26583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26583: -- Component/s: Build > Add `paranamer` dependency to `core` module >

[jira] [Updated] (SPARK-26583) Add `paranamer` dependency to `core` module

[ https://issues.apache.org/jira/browse/SPARK-26583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26583: -- Priority: Major (was: Blocker) > Add `paranamer` dependency to `core` module >

[jira] [Updated] (SPARK-26583) Add `paranamer` dependency to `core` module

[ https://issues.apache.org/jira/browse/SPARK-26583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26583: -- Target Version/s: 2.4.1, 3.0.0 > Add `paranamer` dependency to `core` module >

[jira] [Updated] (SPARK-26583) Add `paranamer` dependency to `core` module

[ https://issues.apache.org/jira/browse/SPARK-26583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26583: -- Description: With Scala-2.12 profile, Spark application fails while Spark is okay. For

[jira] [Commented] (SPARK-26583) Add `paranamer` to `core` module

[ https://issues.apache.org/jira/browse/SPARK-26583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738746#comment-16738746 ] Dongjoon Hyun commented on SPARK-26583: --- This is a minor issue, but this should be fixed before

[jira] [Updated] (SPARK-26583) Add `paranamer` dependency to `core` module

[ https://issues.apache.org/jira/browse/SPARK-26583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26583: -- Summary: Add `paranamer` dependency to `core` module (was: Add `paranamer` to `core` module)

[jira] [Created] (SPARK-26583) Add `paranamer` to `core` module

Dongjoon Hyun created SPARK-26583: - Summary: Add `paranamer` to `core` module Key: SPARK-26583 URL: https://issues.apache.org/jira/browse/SPARK-26583 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-26583) Add `paranamer` to `core` module

[ https://issues.apache.org/jira/browse/SPARK-26583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26583: -- Priority: Blocker (was: Major) > Add `paranamer` to `core` module >

[jira] [Updated] (SPARK-26576) Broadcast hint not applied to partitioned Parquet table

[ https://issues.apache.org/jira/browse/SPARK-26576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-26576: --- Affects Version/s: 2.2.2 > Broadcast hint not applied to partitioned Parquet table >

[jira] [Resolved] (SPARK-26448) retain the difference between 0.0 and -0.0

[ https://issues.apache.org/jira/browse/SPARK-26448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26448. - Resolution: Fixed Fix Version/s: 3.0.0 > retain the difference between 0.0 and -0.0 >

[jira] [Created] (SPARK-26582) Update the optimizer rule NormalizeFloatingNumbers

Xiao Li created SPARK-26582: --- Summary: Update the optimizer rule NormalizeFloatingNumbers Key: SPARK-26582 URL: https://issues.apache.org/jira/browse/SPARK-26582 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24374) SPIP: Support Barrier Execution Mode in Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738622#comment-16738622 ] Thomas Graves commented on SPARK-24374: --- [~luzengxiang] are you just saying when spark tries to

[jira] [Resolved] (SPARK-26414) Race between SparkContext and YARN AM can cause NPE in UI setup code

[ https://issues.apache.org/jira/browse/SPARK-26414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26414. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 3.0.0 > Race

[jira] [Commented] (SPARK-26414) Race between SparkContext and YARN AM can cause NPE in UI setup code

[ https://issues.apache.org/jira/browse/SPARK-26414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738580#comment-16738580 ] Marcelo Vanzin commented on SPARK-26414: I actually ended up fixing this in the change for

[jira] [Resolved] (SPARK-25484) Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25484. --- Resolution: Fixed Assignee: Peter Toth Fix Version/s: 3.0.0 This is

[jira] [Comment Edited] (SPARK-26576) Broadcast hint not applied to partitioned Parquet table

[ https://issues.apache.org/jira/browse/SPARK-26576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737964#comment-16737964 ] John Zhuge edited comment on SPARK-26576 at 1/9/19 5:12 PM: No issue on the

[jira] [Assigned] (SPARK-26254) Move delegation token providers into a separate project

[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26254: Assignee: Apache Spark > Move delegation token providers into a separate project >

[jira] [Assigned] (SPARK-26254) Move delegation token providers into a separate project

[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26254: Assignee: (was: Apache Spark) > Move delegation token providers into a separate

[jira] [Created] (SPARK-26581) Spark Dataset write JSON with Multiline

Anil created SPARK-26581: Summary: Spark Dataset write JSON with Multiline Key: SPARK-26581 URL: https://issues.apache.org/jira/browse/SPARK-26581 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-25433) Add support for PEX in PySpark

[ https://issues.apache.org/jira/browse/SPARK-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738190#comment-16738190 ] Fabian Höring commented on SPARK-25433: --- [~dongjoon] [~hyukjin.kwon] If you think the blog post is

[jira] [Commented] (SPARK-25433) Add support for PEX in PySpark

[ https://issues.apache.org/jira/browse/SPARK-25433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738125#comment-16738125 ] Dongjoon Hyun commented on SPARK-25433: --- Thank you for the pointer, [~hyukjin.kwon] and

[jira] [Assigned] (SPARK-26580) remove Scala 2.11 hack for Scala UDF

[ https://issues.apache.org/jira/browse/SPARK-26580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26580: Assignee: Wenchen Fan (was: Apache Spark) > remove Scala 2.11 hack for Scala UDF >

[jira] [Assigned] (SPARK-26580) remove Scala 2.11 hack for Scala UDF

[ https://issues.apache.org/jira/browse/SPARK-26580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26580: Assignee: Apache Spark (was: Wenchen Fan) > remove Scala 2.11 hack for Scala UDF >

[jira] [Updated] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar Bonilla updated SPARK-26365: -- Affects Version/s: 2.4.0 > spark-submit for k8s cluster doesn't propagate exit code >

[jira] [Commented] (SPARK-26576) Broadcast hint not applied to partitioned Parquet table

[ https://issues.apache.org/jira/browse/SPARK-26576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737964#comment-16737964 ] John Zhuge commented on SPARK-26576: No issue on the master branch. Please note