[jira] [Comment Edited] (SPARK-25807) Mitigate 1-based substr() confusion

[ https://issues.apache.org/jira/browse/SPARK-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663332#comment-16663332 ] kevin yu edited comment on SPARK-25807 at 10/25/18 6:43 AM:

[jira] [Resolved] (SPARK-24572) "eager execution" for R shell, IDE

[ https://issues.apache.org/jira/browse/SPARK-24572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-24572. -- Resolution: Fixed Assignee: Weiqiang Zhuang Fix Version/s: 3.0.0 T

[jira] [Comment Edited] (SPARK-25807) Mitigate 1-based substr() confusion

[ https://issues.apache.org/jira/browse/SPARK-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663332#comment-16663332 ] kevin yu edited comment on SPARK-25807 at 10/25/18 6:42 AM:

[jira] [Commented] (SPARK-25807) Mitigate 1-based substr() confusion

[ https://issues.apache.org/jira/browse/SPARK-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663332#comment-16663332 ] kevin yu commented on SPARK-25807: -- [~oron.navon]: You can also try to implement a simp

[jira] [Resolved] (SPARK-24516) PySpark Bindings for K8S - make Python 3 the default

[ https://issues.apache.org/jira/browse/SPARK-24516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-24516. -- Resolution: Fixed Assignee: Ilan Filonenko Fix Version/s: 3.0.0 Ta

[jira] [Commented] (SPARK-21928) ClassNotFoundException for custom Kryo registrator class during serde in netty threads

[ https://issues.apache.org/jira/browse/SPARK-21928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663313#comment-16663313 ] Sourav Gulati commented on SPARK-21928: --- I am using Spark 2.30 version and I am st

[jira] [Updated] (SPARK-25832) remove newly added map related functions from FunctionRegistry

[ https://issues.apache.org/jira/browse/SPARK-25832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25832: -- Priority: Blocker (was: Major) > remove newly added map related functions from FunctionRegist

[jira] [Assigned] (SPARK-25832) remove newly added map related functions from FunctionRegistry

[ https://issues.apache.org/jira/browse/SPARK-25832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25832: Assignee: Apache Spark (was: Wenchen Fan) > remove newly added map related functions fro

[jira] [Assigned] (SPARK-25832) remove newly added map related functions from FunctionRegistry

[ https://issues.apache.org/jira/browse/SPARK-25832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25832: Assignee: Wenchen Fan (was: Apache Spark) > remove newly added map related functions fro

[jira] [Commented] (SPARK-25832) remove newly added map related functions from FunctionRegistry

[ https://issues.apache.org/jira/browse/SPARK-25832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663233#comment-16663233 ] Apache Spark commented on SPARK-25832: -- User 'cloud-fan' has created a pull request

[jira] [Created] (SPARK-25832) remove newly added map related functions from FunctionRegistry

Wenchen Fan created SPARK-25832: --- Summary: remove newly added map related functions from FunctionRegistry Key: SPARK-25832 URL: https://issues.apache.org/jira/browse/SPARK-25832 Project: Spark

[jira] [Commented] (SPARK-25829) Duplicated map keys are not handled consistently

[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663230#comment-16663230 ] Dongjoon Hyun commented on SPARK-25829: --- Thank you for further investigation! Both

[jira] [Commented] (SPARK-25829) Duplicated map keys are not handled consistently

[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663121#comment-16663121 ] Wenchen Fan commented on SPARK-25829: - If we decide to follow "later entry wins", th

[jira] [Commented] (SPARK-25829) Duplicated map keys are not handled consistently

[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663118#comment-16663118 ] Wenchen Fan commented on SPARK-25829: - More investigation on "later entry wins". If

[jira] [Comment Edited] (SPARK-25829) Duplicated map keys are not handled consistently

[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663103#comment-16663103 ] Wenchen Fan edited comment on SPARK-25829 at 10/25/18 2:20 AM: ---

[jira] [Commented] (SPARK-25829) Duplicated map keys are not handled consistently

[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663103#comment-16663103 ] Wenchen Fan commented on SPARK-25829: - After more thoughts, both the map lookup beha

[jira] [Resolved] (SPARK-24917) Sending a partition over netty results in 2x memory usage

[ https://issues.apache.org/jira/browse/SPARK-24917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24917. --- Resolution: Won't Fix Evidently obsoleted by the update to Netty 4.1.30 > Sending a partition over

[jira] [Updated] (SPARK-25830) should apply "earlier entry wins" in Dataset.collect

[ https://issues.apache.org/jira/browse/SPARK-25830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25830: Description: {code} scala> sql("select map(1,2,1,3)").collect res14: Array[org.apache.spark.sql.Ro

[jira] [Updated] (SPARK-25831) should apply "earlier entry wins" in hive map value converter

[ https://issues.apache.org/jira/browse/SPARK-25831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25831: Description: {code} scala> sql("CREATE TABLE t AS SELECT map(1,2,1,3) as map") res11: org.apache.s

[jira] [Created] (SPARK-25831) should apply "earlier entry wins" in hive map value converter

Wenchen Fan created SPARK-25831: --- Summary: should apply "earlier entry wins" in hive map value converter Key: SPARK-25831 URL: https://issues.apache.org/jira/browse/SPARK-25831 Project: Spark

[jira] [Created] (SPARK-25830) should apply "earlier entry wins" in Dataset.collect

Wenchen Fan created SPARK-25830: --- Summary: should apply "earlier entry wins" in Dataset.collect Key: SPARK-25830 URL: https://issues.apache.org/jira/browse/SPARK-25830 Project: Spark Issue Type

[jira] [Updated] (SPARK-25824) Remove duplicated map entries in `showString`

[ https://issues.apache.org/jira/browse/SPARK-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25824: Issue Type: Sub-task (was: Bug) Parent: SPARK-25829 > Remove duplicated map entries in `s

[jira] [Updated] (SPARK-25829) Duplicated map keys are not handled consistently

[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25829: Description: In Spark SQL, we apply "earlier entry wins" semantic to duplicated map keys. e.g. {c

[jira] [Updated] (SPARK-25829) Duplicated map keys are not handled consistently

[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25829: Description: In Spark SQL, we apply "earlier entry wins" semantic to duplicated map keys. e.g. {c

[jira] [Created] (SPARK-25829) Duplicated map keys are not handled consistently

Wenchen Fan created SPARK-25829: --- Summary: Duplicated map keys are not handled consistently Key: SPARK-25829 URL: https://issues.apache.org/jira/browse/SPARK-25829 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-25824) Remove duplicated map entries in `showString`

[ https://issues.apache.org/jira/browse/SPARK-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663060#comment-16663060 ] Dongjoon Hyun commented on SPARK-25824: --- According to [~cloud_fan]'s analysis, thi

[jira] [Updated] (SPARK-25824) Remove duplicated map entries in `showString`

[ https://issues.apache.org/jira/browse/SPARK-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25824: -- Issue Type: Bug (was: Improvement) > Remove duplicated map entries in `showString` >

[jira] [Commented] (SPARK-25828) Bumping Version of kubernetes.client to latest version

[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663045#comment-16663045 ] Apache Spark commented on SPARK-25828: -- User 'ifilonenko' has created a pull reques

[jira] [Commented] (SPARK-25828) Bumping Version of kubernetes.client to latest version

[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663043#comment-16663043 ] Apache Spark commented on SPARK-25828: -- User 'ifilonenko' has created a pull reques

[jira] [Assigned] (SPARK-25828) Bumping Version of kubernetes.client to latest version

[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25828: Assignee: Apache Spark > Bumping Version of kubernetes.client to latest version > ---

[jira] [Assigned] (SPARK-25828) Bumping Version of kubernetes.client to latest version

[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25828: Assignee: (was: Apache Spark) > Bumping Version of kubernetes.client to latest versio

[jira] [Commented] (SPARK-25828) Bumping Version of kubernetes.client to latest version

[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16663033#comment-16663033 ] Stavros Kontopoulos commented on SPARK-25828: - Cool [~eje] [~ifilonenko] are

[jira] [Resolved] (SPARK-23229) Dataset.hint should use planWithBarrier logical plan

[ https://issues.apache.org/jira/browse/SPARK-23229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23229. --- Resolution: Won't Fix > Dataset.hint should use planWithBarrier logical plan > -

[jira] [Commented] (SPARK-25828) Bumping Version of kubernetes.client to latest version

[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662978#comment-16662978 ] Erik Erlandson commented on SPARK-25828: cc [~skonto] > Bumping Version of kube

[jira] [Updated] (SPARK-25828) Bumping Version of kubernetes.client to latest version

[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilan Filonenko updated SPARK-25828: --- Description: Upgrade the Kubernetes client version to at least [4.0.0|https://mvnrepository

[jira] [Created] (SPARK-25828) Bumping Version of kubernetes.client to latest version

Ilan Filonenko created SPARK-25828: -- Summary: Bumping Version of kubernetes.client to latest version Key: SPARK-25828 URL: https://issues.apache.org/jira/browse/SPARK-25828 Project: Spark Is

[jira] [Resolved] (SPARK-24899) Add example of monotonically_increasing_id standard function to scaladoc

[ https://issues.apache.org/jira/browse/SPARK-24899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24899. --- Resolution: Won't Fix > Add example of monotonically_increasing_id standard function to scaladoc > -

[jira] [Assigned] (SPARK-25490) Refactor KryoBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25490: - Assignee: Gengliang Wang > Refactor KryoBenchmark > -- > >

[jira] [Resolved] (SPARK-25490) Refactor KryoBenchmark

[ https://issues.apache.org/jira/browse/SPARK-25490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25490. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22663 [https://github.c

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662899#comment-16662899 ] Dongjoon Hyun commented on SPARK-25823: --- Right. That one is cosmetic. I send my op

[jira] [Comment Edited] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662899#comment-16662899 ] Dongjoon Hyun edited comment on SPARK-25823 at 10/24/18 9:42 PM: -

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662891#comment-16662891 ] Sean Owen commented on SPARK-25823: --- I was referring to your comment at https://issue

[jira] [Commented] (SPARK-25807) Mitigate 1-based substr() confusion

[ https://issues.apache.org/jira/browse/SPARK-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662830#comment-16662830 ] kevin yu commented on SPARK-25807: -- Thanks Sean, ok, I will leave as it is.  > Mitigat

[jira] [Comment Edited] (SPARK-23015) spark-submit fails when submitting several jobs in parallel

[ https://issues.apache.org/jira/browse/SPARK-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662750#comment-16662750 ] Kevin Grealish edited comment on SPARK-23015 at 10/24/18 8:51 PM:

[jira] [Assigned] (SPARK-25827) Replicating a block > 2gb with encryption fails

[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25827: Assignee: Apache Spark > Replicating a block > 2gb with encryption fails > --

[jira] [Assigned] (SPARK-25827) Replicating a block > 2gb with encryption fails

[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25827: Assignee: (was: Apache Spark) > Replicating a block > 2gb with encryption fails > ---

[jira] [Commented] (SPARK-25827) Replicating a block > 2gb with encryption fails

[ https://issues.apache.org/jira/browse/SPARK-25827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662825#comment-16662825 ] Apache Spark commented on SPARK-25827: -- User 'squito' has created a pull request fo

[jira] [Updated] (SPARK-25807) Mitigate 1-based substr() confusion

[ https://issues.apache.org/jira/browse/SPARK-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25807: -- Target Version/s: (was: 2.4.0, 2.4.1, 2.5.0, 3.0.0) > Mitigate 1-based substr() confusion >

[jira] [Commented] (SPARK-25807) Mitigate 1-based substr() confusion

[ https://issues.apache.org/jira/browse/SPARK-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662817#comment-16662817 ] Sean Owen commented on SPARK-25807: --- They are meant to match Hive, SQL. They should no

[jira] [Created] (SPARK-25827) Replicating a block > 2gb with encryption fails

Imran Rashid created SPARK-25827: Summary: Replicating a block > 2gb with encryption fails Key: SPARK-25827 URL: https://issues.apache.org/jira/browse/SPARK-25827 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662760#comment-16662760 ] Dongjoon Hyun commented on SPARK-25823: --- For me, this is a `data correctness` issu

[jira] [Commented] (SPARK-25815) Kerberos Support in Kubernetes resource manager (Client Mode)

[ https://issues.apache.org/jira/browse/SPARK-25815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662761#comment-16662761 ] Marcelo Vanzin commented on SPARK-25815: I actually have this working (and also

[jira] [Comment Edited] (SPARK-23015) spark-submit fails when submitting several jobs in parallel

[ https://issues.apache.org/jira/browse/SPARK-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662750#comment-16662750 ] Kevin Grealish edited comment on SPARK-23015 at 10/24/18 7:47 PM:

[jira] [Updated] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25737: -- Description: In ancient times in 2013, JavaSparkContext got a superclass JavaSparkContextVarargsWorka

[jira] [Comment Edited] (SPARK-23015) spark-submit fails when submitting several jobs in parallel

[ https://issues.apache.org/jira/browse/SPARK-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662750#comment-16662750 ] Kevin Grealish edited comment on SPARK-23015 at 10/24/18 7:45 PM:

[jira] [Updated] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25737: -- Labels: release-notes (was: ) > Remove JavaSparkContextVarargsWorkaround and standardize union() meth

[jira] [Resolved] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25737. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22729 [https://github.c

[jira] [Commented] (SPARK-23015) spark-submit fails when submitting several jobs in parallel

[ https://issues.apache.org/jira/browse/SPARK-23015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662750#comment-16662750 ] Kevin Grealish commented on SPARK-23015: One workaround is to create a temp dire

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662743#comment-16662743 ] Dongjoon Hyun commented on SPARK-25823: --- Sorry, but who says this issue is cosmeti

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662737#comment-16662737 ] Sean Owen commented on SPARK-25823: --- Got it, you're saying that's cosmetic. But I unde

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662735#comment-16662735 ] Dongjoon Hyun commented on SPARK-25823: --- [~srowen] and [~cloud_fan]. To be clear,

[jira] [Comment Edited] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662726#comment-16662726 ] Dongjoon Hyun edited comment on SPARK-25823 at 10/24/18 7:31 PM: -

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662726#comment-16662726 ] Dongjoon Hyun commented on SPARK-25823: --- [~srowen]. I commend on SPARK-25824. Plea

[jira] [Commented] (SPARK-25824) Remove duplicated map entries in `showString`

[ https://issues.apache.org/jira/browse/SPARK-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662725#comment-16662725 ] Dongjoon Hyun commented on SPARK-25824: --- [~srowen]. Please see the description. Th

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662717#comment-16662717 ] Sean Owen commented on SPARK-25823: --- Hm, another tough one. It's not so much about wha

[jira] [Updated] (SPARK-25826) Kerberos Support in Kubernetes resource manager

[ https://issues.apache.org/jira/browse/SPARK-25826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilan Filonenko updated SPARK-25826: --- Description: This is the umbrella issue for all Kerberos related tasks with relation to Spar

[jira] [Created] (SPARK-25826) Kerberos Support in Kubernetes

Ilan Filonenko created SPARK-25826: -- Summary: Kerberos Support in Kubernetes Key: SPARK-25826 URL: https://issues.apache.org/jira/browse/SPARK-25826 Project: Spark Issue Type: Umbrella

[jira] [Created] (SPARK-25825) Kerberos Support for Long Running Jobs in Kubernetes

Ilan Filonenko created SPARK-25825: -- Summary: Kerberos Support for Long Running Jobs in Kubernetes Key: SPARK-25825 URL: https://issues.apache.org/jira/browse/SPARK-25825 Project: Spark Iss

[jira] [Assigned] (SPARK-25816) Functions does not resolve Columns correctly

[ https://issues.apache.org/jira/browse/SPARK-25816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25816: Assignee: Apache Spark > Functions does not resolve Columns correctly > -

[jira] [Commented] (SPARK-25816) Functions does not resolve Columns correctly

[ https://issues.apache.org/jira/browse/SPARK-25816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662688#comment-16662688 ] Apache Spark commented on SPARK-25816: -- User 'peter-toth' has created a pull reques

[jira] [Assigned] (SPARK-25816) Functions does not resolve Columns correctly

[ https://issues.apache.org/jira/browse/SPARK-25816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25816: Assignee: (was: Apache Spark) > Functions does not resolve Columns correctly > --

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662673#comment-16662673 ] Dongjoon Hyun commented on SPARK-25823: --- Ah, got it. I thought a different one. Fo

[jira] [Commented] (SPARK-25824) Remove duplicated map entries in `showString`

[ https://issues.apache.org/jira/browse/SPARK-25824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662670#comment-16662670 ] Dongjoon Hyun commented on SPARK-25824: --- cc [~maropu] > Remove duplicated map ent

[jira] [Created] (SPARK-25824) Remove duplicated map entries in `showString`

Dongjoon Hyun created SPARK-25824: - Summary: Remove duplicated map entries in `showString` Key: SPARK-25824 URL: https://issues.apache.org/jira/browse/SPARK-25824 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-14989) Upgrade to Jackson 2.7.3

[ https://issues.apache.org/jira/browse/SPARK-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662668#comment-16662668 ] nirav patel commented on SPARK-14989: - Any updates on this? Spark 2.2 still uses Jac

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662666#comment-16662666 ] Wenchen Fan commented on SPARK-25823: - Anyway we should make map lookup and toString

[jira] [Comment Edited] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662660#comment-16662660 ] Dongjoon Hyun edited comment on SPARK-25823 at 10/24/18 6:38 PM: -

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662660#comment-16662660 ] Dongjoon Hyun commented on SPARK-25823: --- BTW, [~cloud_fan]. I'm looking around thi

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662646#comment-16662646 ] Wenchen Fan commented on SPARK-25823: - [~dongjoon] good catch! I think we should upd

[jira] [Updated] (SPARK-25812) Flaky test: PagedTableSuite.pageNavigation

[ https://issues.apache.org/jira/browse/SPARK-25812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25812: -- Component/s: Tests > Flaky test: PagedTableSuite.pageNavigation >

[jira] [Updated] (SPARK-23415) BufferHolderSparkSubmitSuite is flaky

[ https://issues.apache.org/jira/browse/SPARK-23415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23415: -- Component/s: Tests > BufferHolderSparkSubmitSuite is flaky > -

[jira] [Updated] (SPARK-25542) Flaky test: OpenHashMapSuite

[ https://issues.apache.org/jira/browse/SPARK-25542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25542: -- Component/s: Tests > Flaky test: OpenHashMapSuite > > >

[jira] [Updated] (SPARK-23622) Flaky Test: HiveClientSuites

[ https://issues.apache.org/jira/browse/SPARK-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23622: -- Component/s: Tests > Flaky Test: HiveClientSuites > > >

[jira] [Updated] (SPARK-24318) Flaky test: SortShuffleSuite

[ https://issues.apache.org/jira/browse/SPARK-24318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24318: -- Component/s: Tests > Flaky test: SortShuffleSuite > > >

[jira] [Updated] (SPARK-25792) Flaky test: BarrierStageOnSubmittedSuite."submit a barrier ResultStage that requires more slots than current total under local-cluster mode"

[ https://issues.apache.org/jira/browse/SPARK-25792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25792: -- Component/s: Tests > Flaky test: BarrierStageOnSubmittedSuite."submit a barrier ResultStage th

[jira] [Updated] (SPARK-24285) Flaky test: ContinuousSuite.query without test harness

[ https://issues.apache.org/jira/browse/SPARK-24285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24285: -- Component/s: Tests > Flaky test: ContinuousSuite.query without test harness >

[jira] [Updated] (SPARK-24153) Flaky Test: DirectKafkaStreamSuite

[ https://issues.apache.org/jira/browse/SPARK-24153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24153: -- Component/s: Tests > Flaky Test: DirectKafkaStreamSuite > -- >

[jira] [Updated] (SPARK-24239) Flaky test: KafkaContinuousSourceSuite.subscribing topic by name from earliest offsets

[ https://issues.apache.org/jira/browse/SPARK-24239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24239: -- Component/s: Tests > Flaky test: KafkaContinuousSourceSuite.subscribing topic by name from >

[jira] [Updated] (SPARK-24211) Flaky test: StreamingOuterJoinSuite

[ https://issues.apache.org/jira/browse/SPARK-24211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24211: -- Component/s: Tests > Flaky test: StreamingOuterJoinSuite > ---

[jira] [Updated] (SPARK-24173) Flaky Test: VersionsSuite

[ https://issues.apache.org/jira/browse/SPARK-24173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24173: -- Component/s: Tests > Flaky Test: VersionsSuite > - > >

[jira] [Commented] (SPARK-24140) Flaky test: KMeansClusterSuite.task size should be small in both training and prediction

[ https://issues.apache.org/jira/browse/SPARK-24140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662637#comment-16662637 ] Dongjoon Hyun commented on SPARK-24140: --- Sorry, I don't have new links. You may ta

[jira] [Updated] (SPARK-24140) Flaky test: KMeansClusterSuite.task size should be small in both training and prediction

[ https://issues.apache.org/jira/browse/SPARK-24140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24140: -- Component/s: Tests > Flaky test: KMeansClusterSuite.task size should be small in both training

[jira] [Comment Edited] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662618#comment-16662618 ] Dongjoon Hyun edited comment on SPARK-25823 at 10/24/18 5:59 PM: -

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662618#comment-16662618 ] Dongjoon Hyun commented on SPARK-25823: --- Right. This is a long lasting issue when

[jira] [Updated] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25823: -- Labels: correctness (was: ) > map_filter can generate incorrect data > --

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662616#comment-16662616 ] Wenchen Fan commented on SPARK-25823: - BTW one improvement we can do is to remove du

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662613#comment-16662613 ] Wenchen Fan commented on SPARK-25823: - I did a few experiments {code} scala> sql("SE

[jira] [Updated] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25823: -- Description: This is not a regression because this occurs in new high-order functions like `m

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662583#comment-16662583 ] Dongjoon Hyun commented on SPARK-25823: --- cc [~cloud_fan], [~srowen], [~smilegator]

[jira] [Commented] (SPARK-25822) Fix a race condition when releasing a Python worker

[ https://issues.apache.org/jira/browse/SPARK-25822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662582#comment-16662582 ] Apache Spark commented on SPARK-25822: -- User 'zsxwing' has created a pull request f

  1   2   >