[jira] [Commented] (SPARK-26988) Spark overwrites spark.scheduler.pool if set in configs

2019-02-26 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1655#comment-1655 ] Marco Gaido commented on SPARK-26988: - This seems indeed an issue for any property s

[jira] [Commented] (SPARK-26947) Pyspark KMeans Clustering job fails on large values of k

2019-02-26 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1665#comment-1665 ] Marco Gaido commented on SPARK-26947: - Cloud you also please provide the heap dump o

[jira] [Updated] (SPARK-26985) Test "access only some column of the all of columns " fails on big endian

2019-02-26 Thread Anuja Jakhade (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuja Jakhade updated SPARK-26985: -- Description: While running tests on Apache Spark v2.3.2 with AdoptJDK on big endian, I am obs

[jira] [Updated] (SPARK-26985) Test "access only some column of the all of columns " fails on big endian

2019-02-26 Thread Anuja Jakhade (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuja Jakhade updated SPARK-26985: -- Attachment: access only some column of the all of columns.txt > Test "access only some column

[jira] [Commented] (SPARK-26974) Invalid data in grouped cached dataset, formed by joining a large cached dataset with a small dataset

2019-02-26 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1686#comment-1686 ] Marco Gaido commented on SPARK-26974: - Can you please try a newer Spark version (2.4

[jira] [Commented] (SPARK-26602) Once creating and quering udf with incorrect path,followed by querying tables or functions registered with correct path gives the runtime exception within the same ses

2019-02-26 Thread Chakravarthi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1695#comment-1695 ] Chakravarthi commented on SPARK-26602: -- I will be working on this issue. > Once cr

[jira] [Assigned] (SPARK-26977) Warn against subclassing scala.App doesn't work

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26977: Assignee: Apache Spark > Warn against subclassing scala.App doesn't work > --

[jira] [Assigned] (SPARK-26977) Warn against subclassing scala.App doesn't work

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26977: Assignee: (was: Apache Spark) > Warn against subclassing scala.App doesn't work > ---

[jira] [Commented] (SPARK-26927) Race condition may cause dynamic allocation not working

2019-02-26 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777836#comment-16777836 ] liupengcheng commented on SPARK-26927: -- [~Ngone51] Let's say we got the following

[jira] [Commented] (SPARK-24669) Managed table was not cleared of path after drop database cascade

2019-02-26 Thread Udbhav Agrawal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777842#comment-16777842 ] Udbhav Agrawal commented on SPARK-24669: I will work on this > Managed table wa

[jira] [Updated] (SPARK-26940) Observed greater deviation on big endian platform for SingletonReplSuite test case

2019-02-26 Thread Anuja Jakhade (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuja Jakhade updated SPARK-26940: -- Labels: BigEndian (was: ) > Observed greater deviation on big endian platform for SingletonRe

[jira] [Updated] (SPARK-26985) Test "access only some column of the all of columns " fails on big endian

2019-02-26 Thread Anuja Jakhade (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuja Jakhade updated SPARK-26985: -- Labels: BigEndian (was: ) > Test "access only some column of the all of columns " fails on bi

[jira] [Updated] (SPARK-26985) Test "access only some column of the all of columns " fails on big endian

2019-02-26 Thread Anuja Jakhade (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuja Jakhade updated SPARK-26985: -- Description: While running tests on Apache Spark v2.3.2 with AdoptJDK on big endian, I am obs

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xingbo Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778026#comment-16778026 ] Xingbo Jiang commented on SPARK-24615: -- I updated the SPIP and Product docs, please

[jira] [Comment Edited] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xingbo Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778026#comment-16778026 ] Xingbo Jiang edited comment on SPARK-24615 at 2/26/19 3:07 PM: ---

[jira] [Assigned] (SPARK-26978) Avoid magic time constants

2019-02-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26978: - Assignee: Maxim Gekk > Avoid magic time constants > -- > >

[jira] [Commented] (SPARK-26988) Spark overwrites spark.scheduler.pool if set in configs

2019-02-26 Thread Dave DeCaprio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778032#comment-16778032 ] Dave DeCaprio commented on SPARK-26988: --- Yes, it would be an issue for any propert

[jira] [Resolved] (SPARK-26978) Avoid magic time constants

2019-02-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26978. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23878 [https://github.c

[jira] [Commented] (SPARK-26927) Race condition may cause dynamic allocation not working

2019-02-26 Thread wuyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778048#comment-16778048 ] wuyi commented on SPARK-26927: -- I got it, thank you. > Race condition may cause dynamic al

[jira] [Resolved] (SPARK-26712) Single disk broken causing YarnShuffleSerivce not available

2019-02-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26712. --- Resolution: Won't Fix > Single disk broken causing YarnShuffleSerivce not available > --

[jira] [Resolved] (SPARK-26750) Estimate memory overhead should taking multi-cores into account

2019-02-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26750. --- Resolution: Won't Fix > Estimate memory overhead should taking multi-cores into account > --

[jira] [Created] (SPARK-26995) Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy

2019-02-26 Thread Luca Canali (JIRA)
Luca Canali created SPARK-26995: --- Summary: Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy Key: SPARK-26995 URL: https://issues.apache.org/jira/browse/SPARK-26995 P

[jira] [Created] (SPARK-26994) Enhance StructField to accept number format or date format

2019-02-26 Thread Murali Aakula (JIRA)
Murali Aakula created SPARK-26994: - Summary: Enhance StructField to accept number format or date format Key: SPARK-26994 URL: https://issues.apache.org/jira/browse/SPARK-26994 Project: Spark

[jira] [Assigned] (SPARK-26995) Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26995: Assignee: (was: Apache Spark) > Running Spark in Docker image with Alpine Linux 3.9.0

[jira] [Assigned] (SPARK-26995) Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26995: Assignee: Apache Spark > Running Spark in Docker image with Alpine Linux 3.9.0 throws err

[jira] [Updated] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Ilya Peysakhov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Peysakhov updated SPARK-26996: --- Description: Spark 2.4 reports an error when querying a dataframe that has only 1 row and 1

[jira] [Created] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Ilya Peysakhov (JIRA)
Ilya Peysakhov created SPARK-26996: -- Summary: Scalar Subquery not handled properly in Spark 2.4 Key: SPARK-26996 URL: https://issues.apache.org/jira/browse/SPARK-26996 Project: Spark Issue

[jira] [Created] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-26997: -- Summary: k8s integration tests failing after client upgraded to 4.1.2 Key: SPARK-26997 URL: https://issues.apache.org/jira/browse/SPARK-26997 Project: Spark

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778234#comment-16778234 ] shane knapp commented on SPARK-26997: - how exactly are you launching minikube when p

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778240#comment-16778240 ] Marcelo Vanzin commented on SPARK-26997: I'm using virtualbox. > k8s integratio

[jira] [Commented] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778246#comment-16778246 ] Dongjoon Hyun commented on SPARK-26996: --- Thank you for reporting, [~ilya745]. I al

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778247#comment-16778247 ] shane knapp commented on SPARK-26997: - i bet it's a version incompatibility. is the

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread shane knapp (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778411#comment-16778411 ] shane knapp commented on SPARK-26997: - that'd be super. :) -- Shane Knapp UC Ber

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778376#comment-16778376 ] Marcelo Vanzin commented on SPARK-26997: It's linked in the description. I can r

[jira] [Updated] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido updated SPARK-26996: Component/s: (was: Spark Core) SQL > Scalar Subquery not handled properly in

[jira] [Commented] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778453#comment-16778453 ] Marco Gaido commented on SPARK-26996: - I have not been able to reproduce on current

[jira] [Updated] (SPARK-26984) Incompatibility between Spark releases - Some(null)

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26984: -- Fix Version/s: (was: 2.4.2) (was: 2.4.1) > Incompatibility between

[jira] [Updated] (SPARK-26984) Incompatibility between Spark releases - Some(null)

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26984: -- Target Version/s: (was: 2.4.0) > Incompatibility between Spark releases - Some(null) >

[jira] [Commented] (SPARK-26984) Incompatibility between Spark releases - Some(null)

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778457#comment-16778457 ] Dongjoon Hyun commented on SPARK-26984: --- Hi, [~thebluephantom]. Thank you for repo

[jira] [Commented] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778461#comment-16778461 ] Dongjoon Hyun commented on SPARK-26996: --- As a workaround, please turn off `spark.s

[jira] [Resolved] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26996. --- Resolution: Duplicate Fix Version/s: 2.4.1 > Scalar Subquery not handled properly in

[jira] [Comment Edited] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778461#comment-16778461 ] Dongjoon Hyun edited comment on SPARK-26996 at 2/26/19 6:53 PM: --

[jira] [Closed] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-26996. - > Scalar Subquery not handled properly in Spark 2.4 > -

[jira] [Commented] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778466#comment-16778466 ] Dongjoon Hyun commented on SPARK-26996: --- BTW, [~mgaido]. FYI, `branch-2.4` works l

[jira] [Updated] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26996: -- Fix Version/s: (was: 2.4.1) > Scalar Subquery not handled properly in Spark 2.4 > ---

[jira] [Assigned] (SPARK-26969) [Spark] Using ODBC not able to see the data in table when datatype is decimal

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26969: Assignee: (was: Apache Spark) > [Spark] Using ODBC not able to see the data in table

[jira] [Assigned] (SPARK-26969) [Spark] Using ODBC not able to see the data in table when datatype is decimal

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26969: Assignee: Apache Spark > [Spark] Using ODBC not able to see the data in table when dataty

[jira] [Commented] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778485#comment-16778485 ] Marco Gaido commented on SPARK-26996: - Thanks [~dongjoon]! > Scalar Subquery not ha

[jira] [Commented] (SPARK-26996) Scalar Subquery not handled properly in Spark 2.4

2019-02-26 Thread Ilya Peysakhov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778510#comment-16778510 ] Ilya Peysakhov commented on SPARK-26996: thank you [~dongjoon]! > Scalar Subque

[jira] [Comment Edited] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778557#comment-16778557 ] Stavros Kontopoulos edited comment on SPARK-26997 at 2/26/19 8:35 PM:

[jira] [Comment Edited] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778557#comment-16778557 ] Stavros Kontopoulos edited comment on SPARK-26997 at 2/26/19 8:36 PM:

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778557#comment-16778557 ] Stavros Kontopoulos commented on SPARK-26997: - [~vanzin] I run the tests suc

[jira] [Assigned] (SPARK-23836) Support returning StructType to the level support in GroupedMap Arrow's "scalar" UDFS (or similar)

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23836: Assignee: (was: Apache Spark) > Support returning StructType to the level support in

[jira] [Assigned] (SPARK-23836) Support returning StructType to the level support in GroupedMap Arrow's "scalar" UDFS (or similar)

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23836: Assignee: Apache Spark > Support returning StructType to the level support in GroupedMap

[jira] [Comment Edited] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778557#comment-16778557 ] Stavros Kontopoulos edited comment on SPARK-26997 at 2/26/19 8:48 PM:

[jira] [Comment Edited] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778557#comment-16778557 ] Stavros Kontopoulos edited comment on SPARK-26997 at 2/26/19 8:50 PM:

[jira] [Commented] (SPARK-26839) on JDK11, IsolatedClientLoader must be able to load java.sql classes

2019-02-26 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778577#comment-16778577 ] Imran Rashid commented on SPARK-26839: -- Sorry I'm traveling now and don't have acce

[jira] [Comment Edited] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778557#comment-16778557 ] Stavros Kontopoulos edited comment on SPARK-26997 at 2/26/19 9:27 PM:

[jira] [Comment Edited] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

2019-02-26 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778557#comment-16778557 ] Stavros Kontopoulos edited comment on SPARK-26997 at 2/26/19 9:28 PM:

[jira] [Reopened] (SPARK-26742) Bump Kubernetes Client Version to 4.1.1

2019-02-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-26742: Assignee: (was: Jiaxin Shan) Reopened since we had to revert the patch. > Bump Kube

[jira] [Assigned] (SPARK-26742) Bump Kubernetes Client Version to 4.1.1

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26742: Assignee: (was: Apache Spark) > Bump Kubernetes Client Version to 4.1.1 > ---

[jira] [Reopened] (SPARK-25590) kubernetes-model-2.0.0.jar masks default Spark logging config

2019-02-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-25590: Assignee: (was: Jiaxin Shan) Reopened since the patch was reverted. > kubernetes-mo

[jira] [Resolved] (SPARK-25147) GroupedData.apply pandas_udf crashing

2019-02-26 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-25147. -- Resolution: Cannot Reproduce Going to resolve this for now, please reopen if the above suggest

[jira] [Assigned] (SPARK-26742) Bump Kubernetes Client Version to 4.1.1

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26742: Assignee: Apache Spark > Bump Kubernetes Client Version to 4.1.1 > --

[jira] [Commented] (SPARK-26972) Issue with CSV import and inferSchema set to true

2019-02-26 Thread Jean Georges Perrin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778632#comment-16778632 ] Jean Georges Perrin commented on SPARK-26972: - Thanks [~hyukjin.kwon]! I lik

[jira] [Updated] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Attachment: Accelerator-aware scheduling in Apache Spark 3.0.pdf > Accelerator-aware task sche

[jira] [Updated] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24615: -- Attachment: SPIP_ Accelerator-aware scheduling.pdf > Accelerator-aware task scheduling for Spa

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2019-02-26 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778660#comment-16778660 ] Xiangrui Meng commented on SPARK-24615: --- Attached the PDF files. > Accelerator-aw

[jira] [Assigned] (SPARK-22860) Spark workers log ssl passwords passed to the executors

2019-02-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22860: -- Assignee: Jungtaek Lim > Spark workers log ssl passwords passed to the executors > --

[jira] [Resolved] (SPARK-22860) Spark workers log ssl passwords passed to the executors

2019-02-26 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22860. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23820 [https:

[jira] [Created] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-02-26 Thread t oo (JIRA)
t oo created SPARK-26998: Summary: spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode Key: SPARK-26998 URL: https://issues.apache.org/jira/browse/SPARK-26998 P

[jira] [Updated] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

2019-02-26 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-26998: - Labels: SECURITY Security secur security security-issue (was: ) > spark.ssl.keyStorePassword in plaintext on 'p

[jira] [Assigned] (SPARK-26449) Missing Dataframe.transform API in Python API

2019-02-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26449: - Assignee: Hanan Shteingart > Missing Dataframe.transform API in Python API > --

[jira] [Resolved] (SPARK-26449) Missing Dataframe.transform API in Python API

2019-02-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26449. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23877 [https://github.c

[jira] [Resolved] (SPARK-26986) Add JAXB reference impl to build for Java 9+

2019-02-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26986. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23890 [https://github.c

[jira] [Created] (SPARK-26999) SparkSQL CLIDriver parses sql statement incorrectly

2019-02-26 Thread feiwang (JIRA)
feiwang created SPARK-26999: --- Summary: SparkSQL CLIDriver parses sql statement incorrectly Key: SPARK-26999 URL: https://issues.apache.org/jira/browse/SPARK-26999 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-26999) SparkSQL CLIDriver parses sql statement incorrectly

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26999: Assignee: Apache Spark > SparkSQL CLIDriver parses sql statement incorrectly > -

[jira] [Assigned] (SPARK-26999) SparkSQL CLIDriver parses sql statement incorrectly

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26999: Assignee: (was: Apache Spark) > SparkSQL CLIDriver parses sql statement incorrectly

[jira] [Updated] (SPARK-26999) SparkSQL CLIDriver parses sql statement incorrectly

2019-02-26 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-26999: Description: SparkSQLCLIDriver parse sql statement incorrectly, because its processLine method is not cor

[jira] [Updated] (SPARK-26999) SparkSQL CLIDriver parses sql statement incorrectly

2019-02-26 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-26999: Description: SparkSQLCLIDriver parse sql statement incorrectly, because its processLine method is not cor

[jira] [Updated] (SPARK-26999) SparkSQL CLIDriver parses sql statement incorrectly

2019-02-26 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-26999: Description: SparkSQLCLIDriver parse sql statement incorrectly, because its processLine method is not cor

[jira] [Commented] (SPARK-23682) Memory issue with Spark structured streaming

2019-02-26 Thread Chiyu Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778794#comment-16778794 ] Chiyu Zhong commented on SPARK-23682: - [~kabhwan] I have the same issue, I'm using b

[jira] [Created] (SPARK-27000) Functions that has the same name can't be used in Python RDD API

2019-02-26 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-27000: Summary: Functions that has the same name can't be used in Python RDD API Key: SPARK-27000 URL: https://issues.apache.org/jira/browse/SPARK-27000 Project: Spark

[jira] [Updated] (SPARK-26999) SparkSQL CLIDriver parses sql statement incorrectly

2019-02-26 Thread feiwang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feiwang updated SPARK-26999: Attachment: SPARK-26999.png > SparkSQL CLIDriver parses sql statement incorrectly > -

[jira] [Assigned] (SPARK-27000) Functions that has the same name can't be used in Python RDD API

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27000: Assignee: Hyukjin Kwon (was: Apache Spark) > Functions that has the same name can't be u

[jira] [Resolved] (SPARK-26837) Pruning nested fields from object serializers

2019-02-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26837. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 3.0.0 > Pruning neste

[jira] [Assigned] (SPARK-27000) Functions that has the same name can't be used in Python RDD API

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27000: Assignee: Apache Spark (was: Hyukjin Kwon) > Functions that has the same name can't be u

[jira] [Updated] (SPARK-27000) Global function that has the same name can't be overwritten in Python RDD API

2019-02-26 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27000: - Summary: Global function that has the same name can't be overwritten in Python RDD API (was: Fu

[jira] [Resolved] (SPARK-26830) Vectorized dapply, Arrow optimization in native R function execution

2019-02-26 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26830. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23787 [https://gi

[jira] [Assigned] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

2019-02-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22000: --- Assignee: Jungtaek Lim > org.codehaus.commons.compiler.CompileException: toString method is

[jira] [Resolved] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

2019-02-26 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22000. - Resolution: Fixed Fix Version/s: 3.0.0 > org.codehaus.commons.compiler.CompileException:

[jira] [Commented] (SPARK-26995) Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy

2019-02-26 Thread Stijn De Haes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778972#comment-16778972 ] Stijn De Haes commented on SPARK-26995: --- [~lucacanali] as a temporary fix I added

[jira] [Commented] (SPARK-26995) Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy

2019-02-26 Thread Stijn De Haes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778976#comment-16778976 ] Stijn De Haes commented on SPARK-26995: --- I see your PR did the same > Running Spa

[jira] [Assigned] (SPARK-24669) Managed table was not cleared of path after drop database cascade

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24669: Assignee: Apache Spark > Managed table was not cleared of path after drop database cascad

[jira] [Assigned] (SPARK-24669) Managed table was not cleared of path after drop database cascade

2019-02-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24669: Assignee: (was: Apache Spark) > Managed table was not cleared of path after drop data