[jira] [Commented] (SPARK-22660) Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272287#comment-16272287 ] liyunzhang commented on SPARK-22660: not support JDK9 now , i am working on it. > Use position() and

[jira] [Comment Edited] (SPARK-22660) Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272249#comment-16272249 ] liyunzhang edited comment on SPARK-22660 at 11/30/17 7:39 AM: -- some new

[jira] [Commented] (SPARK-22660) Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9

2017-11-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272283#comment-16272283 ] Liang-Chi Hsieh commented on SPARK-22660: - For the error you ping me, from the error message,

[jira] [Commented] (SPARK-22660) Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272281#comment-16272281 ] liyunzhang commented on SPARK-22660: [~viirya]: the error mentioned above does not exist any more

[jira] [Commented] (SPARK-22660) Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272249#comment-16272249 ] liyunzhang commented on SPARK-22660: some new error {code} error]

[jira] [Commented] (SPARK-22660) Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272225#comment-16272225 ] liyunzhang commented on SPARK-22660: Besides above error, there are another errors with scala-2.12

[jira] [Created] (SPARK-22660) Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9

2017-11-29 Thread liyunzhang (JIRA)
liyunzhang created SPARK-22660: -- Summary: Use position() and limit() to fix ambiguity issue in scala-2.12 and JDK9 Key: SPARK-22660 URL: https://issues.apache.org/jira/browse/SPARK-22660 Project: Spark

[jira] [Assigned] (SPARK-22659) remove sun.misc.Cleaner references

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22659: Assignee: (was: Apache Spark) > remove sun.misc.Cleaner references >

[jira] [Commented] (SPARK-22659) remove sun.misc.Cleaner references

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272187#comment-16272187 ] Apache Spark commented on SPARK-22659: -- User 'kellyzly' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22659) remove sun.misc.Cleaner references

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22659: Assignee: Apache Spark > remove sun.misc.Cleaner references >

[jira] [Commented] (SPARK-22659) remove sun.misc.Cleaner references

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272177#comment-16272177 ] liyunzhang commented on SPARK-22659: I am very confused about why this issue exists with java.version

[jira] [Commented] (SPARK-22659) remove sun.misc.Cleaner references

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272123#comment-16272123 ] liyunzhang commented on SPARK-22659: So when you compile with -Pscala-2.12 and jdk8, there is no

[jira] [Commented] (SPARK-22659) remove sun.misc.Cleaner references

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272117#comment-16272117 ] Sean Owen commented on SPARK-22659: --- This isn't related to Scala 2.12 now. JDK 9 isn't supported, and

[jira] [Updated] (SPARK-22659) remove sun.misc.Cleaner references

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated SPARK-22659: --- Description: build with scala-2.12 with following steps 1. change the pom.xml with scala-2.12 {code}

[jira] [Updated] (SPARK-22659) remove sun.misc.Cleaner references

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang updated SPARK-22659: --- Summary: remove sun.misc.Cleaner references (was: Use {{scala.binary.version}} in the artifactId in

[jira] [Comment Edited] (SPARK-22659) Use {{scala.binary.version}} in the artifactId in the pom.xml of common/tags and streaming

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272048#comment-16272048 ] liyunzhang edited comment on SPARK-22659 at 11/30/17 2:31 AM: -- I saw the

[jira] [Commented] (SPARK-22659) Use {{scala.binary.version}} in the artifactId in the pom.xml of common/tags and streaming

2017-11-29 Thread liyunzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272048#comment-16272048 ] liyunzhang commented on SPARK-22659: where is the script? > Use {{scala.binary.version}} in the

[jira] [Commented] (SPARK-22659) Use {{scala.binary.version}} in the artifactId in the pom.xml of common/tags and streaming

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272045#comment-16272045 ] Sean Owen commented on SPARK-22659: --- You can't put vars in the artifact names in Maven. Believe me if

[jira] [Created] (SPARK-22659) Use {{scala.binary.version}} in the artifactId in the pom.xml of common/tags and streaming

2017-11-29 Thread liyunzhang (JIRA)
liyunzhang created SPARK-22659: -- Summary: Use {{scala.binary.version}} in the artifactId in the pom.xml of common/tags and streaming Key: SPARK-22659 URL: https://issues.apache.org/jira/browse/SPARK-22659

[jira] [Commented] (SPARK-22630) Consolidate all configuration properties into one page

2017-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272033#comment-16272033 ] Hyukjin Kwon commented on SPARK-22630: -- +1 for ^. > Consolidate all configuration properties into

[jira] [Commented] (SPARK-22656) Upgrade Arrow to 0.8.0

2017-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272026#comment-16272026 ] Hyukjin Kwon commented on SPARK-22656: -- Hi [~zsxwing], seems a duplicate of SPARK-22324. > Upgrade

[jira] [Assigned] (SPARK-22585) Url encoding of jar path expected?

2017-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22585: Assignee: Jakub Dubovsky > Url encoding of jar path expected? >

[jira] [Resolved] (SPARK-22585) Url encoding of jar path expected?

2017-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22585. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19834

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-29 Thread Leigh Klotz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271975#comment-16271975 ] Leigh Klotz commented on SPARK-22373: -- [~mshen] Thank you. I've hand-upgraded janino and

[jira] [Commented] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271944#comment-16271944 ] Sean Owen commented on SPARK-22657: --- I wouldn't have expected that to work. The user app classloader

[jira] [Resolved] (SPARK-22608) Avoid code duplication regarding CodeGeneration.splitExpressions()

2017-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-22608. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19821

[jira] [Assigned] (SPARK-22608) Avoid code duplication regarding CodeGeneration.splitExpressions()

2017-11-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-22608: --- Assignee: Kazuaki Ishizaki > Avoid code duplication regarding

[jira] [Commented] (SPARK-22658) SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271930#comment-16271930 ] Sean Owen commented on SPARK-22658: --- I don't see a strong reason this needs to be part of Spark. It

[jira] [Updated] (SPARK-22658) SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark

2017-11-29 Thread Andy Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Feng updated SPARK-22658: -- Attachment: SPIP_ TensorFlowOnSpark.pdf > SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of

[jira] [Updated] (SPARK-22658) SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark

2017-11-29 Thread Andy Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Feng updated SPARK-22658: -- Description: TensorFlowOnSpark (TFoS) was released at github for distributed TensorFlow training and

[jira] [Updated] (SPARK-22658) SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark

2017-11-29 Thread Andy Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Feng updated SPARK-22658: -- Description: SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark Authors: Lee

[jira] [Updated] (SPARK-22658) SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark

2017-11-29 Thread Andy Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Feng updated SPARK-22658: -- Description: In Feburary 2017, TensorFlowOnSpark (TFoS) was released for distributed TensorFlow

[jira] [Created] (SPARK-22658) SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark

2017-11-29 Thread Andy Feng (JIRA)
Andy Feng created SPARK-22658: - Summary: SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib of Apache Spark Key: SPARK-22658 URL: https://issues.apache.org/jira/browse/SPARK-22658 Project: Spark

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ./bin/spark-submit --master

[jira] [Commented] (SPARK-22646) Spark on Kubernetes - basic submission client

2017-11-29 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271818#comment-16271818 ] Yinan Li commented on SPARK-22646: -- The "Component/s" field should be updated. > Spark on Kubernetes -

[jira] [Updated] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-22657: Description: To reproduce this issue run: ``` ./bin/spark-submit --master

[jira] [Created] (SPARK-22657) Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used

2017-11-29 Thread Stavros Kontopoulos (JIRA)
Stavros Kontopoulos created SPARK-22657: --- Summary: Hadoop fs implementation classes are not loaded if they are part of the app jar or other jar when --packages flag is used Key: SPARK-22657 URL:

[jira] [Commented] (SPARK-22647) Docker files for image creation

2017-11-29 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271813#comment-16271813 ] Yinan Li commented on SPARK-22647: -- Note: Some reference Dockerfiles are included in

[jira] [Created] (SPARK-22656) Upgrade Arrow to 0.8.0

2017-11-29 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-22656: Summary: Upgrade Arrow to 0.8.0 Key: SPARK-22656 URL: https://issues.apache.org/jira/browse/SPARK-22656 Project: Spark Issue Type: Bug Components:

[jira] [Resolved] (SPARK-20650) Remove JobProgressListener (and other unneeded classes)

2017-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-20650. Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19750

[jira] [Resolved] (SPARK-18935) Use Mesos "Dynamic Reservation" resource for Spark

2017-11-29 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-18935. Resolution: Fixed Assignee: Stavros Kontopoulos Fix Version/s: 2.3.0 > Use

[jira] [Commented] (SPARK-22655) Fail task instead of complete task silently in PythonRunner during shutdown

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271626#comment-16271626 ] Apache Spark commented on SPARK-22655: -- User 'icexelloss' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22655) Fail task instead of complete task silently in PythonRunner during shutdown

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22655: Assignee: Apache Spark > Fail task instead of complete task silently in PythonRunner

[jira] [Assigned] (SPARK-22655) Fail task instead of complete task silently in PythonRunner during shutdown

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22655: Assignee: (was: Apache Spark) > Fail task instead of complete task silently in

[jira] [Commented] (SPARK-22655) Fail task instead of complete task silently in PythonRunner during shutdown

2017-11-29 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271620#comment-16271620 ] Li Jin commented on SPARK-22655: PR: https://github.com/apache/spark/pull/19852 > Fail task instead of

[jira] [Created] (SPARK-22655) Fail task instead of complete task silently in PythonRunner during shutdown

2017-11-29 Thread Li Jin (JIRA)
Li Jin created SPARK-22655: -- Summary: Fail task instead of complete task silently in PythonRunner during shutdown Key: SPARK-22655 URL: https://issues.apache.org/jira/browse/SPARK-22655 Project: Spark

[jira] [Commented] (SPARK-22654) Retry download of Spark from ASF mirror in HiveExternalCatalogVersionsSuite

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271365#comment-16271365 ] Apache Spark commented on SPARK-22654: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22654) Retry download of Spark from ASF mirror in HiveExternalCatalogVersionsSuite

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22654: Assignee: Apache Spark (was: Sean Owen) > Retry download of Spark from ASF mirror in

[jira] [Assigned] (SPARK-22654) Retry download of Spark from ASF mirror in HiveExternalCatalogVersionsSuite

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22654: Assignee: Sean Owen (was: Apache Spark) > Retry download of Spark from ASF mirror in

[jira] [Created] (SPARK-22654) Retry download of Spark from ASF mirror in HiveExternalCatalogVersionsSuite

2017-11-29 Thread Sean Owen (JIRA)
Sean Owen created SPARK-22654: - Summary: Retry download of Spark from ASF mirror in HiveExternalCatalogVersionsSuite Key: SPARK-22654 URL: https://issues.apache.org/jira/browse/SPARK-22654 Project: Spark

[jira] [Commented] (SPARK-22653) executorAddress registered in CoarseGrainedSchedulerBackend.executorDataMap is null

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271254#comment-16271254 ] Apache Spark commented on SPARK-22653: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22653) executorAddress registered in CoarseGrainedSchedulerBackend.executorDataMap is null

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22653: Assignee: Apache Spark > executorAddress registered in

[jira] [Assigned] (SPARK-22653) executorAddress registered in CoarseGrainedSchedulerBackend.executorDataMap is null

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22653: Assignee: (was: Apache Spark) > executorAddress registered in

[jira] [Commented] (SPARK-22653) executorAddress registered in CoarseGrainedSchedulerBackend.executorDataMap is null

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271244#comment-16271244 ] Apache Spark commented on SPARK-22653: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Updated] (SPARK-22599) Avoid extra reading for cached table

2017-11-29 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-22599: Description: In the current implementation of Spark, InMemoryTableExec read all data in a cached table,

[jira] [Commented] (SPARK-22162) Executors and the driver use inconsistent Job IDs during the new RDD commit protocol

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271154#comment-16271154 ] Apache Spark commented on SPARK-22162: -- User 'rezasafi' has created a pull request for this issue:

[jira] [Resolved] (SPARK-22615) Handle more cases in PropagateEmptyRelation

2017-11-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22615. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 2.3.0 > Handle more cases in

[jira] [Commented] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271085#comment-16271085 ] Apache Spark commented on SPARK-22641: -- User 'sethah' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22641: Assignee: Apache Spark > Pyspark UDF relying on column added with withColumn after

[jira] [Assigned] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22641: Assignee: (was: Apache Spark) > Pyspark UDF relying on column added with withColumn

[jira] [Commented] (SPARK-22625) Properly cleanup inheritable thread-locals

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271041#comment-16271041 ] Sean Owen commented on SPARK-22625: --- I don't have any additional info for you. You can propose a PR.

[jira] [Commented] (SPARK-22625) Properly cleanup inheritable thread-locals

2017-11-29 Thread Tolstopyatov Vsevolod (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271038#comment-16271038 ] Tolstopyatov Vsevolod commented on SPARK-22625: --- Ping [~srowen] > Properly cleanup

[jira] [Resolved] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22622. --- Resolution: Duplicate It looks like, nevertheless, something huge is in your closure. It's probably

[jira] [Commented] (SPARK-22636) row count not being set correctly (always 0) after Statement.executeUpdate()

2017-11-29 Thread Sergio Lob (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271013#comment-16271013 ] Sergio Lob commented on SPARK-22636: OK > row count not being set correctly (always 0) after

[jira] [Commented] (SPARK-22634) Update Bouncy castle dependency

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271012#comment-16271012 ] Sean Owen commented on SPARK-22634: --- Right, I see core depends on jets3t, so it's not just provided by

[jira] [Assigned] (SPARK-22646) Spark on Kubernetes - basic submission client

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22646: Assignee: (was: Apache Spark) > Spark on Kubernetes - basic submission client >

[jira] [Commented] (SPARK-22646) Spark on Kubernetes - basic submission client

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16271011#comment-16271011 ] Apache Spark commented on SPARK-22646: -- User 'liyinan926' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22646) Spark on Kubernetes - basic submission client

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22646: Assignee: Apache Spark > Spark on Kubernetes - basic submission client >

[jira] [Updated] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-29 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Duffy updated SPARK-22641: - Description: We seem to have found an issue with PySpark UDFs interacting with {{withColumn}}

[jira] [Commented] (SPARK-22653) executorAddress registered in CoarseGrainedSchedulerBackend.executorDataMap is null

2017-11-29 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270977#comment-16270977 ] Thomas Graves commented on SPARK-22653: --- will have patch up shortly > executorAddress registered

[jira] [Created] (SPARK-22653) executorAddress registered in CoarseGrainedSchedulerBackend.executorDataMap is null

2017-11-29 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-22653: - Summary: executorAddress registered in CoarseGrainedSchedulerBackend.executorDataMap is null Key: SPARK-22653 URL: https://issues.apache.org/jira/browse/SPARK-22653

[jira] [Assigned] (SPARK-22652) remove set methods in ColumnarRow

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22652: Assignee: Wenchen Fan (was: Apache Spark) > remove set methods in ColumnarRow >

[jira] [Commented] (SPARK-22652) remove set methods in ColumnarRow

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270953#comment-16270953 ] Apache Spark commented on SPARK-22652: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-22636) row count not being set correctly (always 0) after Statement.executeUpdate()

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270956#comment-16270956 ] Sean Owen commented on SPARK-22636: --- Yes that's my understanding. We can wait a beat here to see if

[jira] [Assigned] (SPARK-22652) remove set methods in ColumnarRow

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22652: Assignee: Apache Spark (was: Wenchen Fan) > remove set methods in ColumnarRow >

[jira] [Created] (SPARK-22652) remove set methods in ColumnarRow

2017-11-29 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-22652: --- Summary: remove set methods in ColumnarRow Key: SPARK-22652 URL: https://issues.apache.org/jira/browse/SPARK-22652 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-22641) Pyspark UDF relying on column added with withColumn after distinct

2017-11-29 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Duffy updated SPARK-22641: - Description: We seem to have found an issue with PySpark UDFs interacting with {{withColumn}}

[jira] [Commented] (SPARK-22636) row count not being set correctly (always 0) after Statement.executeUpdate()

2017-11-29 Thread Sergio Lob (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270920#comment-16270920 ] Sergio Lob commented on SPARK-22636: Since we are using the Hive JDBC driver to access Spark, I guess

[jira] [Commented] (SPARK-22393) spark-shell can't find imported types in class constructors, extends clause

2017-11-29 Thread Mark Petruska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270912#comment-16270912 ] Mark Petruska commented on SPARK-22393: --- [~srowen], [~rdub], I can now confirm that the original

[jira] [Assigned] (SPARK-22393) spark-shell can't find imported types in class constructors, extends clause

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22393: Assignee: Apache Spark > spark-shell can't find imported types in class constructors,

[jira] [Commented] (SPARK-22393) spark-shell can't find imported types in class constructors, extends clause

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270903#comment-16270903 ] Apache Spark commented on SPARK-22393: -- User 'mpetruska' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22393) spark-shell can't find imported types in class constructors, extends clause

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22393: Assignee: (was: Apache Spark) > spark-shell can't find imported types in class

[jira] [Commented] (SPARK-22393) spark-shell can't find imported types in class constructors, extends clause

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270897#comment-16270897 ] Sean Owen commented on SPARK-22393: --- OK, so this is basically "fixed for Scala 2.12 only"? >

[jira] [Commented] (SPARK-22636) row count not being set correctly (always 0) after Statement.executeUpdate()

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270895#comment-16270895 ] Sean Owen commented on SPARK-22636: --- No. I'm saying the ticket you linked to does not sound like a bug

[jira] [Commented] (SPARK-22636) row count not being set correctly (always 0) after Statement.executeUpdate()

2017-11-29 Thread Sergio Lob (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270865#comment-16270865 ] Sergio Lob commented on SPARK-22636: Are you implying that its not "fixable" in neither Hive nor

[jira] [Commented] (SPARK-22633) spark-submit.cmd cannot handle long arguments

2017-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270753#comment-16270753 ] Hyukjin Kwon commented on SPARK-22633: -- {{spark-submit2.cmd}} is only there for the purpose of

[jira] [Assigned] (SPARK-22651) Calling ImageSchema.readImages initiate multiple Hive clients

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22651: Assignee: Apache Spark > Calling ImageSchema.readImages initiate multiple Hive clients >

[jira] [Resolved] (SPARK-22650) spark2.2 on yarn streaming can't connect hbase

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22650. --- Resolution: Invalid Questions to the mailing list please > spark2.2 on yarn streaming can't

[jira] [Assigned] (SPARK-22651) Calling ImageSchema.readImages initiate multiple Hive clients

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22651: Assignee: (was: Apache Spark) > Calling ImageSchema.readImages initiate multiple Hive

[jira] [Commented] (SPARK-22651) Calling ImageSchema.readImages initiate multiple Hive clients

2017-11-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270721#comment-16270721 ] Apache Spark commented on SPARK-22651: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-22651) Calling ImageSchema.readImages initiate multiple Hive clients

2017-11-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-22651: - Component/s: ML > Calling ImageSchema.readImages initiate multiple Hive clients >

[jira] [Updated] (SPARK-22642) the createdTempDir will not be deleted if an exception occurs

2017-11-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22642: -- Priority: Minor (was: Critical) This is hardly critical > the createdTempDir will not be deleted if

  1   2   >