[jira] [Commented] (SPARK-20399) Can't use same regex pattern between 1.6 and 2.x due to unescaped sql string in parser

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999687#comment-15999687 ] Apache Spark commented on SPARK-20399: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999671#comment-15999671 ] Apache Spark commented on SPARK-13983: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-20629) Copy data when nodes are being shut down

2017-05-06 Thread holdenk (JIRA)
holdenk created SPARK-20629: --- Summary: Copy data when nodes are being shut down Key: SPARK-20629 URL: https://issues.apache.org/jira/browse/SPARK-20629 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-20628) Keep track of nodes which are going to be shut down & avoid scheduling new tasks

2017-05-06 Thread holdenk (JIRA)
holdenk created SPARK-20628: --- Summary: Keep track of nodes which are going to be shut down & avoid scheduling new tasks Key: SPARK-20628 URL: https://issues.apache.org/jira/browse/SPARK-20628 Project:

[jira] [Assigned] (SPARK-20627) Remove pip local version string (PEP440)

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20627: Assignee: (was: Apache Spark) > Remove pip local version string (PEP440) >

[jira] [Assigned] (SPARK-20627) Remove pip local version string (PEP440)

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20627: Assignee: Apache Spark > Remove pip local version string (PEP440) >

[jira] [Commented] (SPARK-20627) Remove pip local version string (PEP440)

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999663#comment-15999663 ] Apache Spark commented on SPARK-20627: -- User 'holdenk' has created a pull request for this issue:

[jira] [Created] (SPARK-20627) Remove pip local version string (PEP440)

2017-05-06 Thread holdenk (JIRA)
holdenk created SPARK-20627: --- Summary: Remove pip local version string (PEP440) Key: SPARK-20627 URL: https://issues.apache.org/jira/browse/SPARK-20627 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-20626) Fix SparkR test on Windows with timestamp time zone

2017-05-06 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20626: Summary: Fix SparkR test on Windows with timestamp time zone Key: SPARK-20626 URL: https://issues.apache.org/jira/browse/SPARK-20626 Project: Spark Issue

[jira] [Created] (SPARK-20625) SQL test running on Windows hadoop command warning

2017-05-06 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20625: Summary: SQL test running on Windows hadoop command warning Key: SPARK-20625 URL: https://issues.apache.org/jira/browse/SPARK-20625 Project: Spark Issue

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2017-05-06 Thread Patrick Grandjean (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999623#comment-15999623 ] Patrick Grandjean commented on SPARK-7768: -- How does UDTRegistration interact with Encoders? >

[jira] [Commented] (SPARK-11010) Fixes and enhancements addressing UDTs' api and several usability concerns

2017-05-06 Thread Patrick Grandjean (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999616#comment-15999616 ] Patrick Grandjean commented on SPARK-11010: --- Is there any workaround available now? > Fixes

[jira] [Created] (SPARK-20624) Consider adding better handling for node shutdown

2017-05-06 Thread holdenk (JIRA)
holdenk created SPARK-20624: --- Summary: Consider adding better handling for node shutdown Key: SPARK-20624 URL: https://issues.apache.org/jira/browse/SPARK-20624 Project: Spark Issue Type:

[jira] [Updated] (SPARK-20623) Application link returns redirect to SPARK_LOCAL_IP and disregards SPARK_PUBLIC_DNS

2017-05-06 Thread Anders Roos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anders Roos updated SPARK-20623: Description: I have a setup with several machines in standalone mode where the master runs on a

[jira] [Created] (SPARK-20623) Application link returns redirect to SPARK_LOCAL_IP and disregards SPARK_PUBLIC_DNS

2017-05-06 Thread Anders Roos (JIRA)
Anders Roos created SPARK-20623: --- Summary: Application link returns redirect to SPARK_LOCAL_IP and disregards SPARK_PUBLIC_DNS Key: SPARK-20623 URL: https://issues.apache.org/jira/browse/SPARK-20623

[jira] [Commented] (SPARK-20622) Parquet partition discovery for non key=value named directories

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999470#comment-15999470 ] Apache Spark commented on SPARK-20622: -- User 'prokod' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20622) Parquet partition discovery for non key=value named directories

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20622: Assignee: (was: Apache Spark) > Parquet partition discovery for non key=value named

[jira] [Assigned] (SPARK-20622) Parquet partition discovery for non key=value named directories

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20622: Assignee: Apache Spark > Parquet partition discovery for non key=value named directories

[jira] [Created] (SPARK-20622) Parquet partition discovery for non key=value named directories

2017-05-06 Thread Noam Asor (JIRA)
Noam Asor created SPARK-20622: - Summary: Parquet partition discovery for non key=value named directories Key: SPARK-20622 URL: https://issues.apache.org/jira/browse/SPARK-20622 Project: Spark

[jira] [Commented] (SPARK-18891) Support for specific collection types

2017-05-06 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-18891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999450#comment-15999450 ] Michal Šenkýř commented on SPARK-18891: --- Changes in master only add support for Scala {{Seq}}

[jira] [Commented] (SPARK-20079) Re registration of AM hangs spark cluster in yarn-client mode

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999431#comment-15999431 ] Apache Spark commented on SPARK-20079: -- User 'witgo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20621) Delete deprecated config parameter in 'spark-env.sh'

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20621: Assignee: Apache Spark > Delete deprecated config parameter in 'spark-env.sh' >

[jira] [Assigned] (SPARK-20621) Delete deprecated config parameter in 'spark-env.sh'

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20621: Assignee: (was: Apache Spark) > Delete deprecated config parameter in 'spark-env.sh'

[jira] [Commented] (SPARK-20621) Delete deprecated config parameter in 'spark-env.sh'

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999416#comment-15999416 ] Apache Spark commented on SPARK-20621: -- User 'ConeyLiu' has created a pull request for this issue:

[jira] [Commented] (SPARK-20608) Standby namenodes should be allowed to included in yarn.spark.access.namenodes to support HDFS HA

2017-05-06 Thread Yuechen Chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999415#comment-15999415 ] Yuechen Chen commented on SPARK-20608: -- [~vanzin] By my knowledge, Spark will gain token in two

[jira] [Created] (SPARK-20621) Delete deprecated config parameter in 'spark-env.sh'

2017-05-06 Thread coneyliu (JIRA)
coneyliu created SPARK-20621: Summary: Delete deprecated config parameter in 'spark-env.sh' Key: SPARK-20621 URL: https://issues.apache.org/jira/browse/SPARK-20621 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-20353) Implement Tensorflow TFRecords file format

2017-05-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20353. --- Resolution: Won't Fix > Implement Tensorflow TFRecords file format >

[jira] [Resolved] (SPARK-20577) Add REST API Documentation in Cluster Mode

2017-05-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20577. --- Resolution: Not A Problem > Add REST API Documentation in Cluster Mode >

[jira] [Resolved] (SPARK-20494) Implement UDF array_unique in Spark with codegen

2017-05-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20494. --- Resolution: Won't Fix > Implement UDF array_unique in Spark with codegen >

[jira] [Resolved] (SPARK-20526) Load doesn't work in PCAModel

2017-05-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20526. --- Resolution: Invalid > Load doesn't work in PCAModel > -- > >

[jira] [Commented] (SPARK-19112) add codec for ZStandard

2017-05-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999363#comment-15999363 ] Sean Owen commented on SPARK-19112: --- The benchmark suggests this is significantly slower than lz4

[jira] [Assigned] (SPARK-20620) Add some unit tests into NullExpressionsSuite

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20620: Assignee: (was: Apache Spark) > Add some unit tests into NullExpressionsSuite >

[jira] [Assigned] (SPARK-20620) Add some unit tests into NullExpressionsSuite

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20620: Assignee: Apache Spark > Add some unit tests into NullExpressionsSuite >

[jira] [Commented] (SPARK-20620) Add some unit tests into NullExpressionsSuite

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999345#comment-15999345 ] Apache Spark commented on SPARK-20620: -- User '10110346' has created a pull request for this issue:

[jira] [Updated] (SPARK-20620) Add some unit tests into NullExpressionsSuite

2017-05-06 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-20620: Description: Add some unit tests into NullExpressionsSuite: 1.add more datatype for exist unit 2.add new

[jira] [Updated] (SPARK-20620) Add some unit tests into NullExpressionsSuite

2017-05-06 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-20620: Description: Add some unit tests into NullExpressionsSuite: 1.add more datatype for exist unit 2.add new

[jira] [Updated] (SPARK-20620) Add some unit tests into NullExpressionsSuite

2017-05-06 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-20620: Description: Add some unit tests into NullExpressionsSuite: 1.add datatype for exist unit 2.add new unit

[jira] [Updated] (SPARK-20620) Add some unit tests into NullExpressionsSuite

2017-05-06 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-20620: Description: Add some unit tests into NullExpressionsSuite > Add some unit tests into NullExpressionsSuite

[jira] [Created] (SPARK-20620) Add some unit tests into NullExpressionsSuite

2017-05-06 Thread liuxian (JIRA)
liuxian created SPARK-20620: --- Summary: Add some unit tests into NullExpressionsSuite Key: SPARK-20620 URL: https://issues.apache.org/jira/browse/SPARK-20620 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-20619) StringIndexer supports multiple ways of label ordering

2017-05-06 Thread Wayne Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wayne Zhang updated SPARK-20619: Description: StringIndexer maps labels to numbers according to the descending order of label

[jira] [Assigned] (SPARK-20619) StringIndexer supports multiple ways of label ordering

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20619: Assignee: (was: Apache Spark) > StringIndexer supports multiple ways of label

[jira] [Commented] (SPARK-20619) StringIndexer supports multiple ways of label ordering

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999319#comment-15999319 ] Apache Spark commented on SPARK-20619: -- User 'actuaryzhang' has created a pull request for this

[jira] [Assigned] (SPARK-20619) StringIndexer supports multiple ways of label ordering

2017-05-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20619: Assignee: Apache Spark > StringIndexer supports multiple ways of label ordering >

[jira] [Created] (SPARK-20619) StringIndexer supports multiple ways of label ordering

2017-05-06 Thread Wayne Zhang (JIRA)
Wayne Zhang created SPARK-20619: --- Summary: StringIndexer supports multiple ways of label ordering Key: SPARK-20619 URL: https://issues.apache.org/jira/browse/SPARK-20619 Project: Spark Issue

[jira] [Created] (SPARK-20618) Support Custom Partitioners in PySpark

2017-05-06 Thread holdenk (JIRA)
holdenk created SPARK-20618: --- Summary: Support Custom Partitioners in PySpark Key: SPARK-20618 URL: https://issues.apache.org/jira/browse/SPARK-20618 Project: Spark Issue Type: Improvement