[jira] [Created] (SPARK-25797) Views created via 2.1 cannot be read via 2.2+

2018-10-22 Thread Chenxiao Mao (JIRA)
Chenxiao Mao created SPARK-25797: Summary: Views created via 2.1 cannot be read via 2.2+ Key: SPARK-25797 URL: https://issues.apache.org/jira/browse/SPARK-25797 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25797) Views created via 2.1 cannot be read via 2.2+

2018-10-22 Thread Chenxiao Mao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxiao Mao updated SPARK-25797: - Description: We ran into this issue when we update our Spark from 2.1 to 2.3. Below's a simple

[jira] [Updated] (SPARK-25797) Views created via 2.1 cannot be read via 2.2+

2018-10-22 Thread Chenxiao Mao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenxiao Mao updated SPARK-25797: - Description: We ran into this issue when we update our Spark from 2.1 to 2.3. Below's a simple

[jira] [Commented] (SPARK-25787) [K8S] Spark can't use data locality information

2018-10-22 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658736#comment-16658736 ] Hyukjin Kwon commented on SPARK-25787: -- Can you describe the steps you took to repr

[jira] [Commented] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658751#comment-16658751 ] Apache Spark commented on SPARK-25793: -- User 'huaxingao' has created a pull request

[jira] [Assigned] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25793: Assignee: Apache Spark > Loading model bug in BisectingKMeans > -

[jira] [Assigned] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25793: Assignee: (was: Apache Spark) > Loading model bug in BisectingKMeans > --

[jira] [Commented] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658752#comment-16658752 ] Apache Spark commented on SPARK-25793: -- User 'huaxingao' has created a pull request

[jira] [Commented] (SPARK-25795) Fix CSV SparkR SQL Example

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658753#comment-16658753 ] Apache Spark commented on SPARK-25795: -- User 'dongjoon-hyun' has created a pull req

[jira] [Created] (SPARK-25798) Internally document type conversion between Pandas data and SQL types in Pandas UDFs

2018-10-22 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-25798: Summary: Internally document type conversion between Pandas data and SQL types in Pandas UDFs Key: SPARK-25798 URL: https://issues.apache.org/jira/browse/SPARK-25798

[jira] [Commented] (SPARK-24196) Spark Thrift Server - SQL Client connections does't show db artefacts

2018-10-22 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658917#comment-16658917 ] Yuming Wang commented on SPARK-24196: - The reason is that [HiveThriftServer2.scala#L

[jira] [Commented] (SPARK-25778) WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails due lack of access

2018-10-22 Thread Greg Senia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658934#comment-16658934 ] Greg Senia commented on SPARK-25778: [~hyukjin.kwon] thanks for the heads up on prio

[jira] [Commented] (SPARK-10816) EventTime based sessionization

2018-10-22 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658935#comment-16658935 ] Jungtaek Lim commented on SPARK-10816: -- Have been thinking about [3] but can't find

[jira] [Created] (SPARK-25799) DataSourceApiV2 scan reuse does not respect options

2018-10-22 Thread JIRA
Max Kießling created SPARK-25799: Summary: DataSourceApiV2 scan reuse does not respect options Key: SPARK-25799 URL: https://issues.apache.org/jira/browse/SPARK-25799 Project: Spark Issue Typ

[jira] [Comment Edited] (SPARK-10816) EventTime based sessionization

2018-10-22 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658935#comment-16658935 ] Jungtaek Lim edited comment on SPARK-10816 at 10/22/18 12:29 PM: -

[jira] [Assigned] (SPARK-25798) Internally document type conversion between Pandas data and SQL types in Pandas UDFs

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25798: Assignee: Apache Spark > Internally document type conversion between Pandas data and SQL

[jira] [Commented] (SPARK-25798) Internally document type conversion between Pandas data and SQL types in Pandas UDFs

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658945#comment-16658945 ] Apache Spark commented on SPARK-25798: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-25798) Internally document type conversion between Pandas data and SQL types in Pandas UDFs

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25798: Assignee: (was: Apache Spark) > Internally document type conversion between Pandas da

[jira] [Created] (SPARK-25800) Remove Netty 3.x

2018-10-22 Thread Zhu, Lipeng (JIRA)
Zhu, Lipeng created SPARK-25800: --- Summary: Remove Netty 3.x Key: SPARK-25800 URL: https://issues.apache.org/jira/browse/SPARK-25800 Project: Spark Issue Type: Task Components: Build

[jira] [Updated] (SPARK-25800) Remove all instances of Netty 3.x in the code base

2018-10-22 Thread Zhu, Lipeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu, Lipeng updated SPARK-25800: Summary: Remove all instances of Netty 3.x in the code base (was: Remove Netty 3.x) > Remove all

[jira] [Updated] (SPARK-25800) Remove all instances of Netty 3.x in the code base

2018-10-22 Thread Zhu, Lipeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu, Lipeng updated SPARK-25800: Description: Currently netty 3.x is unused in spark code base. We can remove all the instances of

[jira] [Commented] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-22 Thread Sanket Reddy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658985#comment-16658985 ] Sanket Reddy commented on SPARK-25692: -- Sure [~tgraves] [~zsxwing] i am taking a lo

[jira] [Commented] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658999#comment-16658999 ] Apache Spark commented on SPARK-24570: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24570: Assignee: Apache Spark > SparkSQL - show schemas/tables in dropdowns of SQL client tools

[jira] [Assigned] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24570: Assignee: (was: Apache Spark) > SparkSQL - show schemas/tables in dropdowns of SQL cl

[jira] [Commented] (SPARK-25795) Fix CSV SparkR SQL Example

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659000#comment-16659000 ] Apache Spark commented on SPARK-25795: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-25753) binaryFiles broken for small files

2018-10-22 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-25753: - Assignee: liuxian > binaryFiles broken for small files > --

[jira] [Resolved] (SPARK-25753) binaryFiles broken for small files

2018-10-22 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-25753. --- Resolution: Fixed Fix Version/s: 3.0.0 > binaryFiles broken for small files > ---

[jira] [Assigned] (SPARK-25800) Remove all instances of Netty 3.x in the code base

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25800: Assignee: (was: Apache Spark) > Remove all instances of Netty 3.x in the code base >

[jira] [Assigned] (SPARK-25800) Remove all instances of Netty 3.x in the code base

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25800: Assignee: Apache Spark > Remove all instances of Netty 3.x in the code base > ---

[jira] [Commented] (SPARK-25800) Remove all instances of Netty 3.x in the code base

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659031#comment-16659031 ] Apache Spark commented on SPARK-25800: -- User 'lipzhu' has created a pull request fo

[jira] [Created] (SPARK-25801) pandas_udf grouped_map fails with input dataframe with more than 255 columns

2018-10-22 Thread Frederik (JIRA)
Frederik created SPARK-25801: Summary: pandas_udf grouped_map fails with input dataframe with more than 255 columns Key: SPARK-25801 URL: https://issues.apache.org/jira/browse/SPARK-25801 Project: Spark

[jira] [Resolved] (SPARK-21993) Close CliSessionState in shutdown hook of SparkSQLCLIDriver

2018-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21993. --- Resolution: Not A Problem > Close CliSessionState in shutdown hook of SparkSQLCLIDriver > --

[jira] [Updated] (SPARK-25798) Internally document type conversion between Pandas data and SQL types in Pandas UDFs

2018-10-22 Thread Yuanjian Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuanjian Li updated SPARK-25798: Description: Currently, UDF's type coercion is not cleanly defined. See also https://github.com/a

[jira] [Commented] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659125#comment-16659125 ] Dongjoon Hyun commented on SPARK-20055: --- This looks too old. Are we working on thi

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659129#comment-16659129 ] Marcelo Vanzin commented on SPARK-25783: Hmm... if the jline classes are in the

[jira] [Updated] (SPARK-25802) Use JDBC Oracle Binds from Spark SQL

2018-10-22 Thread Nathan Loyer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Loyer updated SPARK-25802: - Component/s: (was: Input/Output) SQL > Use JDBC Oracle Binds from Spark SQL

[jira] [Created] (SPARK-25802) Use JDBC Oracle Binds from Spark SQL

2018-10-22 Thread Nathan Loyer (JIRA)
Nathan Loyer created SPARK-25802: Summary: Use JDBC Oracle Binds from Spark SQL Key: SPARK-25802 URL: https://issues.apache.org/jira/browse/SPARK-25802 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659165#comment-16659165 ] Apache Spark commented on SPARK-25793: -- User 'huaxingao' has created a pull request

[jira] [Resolved] (SPARK-25771) Fix improper synchronization in PythonWorkerFactory

2018-10-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-25771. -- Resolution: Fixed Fix Version/s: 3.0.0 > Fix improper synchronization in PythonWorkerFa

[jira] [Resolved] (SPARK-25507) Update documents for the new features in 2.4 release

2018-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25507. --- Resolution: Done > Update documents for the new features in 2.4 release > --

[jira] [Updated] (SPARK-25179) Document the features that require Pyarrow 0.10

2018-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25179: -- Issue Type: Task (was: Sub-task) Parent: (was: SPARK-25507) > Document the features that

[jira] [Commented] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2018-10-22 Thread Jorge Machado (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659253#comment-16659253 ] Jorge Machado commented on SPARK-20055: --- This was taken on another ticket. This Ca

[jira] [Resolved] (SPARK-25627) ContinuousStressSuite - 8 mins 13 sec

2018-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25627. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22662 [https://github.c

[jira] [Assigned] (SPARK-25627) ContinuousStressSuite - 8 mins 13 sec

2018-10-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-25627: - Assignee: Liang-Chi Hsieh > ContinuousStressSuite - 8 mins 13 sec > ---

[jira] [Commented] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659283#comment-16659283 ] Apache Spark commented on SPARK-19851: -- User 'dilipbiswal' has created a pull reque

[jira] [Commented] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659319#comment-16659319 ] Dongjoon Hyun commented on SPARK-20055: --- Thank you, [~jomach]. > Documentation fo

[jira] [Resolved] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-20055. --- Resolution: Fixed Fix Version/s: 2.3.0 The original PR was merged at 2.3.0. The furth

[jira] [Assigned] (SPARK-20055) Documentation for CSV datasets in SQL programming guide

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-20055: - Assignee: Jorge Machado > Documentation for CSV datasets in SQL programming guide > ---

[jira] [Commented] (SPARK-25801) pandas_udf grouped_map fails with input dataframe with more than 255 columns

2018-10-22 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659345#comment-16659345 ] Bryan Cutler commented on SPARK-25801: -- I was able to reproduce with Spark 2.3, and

[jira] [Commented] (SPARK-13587) Support virtualenv in PySpark

2018-10-22 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659374#comment-16659374 ] Ruslan Dautkhanov commented on SPARK-13587: --- We're using conda environments sh

[jira] [Updated] (SPARK-24432) Add support for dynamic resource allocation

2018-10-22 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li updated SPARK-24432: - Affects Version/s: 3.0.0 > Add support for dynamic resource allocation > ---

[jira] [Commented] (SPARK-25796) Enable external shuffle service for kubernetes mode.

2018-10-22 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659579#comment-16659579 ] Yinan Li commented on SPARK-25796: -- See https://issues.apache.org/jira/browse/SPARK-244

[jira] [Commented] (SPARK-25787) [K8S] Spark can't use data locality information

2018-10-22 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659593#comment-16659593 ] Yinan Li commented on SPARK-25787: -- Support for data locality on k8s has not been porte

[jira] [Updated] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-22 Thread Sanket Reddy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanket Reddy updated SPARK-25692: - Attachment: Screen Shot 2018-10-22 at 4.12.41 PM.png > Flaky test: ChunkFetchIntegrationSuite >

[jira] [Commented] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-22 Thread Sanket Reddy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659644#comment-16659644 ] Sanket Reddy commented on SPARK-25692: -- [~zsxwing] I haven't been able to reproduce

[jira] [Created] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-22 Thread Steve Larkin (JIRA)
Steve Larkin created SPARK-25803: Summary: The -n option to docker-image-tool.sh causes other options to be ignored Key: SPARK-25803 URL: https://issues.apache.org/jira/browse/SPARK-25803 Project: Spa

[jira] [Assigned] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25803: Assignee: (was: Apache Spark) > The -n option to docker-image-tool.sh causes other op

[jira] [Commented] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659693#comment-16659693 ] Apache Spark commented on SPARK-25803: -- User 'sel' has created a pull request for t

[jira] [Commented] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-22 Thread Steve Larkin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659694#comment-16659694 ] Steve Larkin commented on SPARK-25803: -- Created a PR: https://github.com/apache/sp

[jira] [Assigned] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25803: Assignee: Apache Spark > The -n option to docker-image-tool.sh causes other options to be

[jira] [Updated] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-22 Thread Steve Larkin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Larkin updated SPARK-25803: - Environment: * OS X 10.14 * iTerm2 * bash3 * Docker 2.0.0.0-beta1-mac75 (27117) (NB: I don't

[jira] [Commented] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659734#comment-16659734 ] Dongjoon Hyun commented on SPARK-25793: --- Hi, [~WeichenXu123]. Could you tell us wh

[jira] [Commented] (SPARK-25678) SPIP: Adding support in Spark for HPC cluster manager (PBS Professional)

2018-10-22 Thread Ilan Filonenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659756#comment-16659756 ] Ilan Filonenko commented on SPARK-25678: Would recommend to look at: https://jir

[jira] [Updated] (SPARK-25795) Fix CSV SparkR SQL Example

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25795: -- Fix Version/s: (was: 2.4.0) 2.4.1 > Fix CSV SparkR SQL Example > --

[jira] [Resolved] (SPARK-25795) Fix CSV SparkR SQL Example

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25795. --- Resolution: Fixed Fix Version/s: 2.4.0 3.0.0 2.3

[jira] [Created] (SPARK-25804) JDOPersistenceManager leak when query via JDBC

2018-10-22 Thread pin_zhang (JIRA)
pin_zhang created SPARK-25804: - Summary: JDOPersistenceManager leak when query via JDBC Key: SPARK-25804 URL: https://issues.apache.org/jira/browse/SPARK-25804 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-25800) Remove all instances of Netty 3.x in the code base

2018-10-22 Thread Zhu, Lipeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhu, Lipeng resolved SPARK-25800. - Resolution: Won't Do > Remove all instances of Netty 3.x in the code base >

[jira] [Commented] (SPARK-24192) Invalid Spark URL in local spark session since upgrading from org.apache.spark:spark-sql_2.11:2.2.1 to org.apache.spark:spark-sql_2.11:2.3.0

2018-10-22 Thread Joel Bondurant (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659913#comment-16659913 ] Joel Bondurant commented on SPARK-24192: Thanks ^^^Prabhu^^^! Downgrading to Spa

[jira] [Commented] (SPARK-25794) Update 1.6 REST API documentation to clarify [app-id]

2018-10-22 Thread Brian Burton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659918#comment-16659918 ] Brian Burton commented on SPARK-25794: -- Sorry, [~hyukjin.kwon], I didn't realize th

[jira] [Updated] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-25793: --- Priority: Major (was: Minor) > Loading model bug in BisectingKMeans > -

[jira] [Updated] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-25793: --- Issue Type: Bug (was: Documentation) > Loading model bug in BisectingKMeans > -

[jira] [Commented] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659938#comment-16659938 ] Weichen Xu commented on SPARK-25793: [~dongjoon] Sorry. Change type to Bug and prior

[jira] [Created] (SPARK-25805) Flaky test: DataFrameSuite.SPARK-25159 unittest failure

2018-10-22 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-25805: Summary: Flaky test: DataFrameSuite.SPARK-25159 unittest failure Key: SPARK-25805 URL: https://issues.apache.org/jira/browse/SPARK-25805 Project: Spark Issue

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659946#comment-16659946 ] Wenchen Fan commented on SPARK-25783: - Since it's not a blocker, and we are not able

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659947#comment-16659947 ] Wenchen Fan commented on SPARK-25783: - I'll put it as a known issue in 2.4.0 release

[jira] [Updated] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25783: Target Version/s: 3.0.0 (was: 2.4.0) > Spark shell fails because of jline incompatibility > -

[jira] [Updated] (SPARK-22809) pyspark is sensitive to imports with dots

2018-10-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-22809: Target Version/s: (was: 2.4.0) > pyspark is sensitive to imports with dots > ---

[jira] [Commented] (SPARK-22809) pyspark is sensitive to imports with dots

2018-10-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659950#comment-16659950 ] Wenchen Fan commented on SPARK-22809: - is this issue fixed? Anyway I'm removing the

[jira] [Commented] (SPARK-21030) extend hint syntax to support any expression for Python and R

2018-10-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659952#comment-16659952 ] Wenchen Fan commented on SPARK-21030: - I'm removing the target version, since it's n

[jira] [Updated] (SPARK-21030) extend hint syntax to support any expression for Python and R

2018-10-22 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21030: Target Version/s: (was: 2.4.0) > extend hint syntax to support any expression for Python and R >

[jira] [Assigned] (SPARK-25805) Flaky test: DataFrameSuite.SPARK-25159 unittest failure

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25805: Assignee: (was: Apache Spark) > Flaky test: DataFrameSuite.SPARK-25159 unittest failu

[jira] [Commented] (SPARK-25805) Flaky test: DataFrameSuite.SPARK-25159 unittest failure

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659961#comment-16659961 ] Apache Spark commented on SPARK-25805: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-25159) json schema inference should only trigger one job

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659963#comment-16659963 ] Apache Spark commented on SPARK-25159: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-25805) Flaky test: DataFrameSuite.SPARK-25159 unittest failure

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659962#comment-16659962 ] Apache Spark commented on SPARK-25805: -- User 'squito' has created a pull request fo

[jira] [Assigned] (SPARK-25805) Flaky test: DataFrameSuite.SPARK-25159 unittest failure

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25805: Assignee: Apache Spark > Flaky test: DataFrameSuite.SPARK-25159 unittest failure > --

[jira] [Commented] (SPARK-25159) json schema inference should only trigger one job

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659964#comment-16659964 ] Apache Spark commented on SPARK-25159: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-25159) json schema inference should only trigger one job

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659970#comment-16659970 ] Apache Spark commented on SPARK-25159: -- User 'squito' has created a pull request fo

[jira] [Commented] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-22 Thread Michael Heuer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659990#comment-16659990 ] Michael Heuer commented on SPARK-25588: --- > Can you try to downgrade parquet and se

[jira] [Commented] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659993#comment-16659993 ] Apache Spark commented on SPARK-24499: -- User 'kiszk' has created a pull request for

[jira] [Commented] (SPARK-25793) Loading model bug in BisectingKMeans

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16660006#comment-16660006 ] Dongjoon Hyun commented on SPARK-25793: --- Thank you, [~WeichenXu123]. > Loading mo

[jira] [Commented] (SPARK-25678) SPIP: Adding support in Spark for HPC cluster manager (PBS Professional)

2018-10-22 Thread Utkarsh Maheshwari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16660078#comment-16660078 ] Utkarsh Maheshwari commented on SPARK-25678: {quote}Would recommend to look

[jira] [Updated] (SPARK-25656) Add an example section about how to use Parquet/ORC library options

2018-10-22 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25656: -- Component/s: Examples > Add an example section about how to use Parquet/ORC library options >

[jira] [Commented] (SPARK-25656) Add an example section about how to use Parquet/ORC library options

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16660091#comment-16660091 ] Apache Spark commented on SPARK-25656: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-25656) Add an example section about how to use Parquet/ORC library options

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25656: Assignee: (was: Apache Spark) > Add an example section about how to use Parquet/ORC l

[jira] [Assigned] (SPARK-25656) Add an example section about how to use Parquet/ORC library options

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25656: Assignee: Apache Spark > Add an example section about how to use Parquet/ORC library opti

[jira] [Commented] (SPARK-25656) Add an example section about how to use Parquet/ORC library options

2018-10-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16660092#comment-16660092 ] Apache Spark commented on SPARK-25656: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-25040) Empty string for double and float types should be nulls in JSON

2018-10-22 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25040: Assignee: Liang-Chi Hsieh > Empty string for double and float types should be nulls in J

[jira] [Resolved] (SPARK-25040) Empty string for double and float types should be nulls in JSON

2018-10-22 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25040. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22787 [https://gi

[jira] [Resolved] (SPARK-25796) Enable external shuffle service for kubernetes mode.

2018-10-22 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma resolved SPARK-25796. - Resolution: Duplicate > Enable external shuffle service for kubernetes mode. > -

  1   2   >