[jira] [Assigned] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

2018-07-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24706: --- Assignee: Yuming Wang > Support ByteType and ShortType pushdown to parquet >

[jira] [Updated] (SPARK-24706) Support ByteType and ShortType pushdown to parquet

2018-07-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24706: Target Version/s: 2.4.0 > Support ByteType and ShortType pushdown to parquet >

[jira] [Commented] (SPARK-24714) AnalysisSuite should use ClassTag to check the runtime instance

2018-07-01 Thread Chia-Ping Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529405#comment-16529405 ] Chia-Ping Tsai commented on SPARK-24714: [~maropu] thank you again. :) > AnalysisSuite should

[jira] [Commented] (SPARK-24708) Document the default spark url of master in standalone is "spark://localhost:7070"

2018-07-01 Thread Chia-Ping Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529404#comment-16529404 ] Chia-Ping Tsai commented on SPARK-24708: [~maropu] Thanks for the kind reminder. > Document the

[jira] [Commented] (SPARK-24708) Document the default spark url of master in standalone is "spark://localhost:7070"

2018-07-01 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529394#comment-16529394 ] Takeshi Yamamuro commented on SPARK-24708: -- Feel free to make a pr and you can discuss there.

[jira] [Updated] (SPARK-24715) sbt build brings a wrong jline versions

2018-07-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24715: - Priority: Critical (was: Major) > sbt build brings a wrong jline versions >

[jira] [Updated] (SPARK-24715) sbt build brings a wrong jline versions

2018-07-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24715: -- Description: During SPARK-24418 (Upgrade Scala to 2.11.12 and 2.12.6), we upgrade `jline`

[jira] [Updated] (SPARK-24715) sbt build brings a wrong jline versions

2018-07-01 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24715: -- Summary: sbt build brings a wrong jline versions (was: sbt build bring a wrong jline

[jira] [Created] (SPARK-24715) sbt build bring a wrong jline versions

2018-07-01 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-24715: - Summary: sbt build bring a wrong jline versions Key: SPARK-24715 URL: https://issues.apache.org/jira/browse/SPARK-24715 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-24714) AnalysisSuite should use ClassTag to check the runtime instance

2018-07-01 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529373#comment-16529373 ] Takeshi Yamamuro commented on SPARK-24714: -- You needn't do that; feel free to make a pr for

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-07-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529371#comment-16529371 ] Hyukjin Kwon commented on SPARK-24530: -- Yea, will post a email to related threads, and try to deal

[jira] [Assigned] (SPARK-24713) AppMatser of spark streaming kafka OOM if there are hundreds of topics consumed

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24713: Assignee: Apache Spark > AppMatser of spark streaming kafka OOM if there are hundreds of

[jira] [Assigned] (SPARK-24713) AppMatser of spark streaming kafka OOM if there are hundreds of topics consumed

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24713: Assignee: (was: Apache Spark) > AppMatser of spark streaming kafka OOM if there are

[jira] [Commented] (SPARK-24713) AppMatser of spark streaming kafka OOM if there are hundreds of topics consumed

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529367#comment-16529367 ] Apache Spark commented on SPARK-24713: -- User 'yuanboliu' has created a pull request for this issue:

[jira] [Commented] (SPARK-24528) Missing optimization for Aggregations/Windowing on a bucketed table

2018-07-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529361#comment-16529361 ] Liang-Chi Hsieh commented on SPARK-24528: - I think we can have a sql config to control

[jira] [Commented] (SPARK-24714) AnalysisSuite should use ClassTag to check the runtime instance

2018-07-01 Thread Chia-Ping Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529356#comment-16529356 ] Chia-Ping Tsai commented on SPARK-24714: I have no permission to assign this Jira to myself.

[jira] [Created] (SPARK-24714) AnalysisSuite should use ClassTag to check the runtime instance

2018-07-01 Thread Chia-Ping Tsai (JIRA)
Chia-Ping Tsai created SPARK-24714: -- Summary: AnalysisSuite should use ClassTag to check the runtime instance Key: SPARK-24714 URL: https://issues.apache.org/jira/browse/SPARK-24714 Project: Spark

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-07-01 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529352#comment-16529352 ] Saisai Shao commented on SPARK-24530: - [~hyukjin.kwon], Spark 2.1.3 and 2.2.2 are on vote, can you

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

2018-07-01 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24530: Target Version/s: 2.1.3, 2.2.2, 2.3.2, 2.4.0 (was: 2.4.0) > Sphinx doesn't render

[jira] [Created] (SPARK-24713) AppMatser of spark streaming kafka OOM if there are hundreds of topics consumed

2018-07-01 Thread Yuanbo Liu (JIRA)
Yuanbo Liu created SPARK-24713: -- Summary: AppMatser of spark streaming kafka OOM if there are hundreds of topics consumed Key: SPARK-24713 URL: https://issues.apache.org/jira/browse/SPARK-24713 Project:

[jira] [Updated] (SPARK-24711) Integration tests will not work with exclude/include tags

2018-07-01 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24711: Priority: Minor (was: Major) > Integration tests will not work with

[jira] [Commented] (SPARK-21809) Change Stage Page to use datatables to support sorting columns and searching

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529206#comment-16529206 ] Apache Spark commented on SPARK-21809: -- User 'pgandhi999' has created a pull request for this

[jira] [Updated] (SPARK-24712) TrainValidationSplit ignores label column name and forces to be "label"

2018-07-01 Thread Pablo J. Villacorta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pablo J. Villacorta updated SPARK-24712: Description: When a TrainValidationSplit is fit on a Pipeline containing a ML

[jira] [Created] (SPARK-24712) TrainValidationSplit ignores label column name and forces to be "label"

2018-07-01 Thread Pablo J. Villacorta (JIRA)
Pablo J. Villacorta created SPARK-24712: --- Summary: TrainValidationSplit ignores label column name and forces to be "label" Key: SPARK-24712 URL: https://issues.apache.org/jira/browse/SPARK-24712

[jira] [Updated] (SPARK-24711) Integration tests will not work with exclude/include tags

2018-07-01 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24711: Description: I tried to exclude some tests when adding mine and I got errors of

[jira] [Updated] (SPARK-24711) Integration tests will not work with exclude/include tags

2018-07-01 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24711: Description: I tried to exclude some tests when adding mine and I got errors of

[jira] [Created] (SPARK-24711) Integration tests will not work with exclude/include tags

2018-07-01 Thread Stavros Kontopoulos (JIRA)
Stavros Kontopoulos created SPARK-24711: --- Summary: Integration tests will not work with exclude/include tags Key: SPARK-24711 URL: https://issues.apache.org/jira/browse/SPARK-24711 Project:

[jira] [Commented] (SPARK-24165) UDF within when().otherwise() raises NullPointerException

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529108#comment-16529108 ] Apache Spark commented on SPARK-24165: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24165) UDF within when().otherwise() raises NullPointerException

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24165: Assignee: Apache Spark > UDF within when().otherwise() raises NullPointerException >

[jira] [Assigned] (SPARK-24165) UDF within when().otherwise() raises NullPointerException

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24165: Assignee: (was: Apache Spark) > UDF within when().otherwise() raises

[jira] [Created] (SPARK-24710) Information Gain Ratio for decision trees

2018-07-01 Thread Pablo J. Villacorta (JIRA)
Pablo J. Villacorta created SPARK-24710: --- Summary: Information Gain Ratio for decision trees Key: SPARK-24710 URL: https://issues.apache.org/jira/browse/SPARK-24710 Project: Spark

[jira] [Comment Edited] (SPARK-24642) Add a function which infers schema from a JSON column

2018-07-01 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529030#comment-16529030 ] Maxim Gekk edited comment on SPARK-24642 at 7/1/18 10:05 AM: - [~rxin] I

[jira] [Assigned] (SPARK-24709) Inferring schema from JSON string literal

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24709: Assignee: Apache Spark > Inferring schema from JSON string literal >

[jira] [Assigned] (SPARK-24709) Inferring schema from JSON string literal

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24709: Assignee: (was: Apache Spark) > Inferring schema from JSON string literal >

[jira] [Commented] (SPARK-24709) Inferring schema from JSON string literal

2018-07-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529038#comment-16529038 ] Apache Spark commented on SPARK-24709: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-24642) Add a function which infers schema from a JSON column

2018-07-01 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529030#comment-16529030 ] Maxim Gekk commented on SPARK-24642: I created new ticket SPARK-24709 which aims to add simpler

[jira] [Resolved] (SPARK-24642) Add a function which infers schema from a JSON column

2018-07-01 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-24642. Resolution: Won't Fix > Add a function which infers schema from a JSON column >

[jira] [Created] (SPARK-24709) Inferring schema from JSON string literal

2018-07-01 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-24709: -- Summary: Inferring schema from JSON string literal Key: SPARK-24709 URL: https://issues.apache.org/jira/browse/SPARK-24709 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-24621) WebUI - application 'name' urls point to http instead of https (even when ssl enabled)

2018-07-01 Thread t oo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528572#comment-16528572 ] t oo edited comment on SPARK-24621 at 7/1/18 9:36 AM: --

[jira] [Created] (SPARK-24708) Document the default spark url of master in standalone is "spark://localhost:7070"

2018-07-01 Thread Chia-Ping Tsai (JIRA)
Chia-Ping Tsai created SPARK-24708: -- Summary: Document the default spark url of master in standalone is "spark://localhost:7070" Key: SPARK-24708 URL: https://issues.apache.org/jira/browse/SPARK-24708

[jira] [Commented] (SPARK-20427) Issue with Spark interpreting Oracle datatype NUMBER

2018-07-01 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529013#comment-16529013 ] Yuming Wang commented on SPARK-20427: - [~ORichard]. Please try to use {{customSchema}} to specifying

[jira] [Commented] (SPARK-20427) Issue with Spark interpreting Oracle datatype NUMBER

2018-07-01 Thread Oliver Richardson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529005#comment-16529005 ] Oliver Richardson commented on SPARK-20427: --- I'm still getting the same problem even in newest