[jira] [Updated] (SPARK-26260) Task Summary Metrics for Stage Page: Efficient implementation for SHS when using disk store.

2018-12-03 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26260: --- Summary: Task Summary Metrics for Stage Page: Efficient implementation for SHS when using disk store.

[jira] [Updated] (SPARK-26260) Summary Task Metrics for Stage Page: Efficient implementation for SHS when using disk store.

2018-12-03 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26260: --- Summary: Summary Task Metrics for Stage Page: Efficient implementation for SHS when using disk store.

[jira] [Resolved] (SPARK-25573) Combine resolveExpression and resolve in the rule ResolveReferences

2018-12-03 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25573. - Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 3.0.0 > Combine

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708243#comment-16708243 ] Apache Spark commented on SPARK-26155: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708241#comment-16708241 ] Apache Spark commented on SPARK-26155: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-26206) Spark structured streaming with kafka integration fails in update mode

2018-12-03 Thread indraneel r (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708245#comment-16708245 ] indraneel r commented on SPARK-26206: - [~kabhwan]  Here are some of my observations:  - The error

[jira] [Assigned] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26155: Assignee: (was: Apache Spark) > Spark SQL performance degradation after apply

[jira] [Assigned] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26155: Assignee: Apache Spark > Spark SQL performance degradation after apply SPARK-21052 with

[jira] [Commented] (SPARK-26051) Can't create table with column name '22222d'

2018-12-03 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708233#comment-16708233 ] Dilip Biswal commented on SPARK-26051: -- [~xiejuntao1...@163.com] Hello, i took a quick look at

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708225#comment-16708225 ] Wenchen Fan commented on SPARK-26155: - great to hear that! looking forward to your patch :) > Spark

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-12-03 Thread Ke Jia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708223#comment-16708223 ] Ke Jia commented on SPARK-26155: [~cloud_fan] [~viirya]  Spark2.3 with the optimized patch can have the

[jira] [Commented] (SPARK-26244) Do not use case class as public API

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708220#comment-16708220 ] Wenchen Fan commented on SPARK-26244: - I took a quick look, and seems most of the public case class

[jira] [Commented] (SPARK-26155) Spark SQL performance degradation after apply SPARK-21052 with Q19 of TPC-DS in 3TB scale

2018-12-03 Thread Yang Jie (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708219#comment-16708219 ] Yang Jie commented on SPARK-26155: -- [~cloud_fan]  [~viirya] maybe we needn't revert this patch, offline

[jira] [Commented] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708218#comment-16708218 ] Hyukjin Kwon commented on SPARK-26259: -- Can you try the examples against the current master branch

[jira] [Assigned] (SPARK-26262) Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26262: Assignee: Apache Spark > Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false >

[jira] [Commented] (SPARK-26262) Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708184#comment-16708184 ] Apache Spark commented on SPARK-26262: -- User 'maropu' has created a pull request for this issue:

[jira] [Commented] (SPARK-26262) Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708185#comment-16708185 ] Apache Spark commented on SPARK-26262: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26262) Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26262: Assignee: (was: Apache Spark) > Run SQLQueryTestSuite with

[jira] [Created] (SPARK-26262) Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false

2018-12-03 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-26262: Summary: Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false Key: SPARK-26262 URL: https://issues.apache.org/jira/browse/SPARK-26262 Project: Spark

[jira] [Comment Edited] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread PoojaMurarka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708161#comment-16708161 ] PoojaMurarka edited comment on SPARK-26259 at 12/4/18 4:14 AM: --- The fix

[jira] [Commented] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread PoojaMurarka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708161#comment-16708161 ] PoojaMurarka commented on SPARK-26259: -- The fix for using custom record delimiters seems to be only

[jira] [Commented] (SPARK-26261) Spark does not check completeness temporary file

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708153#comment-16708153 ] Hyukjin Kwon commented on SPARK-26261: -- Mind if I ask the initial test you ran? > Spark does not

[jira] [Commented] (SPARK-26251) isnan function not picking non-numeric values

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708129#comment-16708129 ] Hyukjin Kwon commented on SPARK-26251: -- Why does it should pick "po box 7896"? > isnan function

[jira] [Commented] (SPARK-26250) Fail to run dataframe.R examples

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708125#comment-16708125 ] Hyukjin Kwon commented on SPARK-26250: -- Please avoid to set a target version which is usually

[jira] [Resolved] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26259. -- Resolution: Duplicate > RecordSeparator other than newline discovers incorrect schema >

[jira] [Updated] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26259: - Component/s: (was: Spark Core) SQL > RecordSeparator other than newline

[jira] [Commented] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708136#comment-16708136 ] Hyukjin Kwon commented on SPARK-26259: -- This is fixed in SPARK-26108 which should be availabe from

[jira] [Updated] (SPARK-21289) Text based formats do not support custom end-of-line delimiters

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-21289: - Priority: Major (was: Minor) > Text based formats do not support custom end-of-line delimiters

[jira] [Commented] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708134#comment-16708134 ] Hyukjin Kwon commented on SPARK-26259: -- please avoid to set the fixed version which is usually set

[jira] [Updated] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26259: - Fix Version/s: (was: 2.4.1) > RecordSeparator other than newline discovers incorrect schema

[jira] [Commented] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708132#comment-16708132 ] Hyukjin Kwon commented on SPARK-26255: -- it would be great if some snapshots from UI is uploaded

[jira] [Updated] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26255: - Component/s: (was: Spark Core) Web UI SQL > Custom

[jira] [Commented] (SPARK-26149) Read UTF8String from Parquet/ORC may be incorrect

2018-12-03 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708119#comment-16708119 ] Yuming Wang commented on SPARK-26149: - This is not a Spark bug, but a Hive bug.

[jira] [Issue Comment Deleted] (SPARK-26251) isnan function not picking non-numeric values

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26251: - Comment: was deleted (was: Why does it should pick "po box 7896"?) > isnan function not

[jira] [Updated] (SPARK-26251) isnan function not picking non-numeric values

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26251: - Component/s: (was: Spark Core) SQL > isnan function not picking

[jira] [Commented] (SPARK-26251) isnan function not picking non-numeric values

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708128#comment-16708128 ] Hyukjin Kwon commented on SPARK-26251: -- Why does it should pick "po box 7896"? > isnan function

[jira] [Updated] (SPARK-26149) Read UTF8String from Parquet/ORC may be incorrect

2018-12-03 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-26149: Attachment: image-2018-12-04-10-55-49-369.png > Read UTF8String from Parquet/ORC may be incorrect

[jira] [Commented] (SPARK-26250) Fail to run dataframe.R examples

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708126#comment-16708126 ] Hyukjin Kwon commented on SPARK-26250: -- The error looks literally {{Rscript}} is not installed in

[jira] [Resolved] (SPARK-26250) Fail to run dataframe.R examples

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26250. -- Resolution: Invalid > Fail to run dataframe.R examples > > >

[jira] [Updated] (SPARK-26250) Fail to run dataframe.R examples

2018-12-03 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26250: - Target Version/s: (was: 2.4.0) > Fail to run dataframe.R examples >

[jira] [Created] (SPARK-26261) Spark does not check completeness temporary file

2018-12-03 Thread Jialin LIu (JIRA)
Jialin LIu created SPARK-26261: -- Summary: Spark does not check completeness temporary file Key: SPARK-26261 URL: https://issues.apache.org/jira/browse/SPARK-26261 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-26149) Read UTF8String from Parquet/ORC may be incorrect

2018-12-03 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-26149. - Resolution: Not A Problem > Read UTF8String from Parquet/ORC may be incorrect >

[jira] [Updated] (SPARK-26260) Summary Task Metrics for Stage Page: Efficient implimentation for SHS when using disk store.

2018-12-03 Thread shahid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shahid updated SPARK-26260: --- Description: Currently, tasks summary metrics is calculated based on all the tasks, instead of successful

[jira] [Commented] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16708027#comment-16708027 ] Apache Spark commented on SPARK-25498: -- User 'maropu' has created a pull request for this issue:

[jira] [Created] (SPARK-26260) Summary Task Metrics for Stage Page: Efficient implimentation for SHS when using disk store.

2018-12-03 Thread shahid (JIRA)
shahid created SPARK-26260: -- Summary: Summary Task Metrics for Stage Page: Efficient implimentation for SHS when using disk store. Key: SPARK-26260 URL: https://issues.apache.org/jira/browse/SPARK-26260

[jira] [Assigned] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-12-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26083: -- Assignee: Qi Shao > Pyspark command is not working properly with default Docker

[jira] [Comment Edited] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2018-12-03 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707997#comment-16707997 ] Stavros Kontopoulos edited comment on SPARK-26247 at 12/3/18 11:59 PM:

[jira] [Commented] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2018-12-03 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707997#comment-16707997 ] Stavros Kontopoulos commented on SPARK-26247: - Hi [~aholler], Yes I agree there are

[jira] [Resolved] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-12-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26083. Resolution: Fixed Fix Version/s: (was: 2.4.1) 3.0.0 Issue

[jira] [Updated] (SPARK-26219) Executor summary is not getting updated for failure jobs in history server UI

2018-12-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-26219: --- Fix Version/s: 2.4.1 > Executor summary is not getting updated for failure jobs in history

[jira] [Commented] (SPARK-26239) Add configurable auth secret source in k8s backend

2018-12-03 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707962#comment-16707962 ] Matt Cheah commented on SPARK-26239: It could work in client mode but is less useful there overall

[jira] [Updated] (SPARK-26256) Add proper labels when deleting pods

2018-12-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-26256: --- Fix Version/s: 2.4.1 > Add proper labels when deleting pods >

[jira] [Assigned] (SPARK-26256) Add proper labels when deleting pods

2018-12-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26256: -- Assignee: Stavros Kontopoulos > Add proper labels when deleting pods >

[jira] [Resolved] (SPARK-26256) Add proper labels when deleting pods

2018-12-03 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26256. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23209

[jira] [Commented] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2018-12-03 Thread Anne Holler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707907#comment-16707907 ] Anne Holler commented on SPARK-26247: - Hi, [~skonto], My basic take on model representation is that

[jira] [Commented] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707901#comment-16707901 ] Apache Spark commented on SPARK-19712: -- User 'dilipbiswal' has created a pull request for this

[jira] [Commented] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707902#comment-16707902 ] Apache Spark commented on SPARK-19712: -- User 'dilipbiswal' has created a pull request for this

[jira] [Created] (SPARK-26259) RecordSeparator other than newline discovers incorrect schema

2018-12-03 Thread PoojaMurarka (JIRA)
PoojaMurarka created SPARK-26259: Summary: RecordSeparator other than newline discovers incorrect schema Key: SPARK-26259 URL: https://issues.apache.org/jira/browse/SPARK-26259 Project: Spark

[jira] [Commented] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2018-12-03 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707856#comment-16707856 ] Stavros Kontopoulos commented on SPARK-26247: - Some pipelines export models in pmml (I know

[jira] [Commented] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2018-12-03 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707815#comment-16707815 ] Stavros Kontopoulos commented on SPARK-26247: - Thanx a lot! If you make it accessible would

[jira] [Commented] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2018-12-03 Thread Anne Holler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707812#comment-16707812 ] Anne Holler commented on SPARK-26247: - Hi, [~skonto], I just attached a pdf of the document to this

[jira] [Updated] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2018-12-03 Thread Anne Holler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Holler updated SPARK-26247: Attachment: SPIPMlModelExtensionForOnlineServing.pdf > SPIP - ML Model Extension for no-Spark

[jira] [Commented] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2018-12-03 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707798#comment-16707798 ] Stavros Kontopoulos commented on SPARK-26247: - [~aholler] could you provide public access to

[jira] [Commented] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707738#comment-16707738 ] Dongjoon Hyun commented on SPARK-26233: --- Thanks. I raised the priority because this is a

[jira] [Updated] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26233: -- Component/s: (was: Java API) SQL > Incorrect decimal value with java

[jira] [Commented] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707729#comment-16707729 ] Marco Gaido commented on SPARK-26233: - [~dongjoon] I think so. SPARK-24957 was a long standing issue

[jira] [Updated] (SPARK-26258) Include JobGroupId in the callerContext

2018-12-03 Thread Aihua Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated SPARK-26258: - Description: SPARK-15857 adds the support of callerContext for HDFS and Yarn. Currently, Spark

[jira] [Updated] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26233: -- Priority: Blocker (was: Minor) > Incorrect decimal value with java beans and

[jira] [Updated] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26233: -- Labels: correctness (was: ) > Incorrect decimal value with java beans and first/last/max...

[jira] [Commented] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707717#comment-16707717 ] Dongjoon Hyun commented on SPARK-26233: --- Hi, [~mcanes] and [~mgaido]. This seems to exist for a

[jira] [Updated] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26233: -- Affects Version/s: 2.0.2 > Incorrect decimal value with java beans and first/last/max...

[jira] [Updated] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26233: -- Affects Version/s: 2.1.3 > Incorrect decimal value with java beans and first/last/max...

[jira] [Updated] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26233: -- Affects Version/s: 2.2.2 > Incorrect decimal value with java beans and first/last/max...

[jira] [Created] (SPARK-26258) Include JobGroupId in the callerContext

2018-12-03 Thread Aihua Xu (JIRA)
Aihua Xu created SPARK-26258: Summary: Include JobGroupId in the callerContext Key: SPARK-26258 URL: https://issues.apache.org/jira/browse/SPARK-26258 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-26257) SPIP: Interop Support for Spark Language Extensions

2018-12-03 Thread Tyson Condie (JIRA)
Tyson Condie created SPARK-26257: Summary: SPIP: Interop Support for Spark Language Extensions Key: SPARK-26257 URL: https://issues.apache.org/jira/browse/SPARK-26257 Project: Spark Issue

[jira] [Commented] (SPARK-26254) Move delegation token providers into a separate project

2018-12-03 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707679#comment-16707679 ] Steve Loughran commented on SPARK-26254: +HBase I don't have any opinions on the best place;

[jira] [Resolved] (SPARK-25515) Add a config property for disabling auto deletion of PODS for debugging.

2018-12-03 Thread Yinan Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yinan Li resolved SPARK-25515. -- Resolution: Fixed Fix Version/s: 3.0.0 > Add a config property for disabling auto deletion of

[jira] [Comment Edited] (SPARK-26213) Custom Receiver for Structured streaming

2018-12-03 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16705886#comment-16705886 ] Gabor Somogyi edited comment on SPARK-26213 at 12/3/18 4:27 PM: On the

[jira] [Assigned] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26233: Assignee: Apache Spark > Incorrect decimal value with java beans and first/last/max...

[jira] [Commented] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707465#comment-16707465 ] Apache Spark commented on SPARK-26233: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26233) Incorrect decimal value with java beans and first/last/max... functions

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26233: Assignee: (was: Apache Spark) > Incorrect decimal value with java beans and

[jira] [Resolved] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25498. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22512

[jira] [Assigned] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25498: --- Assignee: Takeshi Yamamuro > Fix SQLQueryTestSuite failures when the interpreter mode

[jira] [Resolved] (SPARK-26235) Change log level for ClassNotFoundException/NoClassDefFoundError in SparkSubmit to Error

2018-12-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26235. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23189

[jira] [Assigned] (SPARK-26235) Change log level for ClassNotFoundException/NoClassDefFoundError in SparkSubmit to Error

2018-12-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26235: - Assignee: Gengliang Wang > Change log level for ClassNotFoundException/NoClassDefFoundError in

[jira] [Assigned] (SPARK-26181) the `hasMinMaxStats` method of `ColumnStatsMap` is not correct

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26181: --- Assignee: Adrian Wang > the `hasMinMaxStats` method of `ColumnStatsMap` is not correct >

[jira] [Resolved] (SPARK-26181) the `hasMinMaxStats` method of `ColumnStatsMap` is not correct

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26181. - Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by pull

[jira] [Updated] (SPARK-26173) Prior regularization for Logistic Regression

2018-12-03 Thread Facundo Bellosi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Facundo Bellosi updated SPARK-26173: Description: This feature enables Maximum A Posteriori (MAP) optimization for Logistic

[jira] [Assigned] (SPARK-26256) Add proper labels when deleting pods

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26256: Assignee: Apache Spark > Add proper labels when deleting pods >

[jira] [Commented] (SPARK-25530) data source v2 API refactor (batch write)

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707365#comment-16707365 ] Apache Spark commented on SPARK-25530: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26256) Add proper labels when deleting pods

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26256: Assignee: (was: Apache Spark) > Add proper labels when deleting pods >

[jira] [Commented] (SPARK-26256) Add proper labels when deleting pods

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707370#comment-16707370 ] Apache Spark commented on SPARK-26256: -- User 'skonto' has created a pull request for this issue:

[jira] [Commented] (SPARK-25530) data source v2 API refactor (batch write)

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707361#comment-16707361 ] Apache Spark commented on SPARK-25530: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25530) data source v2 API refactor (batch write)

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25530: Assignee: (was: Apache Spark) > data source v2 API refactor (batch write) >

[jira] [Assigned] (SPARK-25530) data source v2 API refactor (batch write)

2018-12-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25530: Assignee: Apache Spark > data source v2 API refactor (batch write) >

[jira] [Updated] (SPARK-25530) data source v2 API refactor (batch write)

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25530: Summary: data source v2 API refactor (batch write) (was: data source v2 write side API

[jira] [Updated] (SPARK-25530) data source v2 API refactor (batch write)

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25530: Description: Adjust the batch write API to match the read API after refactor > data source v2 API

[jira] [Updated] (SPARK-25530) data source v2 API refactor (batch write)

2018-12-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25530: Description: (was: refactor the write side API according to this abstraction {code} batch:

[jira] [Resolved] (SPARK-26253) Task Summary Metrics Table on Stage Page shows empty table when no data is present

2018-12-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26253. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23205

[jira] [Assigned] (SPARK-26253) Task Summary Metrics Table on Stage Page shows empty table when no data is present

2018-12-03 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26253: - Assignee: Parth Gandhi This should just be a follow-up to SPARK-21089, not a new issue. >

  1   2   >