[jira] [Commented] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-18 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425995#comment-15425995 ] Yi Zhou commented on SPARK-17122: - Thanks [~dongjoon] for your clarity.. > Failed to dro

[jira] [Commented] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-17 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425926#comment-15425926 ] Yi Zhou commented on SPARK-17122: - Thanks [~dongjoon] for your quick response. But the pr

[jira] [Created] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-17 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-17122: --- Summary: Failed to drop database when use the database in Spark 2.0 Key: SPARK-17122 URL: https://issues.apache.org/jira/browse/SPARK-17122 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16515) [SPARK][SQL] transformation script got failure for python script

2016-07-12 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374234#comment-15374234 ] Yi Zhou commented on SPARK-16515: - Thank you for your quick attention for this issue. >

[jira] [Created] (SPARK-16515) [SPARK][SQL] transformation script got failure for python script

2016-07-12 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-16515: --- Summary: [SPARK][SQL] transformation script got failure for python script Key: SPARK-16515 URL: https://issues.apache.org/jira/browse/SPARK-16515 Project: Spark Issue

[jira] [Created] (SPARK-16415) [Spark][SQL] - Failed to create table due to catalog string error

2016-07-07 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-16415: --- Summary: [Spark][SQL] - Failed to create table due to catalog string error Key: SPARK-16415 URL: https://issues.apache.org/jira/browse/SPARK-16415 Project: Spark Issu

[jira] [Commented] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-07-05 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363691#comment-15363691 ] Yi Zhou commented on SPARK-15730: - Thanks a lot [~chenghao] and [~yhuai] ! > [Spark SQL]

[jira] [Commented] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-06-14 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15329408#comment-15329408 ] Yi Zhou commented on SPARK-15730: - Hi [~chenghao] This PR is tested and it's OK for my ca

[jira] [Created] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-06-02 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-15730: --- Summary: [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session Key: SPARK-15730 URL: https://issues.apache.org/jira/browse/SPARK-15730

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-26 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301988#comment-15301988 ] Yi Zhou commented on SPARK-15034: - Hi [~yhuai] I can integrate Spark SQL with hive metast

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301468#comment-15301468 ] Yi Zhou commented on SPARK-15345: - I'm confused that it worked well with Apache Spark 1.6

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301456#comment-15301456 ] Yi Zhou commented on SPARK-15345: - I issued 'show databases;' , ' use XXX' and 'show tabl

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301427#comment-15301427 ] Yi Zhou commented on SPARK-15345: - Hi [~jeffzhang] In my test environment, i installed CD

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301342#comment-15301342 ] Yi Zhou commented on SPARK-13955: - Thanks a lot [~jerryshao]! > Spark in yarn mode fails

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300088#comment-15300088 ] Yi Zhou commented on SPARK-15345: - Hi [~jeffzhang] I rebuild Spark 2.0 with https://githu

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15300014#comment-15300014 ] Yi Zhou commented on SPARK-13955: - Hi [~jerryshao] I build a Spark 2.0 snapshot package w

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299587#comment-15299587 ] Yi Zhou commented on SPARK-15345: - OK Thanks ! > SparkSession's conf doesn't take effect

[jira] [Comment Edited] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299568#comment-15299568 ] Yi Zhou edited comment on SPARK-15345 at 5/25/16 6:56 AM: -- 1) Sp

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299568#comment-15299568 ] Yi Zhou commented on SPARK-15345: - 1) Spark SQL can't find existing hive metastore databa

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296105#comment-15296105 ] Yi Zhou commented on SPARK-15396: - Thanks a lot for your respone ! > [Spark] [SQL] [DOC]

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296088#comment-15296088 ] Yi Zhou commented on SPARK-15396: - BTW, I only replaced 1.6 with 2.0 package. > [Spark]

[jira] [Comment Edited] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296086#comment-15296086 ] Yi Zhou edited comment on SPARK-15396 at 5/23/16 8:22 AM: -- hive-

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296086#comment-15296086 ] Yi Zhou commented on SPARK-15396: - hive-site.xml put in right 'conf' folder in Spark home

[jira] [Comment Edited] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296010#comment-15296010 ] Yi Zhou edited comment on SPARK-15396 at 5/23/16 7:02 AM: -- sorry

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296010#comment-15296010 ] Yi Zhou commented on SPARK-15396: - sorry i can't access the http://www.hiregion.com/2010

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295920#comment-15295920 ] Yi Zhou commented on SPARK-15396: - these parameters worked well in Spark 1.6.1 but no sur

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295874#comment-15295874 ] Yi Zhou commented on SPARK-15396: - Add, 1) Spark SQL can't find existing hive databases i

[jira] [Comment Edited] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295836#comment-15295836 ] Yi Zhou edited comment on SPARK-15396 at 5/23/16 2:06 AM: -- Hi ,

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295836#comment-15295836 ] Yi Zhou commented on SPARK-15396: - Hi , even if setting 'spark.sql.warehouse.dir' in spa

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-19 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292672#comment-15292672 ] Yi Zhou commented on SPARK-15396: - It seem it is not only a doc issue , it may be functio

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-19 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292655#comment-15292655 ] Yi Zhou commented on SPARK-15396: - Hi [~rxin] I saw a bug fix https://issues.apache.org/j

[jira] [Created] (SPARK-15397) [Spark][SQL] 'locate' UDF got different result with boundary value case compared to Hive engine

2016-05-18 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-15397: --- Summary: [Spark][SQL] 'locate' UDF got different result with boundary value case compared to Hive engine Key: SPARK-15397 URL: https://issues.apache.org/jira/browse/SPARK-15397

[jira] [Commented] (SPARK-15396) [Spark] [SQL] It can't connect hive metastore database

2016-05-18 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290306#comment-15290306 ] Yi Zhou commented on SPARK-15396: - BTW, where can i found this parameter in doc ? i check

[jira] [Commented] (SPARK-15396) [Spark] [SQL] It can't connect hive metastore database

2016-05-18 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290295#comment-15290295 ] Yi Zhou commented on SPARK-15396: - Thanks ! I will try it. > [Spark] [SQL] It can't conn

[jira] [Updated] (SPARK-15396) [Spark] [SQL] It can't connect hive metastore database

2016-05-18 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-15396: Description: I am try to run Spark SQL using bin/spark-sql with Spark 2.0 master code(commit ba181c0c7a32b

[jira] [Created] (SPARK-15396) [Spark] [SQL] It can't connect hive metastore database

2016-05-18 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-15396: --- Summary: [Spark] [SQL] It can't connect hive metastore database Key: SPARK-15396 URL: https://issues.apache.org/jira/browse/SPARK-15396 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15219) [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization

2016-05-09 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276055#comment-15276055 ] Yi Zhou commented on SPARK-15219: - Posted the core physical plan > [Spark SQL] it don't

[jira] [Updated] (SPARK-15219) [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization

2016-05-09 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-15219: Description: We observed an interesting thing about broadcast Hash join( similar to Map Join in Hive) when

[jira] [Updated] (SPARK-15219) [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization

2016-05-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-15219: Description: We observed an interesting thing about broadcast Hash join( similar to Map Join in Hive) when

[jira] [Created] (SPARK-15219) [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization

2016-05-08 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-15219: --- Summary: [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization Key: SPARK-15219 URL: https://issues.apache.org/jira/browse/SPARK-15219

[jira] [Commented] (SPARK-8428) TimSort Comparison method violates its general contract with CLUSTER BY

2016-05-05 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273534#comment-15273534 ] Yi Zhou commented on SPARK-8428: We found the similar issue with Spark 1.6.1 in our larger

[jira] [Comment Edited] (SPARK-11293) Spillable collections leak shuffle memory

2016-03-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15206635#comment-15206635 ] Yi Zhou edited comment on SPARK-11293 at 3/22/16 3:56 PM: -- seem

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-03-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15206635#comment-15206635 ] Yi Zhou commented on SPARK-11293: - seem to hit the issue with Spark 1.6.1 not sure if thi

[jira] [Commented] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026283#comment-15026283 ] Yi Zhou commented on SPARK-11972: - Thanks [~adrian-wang] ! the case passed now after app

[jira] [Updated] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-11972: Priority: Critical (was: Major) > [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after

[jira] [Updated] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-11972: Description: Reproduce Steps: /usr/lib/spark/bin/spark-sql -v --driver-memory 4g --executor-memory 7g --ex

[jira] [Updated] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-11972: Description: Reproduce Steps: /usr/lib/spark/bin/spark-sql -v --driver-memory 4g --executor-memory 7g --ex

[jira] [Created] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-11972: --- Summary: [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session Key: SPARK-11972 URL: https://issues.apache.org/jira/browse/SPARK-11972

[jira] [Created] (SPARK-10866) [Spark SQL] [UDF] the floor function got wrong return value type

2015-09-28 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10866: --- Summary: [Spark SQL] [UDF] the floor function got wrong return value type Key: SPARK-10866 URL: https://issues.apache.org/jira/browse/SPARK-10866 Project: Spark Issue

[jira] [Updated] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-09-28 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10865: Description: As per ceil/ceiling definition,it should get BIGINT return value -ceil(DOUBLE a), ceiling(DOUB

[jira] [Created] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-09-28 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10865: --- Summary: [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type Key: SPARK-10865 URL: https://issues.apache.org/jira/browse/SPARK-10865 Project: Spark

[jira] [Commented] (SPARK-10474) TungstenAggregation cannot acquire memory for pointer array after switching to sort-based

2015-09-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906291#comment-14906291 ] Yi Zhou commented on SPARK-10474: - Hi [~andrewor14] [~yhuai]. It's OK for me and get no e

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903740#comment-14903740 ] Yi Zhou commented on SPARK-10733: - yes. i still got error after applying the commit. >

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902563#comment-14902563 ] Yi Zhou commented on SPARK-10733: - Apply https://github.com/andrewor14/spark/commit/2a9c

[jira] [Commented] (SPARK-10474) TungstenAggregation cannot acquire memory for pointer array after switching to sort-based

2015-09-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902230#comment-14902230 ] Yi Zhou commented on SPARK-10474: - see the my comments in https://issues.apache.org/jira/

[jira] [Comment Edited] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901959#comment-14901959 ] Yi Zhou edited comment on SPARK-10733 at 9/22/15 5:18 AM: -- 1) I

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901959#comment-14901959 ] Yi Zhou commented on SPARK-10733: - 1) I have never set `spark.task.cpus` and only use by

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901951#comment-14901951 ] Yi Zhou commented on SPARK-10733: - Key SQL Query: INSERT INTO TABLE test_table SELECT s

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900562#comment-14900562 ] Yi Zhou commented on SPARK-10474: - Hi [~andrewor14] I found a new error like below after

[jira] [Comment Edited] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746929#comment-14746929 ] Yi Zhou edited comment on SPARK-10474 at 9/16/15 5:55 AM: -- BTW,

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746929#comment-14746929 ] Yi Zhou commented on SPARK-10474: - BTW, the "spark.shuffle.safetyFraction" is not public

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746881#comment-14746881 ] Yi Zhou commented on SPARK-10474: - Thanks [~chenghao]. It's better not to throw such exce

[jira] [Commented] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734721#comment-14734721 ] Yi Zhou commented on SPARK-5791: [~yhuai], Yes. Thank you ! > [Spark SQL] show poor perfo

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G i

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G i

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two table do cross join

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Summary: [Spark SQL] Come across lost task(timeout) or GC OOM error when two table do cross join (was: [S

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Summary: [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join (was: [

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Issue Type: Bug (was: Improvement) > [Spark SQL] Come across lost task(timeout) or GC OOM error when cros

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Issue Type: Improvement (was: Bug) > [Spark SQL] Come across lost task(timeout) or GC OOM error when cros

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G i

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G i

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G i

[jira] [Created] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10484: --- Summary: [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen Key: SPARK-10484 URL: https://issues.apache.org/jira/browse/SPARK-10484 Project: Spa

[jira] [Updated] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-07 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10474: Description: In aggregation case, a Lost task happened with below error. java.io.IOException: Could not

[jira] [Updated] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-07 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10474: Description: In aggregation case, a Lost task happened with below error. java.io.IOException: Could not

[jira] [Updated] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-07 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10474: Description: In aggregation case, a Lost task happened with below error. I am not sure if the root cause

[jira] [Created] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-07 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10474: --- Summary: Aggregation failed with unable to acquire memory Key: SPARK-10474 URL: https://issues.apache.org/jira/browse/SPARK-10474 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10310) [Spark SQL] All result records will be popluated into ONE line during the script transform due to missing the correct line/filed delimiter

2015-09-06 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733195#comment-14733195 ] Yi Zhou commented on SPARK-10310: - Hi [~marmbrus] Could you please help to review and eva

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated into ONE line during the script transform due to missing the correct line/filed delimiter

2015-09-06 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Summary: [Spark SQL] All result records will be popluated into ONE line during the script transform due to

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated into ONE line during the script transform due to missing the correct line/filed delimeter

2015-08-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Summary: [Spark SQL] All result records will be popluated into ONE line during the script transform due to

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed delimeter

2015-08-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Description: There is real case using python stream script in Spark SQL query. We found that all result re

[jira] [Commented] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed delimeter

2015-08-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716264#comment-14716264 ] Yi Zhou commented on SPARK-10310: - This issue found in real-world case. Hopefully it can

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed delimeter

2015-08-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Description: There is real case using python stream script in Spark SQL query. We found that all result re

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed delimeter

2015-08-26 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Summary: [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed separator

2015-08-26 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Description: There is real case using python stream script in Spark SQL query. We found that all result re

[jira] [Created] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed separator

2015-08-26 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10310: --- Summary: [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed separator Key: SPARK-10310 URL: https://issues.apache.org/jira/browse/SPARK-10310

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

2015-08-26 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715855#comment-14715855 ] Yi Zhou commented on SPARK-9228: OK. Thanks again [~davies] > Combine unsafe and codegen

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

2015-08-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712360#comment-14712360 ] Yi Zhou commented on SPARK-9228: Hi [~davies] I also saw the 'spark.unsafe.offHeap' parame

[jira] [Commented] (SPARK-10215) Div of Decimal returns null

2015-08-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711327#comment-14711327 ] Yi Zhou commented on SPARK-10215: - This issue cause cases which is relative to 'decimal'

[jira] [Commented] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706345#comment-14706345 ] Yi Zhou commented on SPARK-10130: - Hopefully it can be fixed in Spark 1.5.0 actually it

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

2015-08-20 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706315#comment-14706315 ] Yi Zhou commented on SPARK-9228: Thanks [~davies] ! > Combine unsafe and codegen into a s

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

2015-08-20 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706152#comment-14706152 ] Yi Zhou commented on SPARK-9228: Hi [~davies] After introducing the 'spark.sql.tungsten.en

[jira] [Closed] (SPARK-7268) [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN

2015-07-28 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou closed SPARK-7268. -- Resolution: Done > [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN > -

[jira] [Commented] (SPARK-7268) [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN

2015-07-28 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644462#comment-14644462 ] Yi Zhou commented on SPARK-7268: In latest spark 1.5 master code , the issue is not existe

[jira] [Closed] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

2015-07-28 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou closed SPARK-5791. -- Resolution: Done > [Spark SQL] show poor performance when multiple table do join operation > ---

[jira] [Commented] (SPARK-9374) [Spark SQL] Throw out erorr of "AnalysisException: nondeterministic expressions are only allowed in Project or Filter" during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643724#comment-14643724 ] Yi Zhou commented on SPARK-9374: We can't found this issue in spark 1.4. it seems it was

[jira] [Updated] (SPARK-9374) [Spark SQL] Throw out erorr of "AnalysisException: nondeterministic expressions are only allowed in Project or Filter" during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-9374: --- Summary: [Spark SQL] Throw out erorr of "AnalysisException: nondeterministic expressions are only allowed in P

[jira] [Updated] (SPARK-9374) [Spark SQL] UDFUnixTimeStamp throw out erorr of "AnalysisException: nondeterministic expressions are only allowed in Project or Filter" during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-9374: --- Summary: [Spark SQL] UDFUnixTimeStamp throw out erorr of "AnalysisException: nondeterministic expressions are

[jira] [Updated] (SPARK-9374) [Spark SQL] Throw out erorr of "AnalysisException: nondeterministic expressions are only allowed in Project or Filter" during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-9374: --- Description: #Spark SQL Query INSERT INTO TABLE TEST_QUERY_0_result SELECT w_state, i_

[jira] [Created] (SPARK-9374) [Spark SQL] Throw out erorr of "AnalysisException: nondeterministic expressions are only allowed in Project or Filter" during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-9374: -- Summary: [Spark SQL] Throw out erorr of "AnalysisException: nondeterministic expressions are only allowed in Project or Filter" during the spark sql parse phase Key: SPARK-9374 URL: https://

  1   2   >