[jira] [Commented] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-18 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425995#comment-15425995 ] Yi Zhou commented on SPARK-17122: - Thanks [~dongjoon] for your clarity.. > Failed to drop database when

[jira] [Commented] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-17 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425926#comment-15425926 ] Yi Zhou commented on SPARK-17122: - Thanks [~dongjoon] for your quick response. But the problem is this

[jira] [Created] (SPARK-17122) Failed to drop database when use the database in Spark 2.0

2016-08-17 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-17122: --- Summary: Failed to drop database when use the database in Spark 2.0 Key: SPARK-17122 URL: https://issues.apache.org/jira/browse/SPARK-17122 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16515) [SPARK][SQL] transformation script got failure for python script

2016-07-12 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15374234#comment-15374234 ] Yi Zhou commented on SPARK-16515: - Thank you for your quick attention for this issue. > [SPARK][SQL]

[jira] [Created] (SPARK-16515) [SPARK][SQL] transformation script got failure for python script

2016-07-12 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-16515: --- Summary: [SPARK][SQL] transformation script got failure for python script Key: SPARK-16515 URL: https://issues.apache.org/jira/browse/SPARK-16515 Project: Spark

[jira] [Created] (SPARK-16415) [Spark][SQL] - Failed to create table due to catalog string error

2016-07-07 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-16415: --- Summary: [Spark][SQL] - Failed to create table due to catalog string error Key: SPARK-16415 URL: https://issues.apache.org/jira/browse/SPARK-16415 Project: Spark

[jira] [Commented] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-07-05 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15363691#comment-15363691 ] Yi Zhou commented on SPARK-15730: - Thanks a lot [~chenghao] and [~yhuai] ! > [Spark SQL] the value of

[jira] [Commented] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-06-14 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329408#comment-15329408 ] Yi Zhou commented on SPARK-15730: - Hi [~chenghao] This PR is tested and it's OK for my case. > [Spark

[jira] [Created] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

2016-06-02 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-15730: --- Summary: [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session Key: SPARK-15730 URL: https://issues.apache.org/jira/browse/SPARK-15730

[jira] [Commented] (SPARK-15034) Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir

2016-05-26 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301988#comment-15301988 ] Yi Zhou commented on SPARK-15034: - Hi [~yhuai] I can integrate Spark SQL with hive metastore well in

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301468#comment-15301468 ] Yi Zhou commented on SPARK-15345: - I'm confused that it worked well with Apache Spark 1.6 and can't work

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301456#comment-15301456 ] Yi Zhou commented on SPARK-15345: - I issued 'show databases;' , ' use XXX' and 'show tables;' and found

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301427#comment-15301427 ] Yi Zhou commented on SPARK-15345: - Hi [~jeffzhang] In my test environment, i installed CDH 5.7.0 Hive(

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301342#comment-15301342 ] Yi Zhou commented on SPARK-13955: - Thanks a lot [~jerryshao]! > Spark in yarn mode fails >

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300088#comment-15300088 ] Yi Zhou commented on SPARK-15345: - Hi [~jeffzhang] I rebuild Spark 2.0 with

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15300014#comment-15300014 ] Yi Zhou commented on SPARK-13955: - Hi [~jerryshao] I build a Spark 2.0 snapshot package which included a

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299587#comment-15299587 ] Yi Zhou commented on SPARK-15345: - OK Thanks ! > SparkSession's conf doesn't take effect when there's

[jira] [Comment Edited] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299568#comment-15299568 ] Yi Zhou edited comment on SPARK-15345 at 5/25/16 6:56 AM: -- 1) Spark SQL can't

[jira] [Commented] (SPARK-15345) SparkSession's conf doesn't take effect when there's already an existing SparkContext

2016-05-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299568#comment-15299568 ] Yi Zhou commented on SPARK-15345: - 1) Spark SQL can't find existing hive metastore database in spark-sql

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296105#comment-15296105 ] Yi Zhou commented on SPARK-15396: - Thanks a lot for your respone ! > [Spark] [SQL] [DOC] It can't

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296088#comment-15296088 ] Yi Zhou commented on SPARK-15396: - BTW, I only replaced 1.6 with 2.0 package. > [Spark] [SQL] [DOC] It

[jira] [Comment Edited] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296086#comment-15296086 ] Yi Zhou edited comment on SPARK-15396 at 5/23/16 8:22 AM: -- hive-site.xml put in

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296086#comment-15296086 ] Yi Zhou commented on SPARK-15396: - hive-site.xml put in right 'conf' folder in Spark home. my concerning

[jira] [Comment Edited] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296010#comment-15296010 ] Yi Zhou edited comment on SPARK-15396 at 5/23/16 7:02 AM: -- sorry i can't access

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-23 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296010#comment-15296010 ] Yi Zhou commented on SPARK-15396: - sorry i can't access the

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295920#comment-15295920 ] Yi Zhou commented on SPARK-15396: - these parameters worked well in Spark 1.6.1 but no sure why doesn't

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295874#comment-15295874 ] Yi Zhou commented on SPARK-15396: - Add, 1) Spark SQL can't find existing hive databases in spark-sql

[jira] [Comment Edited] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295836#comment-15295836 ] Yi Zhou edited comment on SPARK-15396 at 5/23/16 2:06 AM: -- Hi , even if setting

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295836#comment-15295836 ] Yi Zhou commented on SPARK-15396: - Hi , even if setting 'spark.sql.warehouse.dir' in spark-defaults.conf

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-19 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292672#comment-15292672 ] Yi Zhou commented on SPARK-15396: - It seem it is not only a doc issue , it may be functional issue. >

[jira] [Commented] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-19 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292655#comment-15292655 ] Yi Zhou commented on SPARK-15396: - Hi [~rxin] I saw a bug fix

[jira] [Created] (SPARK-15397) [Spark][SQL] 'locate' UDF got different result with boundary value case compared to Hive engine

2016-05-18 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-15397: --- Summary: [Spark][SQL] 'locate' UDF got different result with boundary value case compared to Hive engine Key: SPARK-15397 URL: https://issues.apache.org/jira/browse/SPARK-15397

[jira] [Commented] (SPARK-15396) [Spark] [SQL] It can't connect hive metastore database

2016-05-18 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290306#comment-15290306 ] Yi Zhou commented on SPARK-15396: - BTW, where can i found this parameter in doc ? i checked

[jira] [Commented] (SPARK-15396) [Spark] [SQL] It can't connect hive metastore database

2016-05-18 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290295#comment-15290295 ] Yi Zhou commented on SPARK-15396: - Thanks ! I will try it. > [Spark] [SQL] It can't connect hive

[jira] [Updated] (SPARK-15396) [Spark] [SQL] It can't connect hive metastore database

2016-05-18 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-15396: Description: I am try to run Spark SQL using bin/spark-sql with Spark 2.0 master code(commit

[jira] [Created] (SPARK-15396) [Spark] [SQL] It can't connect hive metastore database

2016-05-18 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-15396: --- Summary: [Spark] [SQL] It can't connect hive metastore database Key: SPARK-15396 URL: https://issues.apache.org/jira/browse/SPARK-15396 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15219) [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization

2016-05-09 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276055#comment-15276055 ] Yi Zhou commented on SPARK-15219: - Posted the core physical plan > [Spark SQL] it don't support to

[jira] [Updated] (SPARK-15219) [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization

2016-05-09 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-15219: Description: We observed an interesting thing about broadcast Hash join( similar to Map Join in Hive)

[jira] [Updated] (SPARK-15219) [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization

2016-05-09 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-15219: Description: We observed an interesting thing about broadcast Hash join( similar to Map Join in Hive)

[jira] [Created] (SPARK-15219) [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization

2016-05-09 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-15219: --- Summary: [Spark SQL] it don't support to detect runtime temporary table for enabling broadcast hash join optimization Key: SPARK-15219 URL:

[jira] [Commented] (SPARK-8428) TimSort Comparison method violates its general contract with CLUSTER BY

2016-05-05 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15273534#comment-15273534 ] Yi Zhou commented on SPARK-8428: We found the similar issue with Spark 1.6.1 in our larger data size

[jira] [Comment Edited] (SPARK-11293) Spillable collections leak shuffle memory

2016-03-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206635#comment-15206635 ] Yi Zhou edited comment on SPARK-11293 at 3/22/16 3:56 PM: -- seem to hit the issue

[jira] [Commented] (SPARK-11293) Spillable collections leak shuffle memory

2016-03-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206635#comment-15206635 ] Yi Zhou commented on SPARK-11293: - seem to hit the issue with Spark 1.6.1 not sure if this is relative to

[jira] [Created] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-11972: --- Summary: [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session Key: SPARK-11972 URL: https://issues.apache.org/jira/browse/SPARK-11972

[jira] [Updated] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-11972: Description: Reproduce Steps: /usr/lib/spark/bin/spark-sql -v --driver-memory 4g --executor-memory 7g

[jira] [Updated] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-11972: Description: Reproduce Steps: /usr/lib/spark/bin/spark-sql -v --driver-memory 4g --executor-memory 7g

[jira] [Updated] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-11972: Priority: Critical (was: Major) > [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after

[jira] [Commented] (SPARK-11972) [Spark SQL] the value of 'hiveconf' parameter in CLI can't be got after enter spark-sql session

2015-11-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026283#comment-15026283 ] Yi Zhou commented on SPARK-11972: - Thanks [~adrian-wang] ! the case passed now after applying the patch.

[jira] [Created] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-09-28 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10865: --- Summary: [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type Key: SPARK-10865 URL: https://issues.apache.org/jira/browse/SPARK-10865 Project: Spark

[jira] [Updated] (SPARK-10865) [Spark SQL] [UDF] the ceil/ceiling function got wrong return value type

2015-09-28 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10865: Description: As per ceil/ceiling definition,it should get BIGINT return value -ceil(DOUBLE a),

[jira] [Created] (SPARK-10866) [Spark SQL] [UDF] the floor function got wrong return value type

2015-09-28 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10866: --- Summary: [Spark SQL] [UDF] the floor function got wrong return value type Key: SPARK-10866 URL: https://issues.apache.org/jira/browse/SPARK-10866 Project: Spark

[jira] [Commented] (SPARK-10474) TungstenAggregation cannot acquire memory for pointer array after switching to sort-based

2015-09-24 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906291#comment-14906291 ] Yi Zhou commented on SPARK-10474: - Hi [~andrewor14] [~yhuai]. It's OK for me and get no errors. Thanks !

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903740#comment-14903740 ] Yi Zhou commented on SPARK-10733: - yes. i still got error after applying the commit. >

[jira] [Commented] (SPARK-10474) TungstenAggregation cannot acquire memory for pointer array after switching to sort-based

2015-09-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902230#comment-14902230 ] Yi Zhou commented on SPARK-10474: - see the my comments in

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-22 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902563#comment-14902563 ] Yi Zhou commented on SPARK-10733: - Apply

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901959#comment-14901959 ] Yi Zhou commented on SPARK-10733: - 1) I have never set `spark.task.cpus` and only use by default. 2)

[jira] [Comment Edited] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901959#comment-14901959 ] Yi Zhou edited comment on SPARK-10733 at 9/22/15 5:18 AM: -- 1) I have never set

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900562#comment-14900562 ] Yi Zhou commented on SPARK-10474: - Hi [~andrewor14] I found a new error like below after apply this PR.

[jira] [Commented] (SPARK-10733) TungstenAggregation cannot acquire page after switching to sort-based

2015-09-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901951#comment-14901951 ] Yi Zhou commented on SPARK-10733: - Key SQL Query: INSERT INTO TABLE test_table SELECT ss.ss_customer_sk

[jira] [Comment Edited] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746929#comment-14746929 ] Yi Zhou edited comment on SPARK-10474 at 9/16/15 5:55 AM: -- BTW, the

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746929#comment-14746929 ] Yi Zhou commented on SPARK-10474: - BTW, the "spark.shuffle.safetyFraction" is not public .. >

[jira] [Commented] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-15 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14746881#comment-14746881 ] Yi Zhou commented on SPARK-10474: - Thanks [~chenghao]. It's better not to throw such exception. >

[jira] [Created] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10484: --- Summary: [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen Key: SPARK-10484 URL: https://issues.apache.org/jira/browse/SPARK-10484 Project:

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Issue Type: Improvement (was: Bug) > [Spark SQL] Come across lost task(timeout) or GC OOM error when

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Issue Type: Bug (was: Improvement) > [Spark SQL] Come across lost task(timeout) or GC OOM error when

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two table do cross join

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Summary: [Spark SQL] Come across lost task(timeout) or GC OOM error when two table do cross join (was:

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Summary: [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join (was:

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when cross join happen

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G

[jira] [Updated] (SPARK-10484) [Spark SQL] Come across lost task(timeout) or GC OOM error when two tables do cross join

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10484: Description: Found that it lost task or GC OOM when below cross join happen. The left big table is ~1.2G

[jira] [Commented] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

2015-09-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734721#comment-14734721 ] Yi Zhou commented on SPARK-5791: [~yhuai], Yes. Thank you ! > [Spark SQL] show poor performance when

[jira] [Updated] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-07 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10474: Description: In aggregation case, a Lost task happened with below error. I am not sure if the root cause

[jira] [Created] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-07 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10474: --- Summary: Aggregation failed with unable to acquire memory Key: SPARK-10474 URL: https://issues.apache.org/jira/browse/SPARK-10474 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-07 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10474: Description: In aggregation case, a Lost task happened with below error. java.io.IOException: Could not

[jira] [Updated] (SPARK-10474) Aggregation failed with unable to acquire memory

2015-09-07 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10474: Description: In aggregation case, a Lost task happened with below error. java.io.IOException: Could not

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated into ONE line during the script transform due to missing the correct line/filed delimiter

2015-09-06 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Summary: [Spark SQL] All result records will be popluated into ONE line during the script transform due to

[jira] [Commented] (SPARK-10310) [Spark SQL] All result records will be popluated into ONE line during the script transform due to missing the correct line/filed delimiter

2015-09-06 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14733195#comment-14733195 ] Yi Zhou commented on SPARK-10310: - Hi [~marmbrus] Could you please help to review and evaluate this

[jira] [Commented] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed delimeter

2015-08-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716264#comment-14716264 ] Yi Zhou commented on SPARK-10310: - This issue found in real-world case. Hopefully it can

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed delimeter

2015-08-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Description: There is real case using python stream script in Spark SQL query. We found that all result

[jira] [Updated] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed delimeter

2015-08-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-10310: Description: There is real case using python stream script in Spark SQL query. We found that all result

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

2015-08-26 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715855#comment-14715855 ] Yi Zhou commented on SPARK-9228: OK. Thanks again [~davies] Combine unsafe and codegen

[jira] [Created] (SPARK-10310) [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed separator

2015-08-26 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-10310: --- Summary: [Spark SQL] All result records will be popluated in ONE line due to missing the correct line/filed separator Key: SPARK-10310 URL:

[jira] [Commented] (SPARK-10215) Div of Decimal returns null

2015-08-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14711327#comment-14711327 ] Yi Zhou commented on SPARK-10215: - This issue cause cases which is relative to 'decimal'

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

2015-08-25 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712360#comment-14712360 ] Yi Zhou commented on SPARK-9228: Hi [~davies] I also saw the 'spark.unsafe.offHeap'

[jira] [Commented] (SPARK-10130) type coercion for IF should have children resolved first

2015-08-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706345#comment-14706345 ] Yi Zhou commented on SPARK-10130: - Hopefully it can be fixed in Spark 1.5.0 actually it

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

2015-08-21 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706315#comment-14706315 ] Yi Zhou commented on SPARK-9228: Thanks [~davies] ! Combine unsafe and codegen into a

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

2015-08-20 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706152#comment-14706152 ] Yi Zhou commented on SPARK-9228: Hi [~davies] After introducing the

[jira] [Closed] (SPARK-5791) [Spark SQL] show poor performance when multiple table do join operation

2015-07-28 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou closed SPARK-5791. -- Resolution: Done [Spark SQL] show poor performance when multiple table do join operation

[jira] [Closed] (SPARK-7268) [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN

2015-07-28 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou closed SPARK-7268. -- Resolution: Done [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN

[jira] [Commented] (SPARK-7268) [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN

2015-07-28 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644462#comment-14644462 ] Yi Zhou commented on SPARK-7268: In latest spark 1.5 master code , the issue is not

[jira] [Commented] (SPARK-9374) [Spark SQL] Throw out erorr of AnalysisException: nondeterministic expressions are only allowed in Project or Filter during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14643724#comment-14643724 ] Yi Zhou commented on SPARK-9374: We can't found this issue in spark 1.4. it seems it was

[jira] [Updated] (SPARK-9374) [Spark SQL] Throw out erorr of AnalysisException: nondeterministic expressions are only allowed in Project or Filter during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-9374: --- Description: #Spark SQL Query INSERT INTO TABLE TEST_QUERY_0_result SELECT w_state,

[jira] [Created] (SPARK-9374) [Spark SQL] Throw out erorr of AnalysisException: nondeterministic expressions are only allowed in Project or Filter during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-9374: -- Summary: [Spark SQL] Throw out erorr of AnalysisException: nondeterministic expressions are only allowed in Project or Filter during the spark sql parse phase Key: SPARK-9374 URL:

[jira] [Updated] (SPARK-9374) [Spark SQL] UDFUnixTimeStamp throw out erorr of AnalysisException: nondeterministic expressions are only allowed in Project or Filter during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-9374: --- Summary: [Spark SQL] UDFUnixTimeStamp throw out erorr of AnalysisException: nondeterministic expressions are

[jira] [Updated] (SPARK-9374) [Spark SQL] Throw out erorr of AnalysisException: nondeterministic expressions are only allowed in Project or Filter during the spark sql parse phase

2015-07-27 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Zhou updated SPARK-9374: --- Summary: [Spark SQL] Throw out erorr of AnalysisException: nondeterministic expressions are only allowed in

[jira] [Commented] (SPARK-7119) ScriptTransform doesn't consider the output data type

2015-07-08 Thread Yi Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619672#comment-14619672 ] Yi Zhou commented on SPARK-7119: The issue blocked Spark SQL query relative to

[jira] [Created] (SPARK-7623) Spark prints SelectChannelConnector@0.0.0.0:4040: java.net.BindException: Address already in use java.net.BindException: Address already in use when run 2 spark in par

2015-05-13 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-7623: -- Summary: Spark prints SelectChannelConnector@0.0.0.0:4040: java.net.BindException: Address already in use java.net.BindException: Address already in use when run 2 spark in parallel Key: SPARK-7623

[jira] [Created] (SPARK-7268) [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN

2015-04-29 Thread Yi Zhou (JIRA)
Yi Zhou created SPARK-7268: -- Summary: [Spark SQL] Throw 'Shutdown hooks cannot be modified during shutdown' on YARN Key: SPARK-7268 URL: https://issues.apache.org/jira/browse/SPARK-7268 Project: Spark

  1   2   >