[jira] [Resolved] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-25 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai resolved SPARK-30349. --- Resolution: Fixed > The result is wrong when joining tables with selecting the same columns >

[jira] [Commented] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-25 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003217#comment-17003217 ] cen yuhai commented on SPARK-30349: --- 3.0 preview rc2 is ok > The result is wrong when joining tables

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-25 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: {code:sql} // code placeholder with tmp as( select log_date, buvid, manga_id,

[jira] [Issue Comment Deleted] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Comment: was deleted (was: {code} select a.name, b.name from a join b on a.id = b.id {code} this sql

[jira] [Comment Edited] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003065#comment-17003065 ] cen yuhai edited comment on SPARK-30349 at 12/25/19 3:49 AM: - {code} select

[jira] [Commented] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003065#comment-17003065 ] cen yuhai commented on SPARK-30349: --- {code} select a.name, b.name from a join b on a.id = b.id {code}

[jira] [Commented] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003063#comment-17003063 ] cen yuhai commented on SPARK-30349: --- [~smilegator] > The result is wrong when joining tables with

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: {code:sql} // code placeholder with tmp as( select log_date, buvid, manga_id,

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Attachment: screenshot-2.png > The result is wrong when joining tables with selecting the same

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: {code:sql} // code placeholder with tmp as( select log_date, buvid, manga_id,

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Attachment: screenshot-1.png > The result is wrong when joining tables with selecting the same

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: {code:java} // code placeholder with tmp as( select log_date, buvid, manga_id,

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: {code:java} // code placeholder with tmp as( select log_date, buvid, manga_id,

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: {code:java} // code placeholder {code} with tmp as( select log_date, buvid,

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: {code:java} // code placeholder with tmp as( select log_date, buvid, manga_id,

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: ```sql       ```sql with tmp as( select log_date, buvid, manga_id,

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: ```sql with tmp as( select log_date, buvid, manga_id, sum(readtime) readtime from

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Description: ```sql with tmp as( select log_date, buvid, manga_id, sum(readtime) readtime from

[jira] [Updated] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-30349: -- Environment: hadoop-2.8.4 spark 2.4.4 (was:       ```sql with tmp as( select log_date, buvid,

[jira] [Created] (SPARK-30349) The result is wrong when joining tables with selecting the same columns

2019-12-24 Thread cen yuhai (Jira)
cen yuhai created SPARK-30349: - Summary: The result is wrong when joining tables with selecting the same columns Key: SPARK-30349 URL: https://issues.apache.org/jira/browse/SPARK-30349 Project: Spark

[jira] [Commented] (SPARK-22748) Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup;

2017-12-10 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16285210#comment-16285210 ] cen yuhai commented on SPARK-22748: --- your sql is ok > Error in query: grouping_id() can only be used

[jira] [Updated] (SPARK-22748) Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup;

2017-12-09 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-22748: -- Description: sql {code} create table temp.test_grouping_replace(a int, b int); select grouping__id

[jira] [Created] (SPARK-22748) Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup;

2017-12-09 Thread cen yuhai (JIRA)
cen yuhai created SPARK-22748: - Summary: Error in query: grouping_id() can only be used with GroupingSets/Cube/Rollup; Key: SPARK-22748 URL: https://issues.apache.org/jira/browse/SPARK-22748 Project:

[jira] [Commented] (SPARK-22491) union all can't execute parallel with group by

2017-11-10 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247646#comment-16247646 ] cen yuhai commented on SPARK-22491: --- If I set spark.sql.adaptive.enabled=false, it is ok. > union all

[jira] [Updated] (SPARK-22491) union all can't execute parallel with group by

2017-11-10 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-22491: -- Description: test sql set spark.sql.adaptive.enabled=true; {code} create table temp.test_yuhai_group

[jira] [Commented] (SPARK-22491) union all can't execute parallel with group by

2017-11-10 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247202#comment-16247202 ] cen yuhai commented on SPARK-22491: --- [~smilegator] [~cloud_fan] > union all can't execute parallel

[jira] [Updated] (SPARK-22491) union all can't execute parallel with group by

2017-11-10 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-22491: -- Description: test sql {code} create table temp.test_yuhai_group as select city_name

[jira] [Updated] (SPARK-22491) union all can't execute parallel with group by

2017-11-10 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-22491: -- Attachment: screenshot-1.png > union all can't execute parallel with group by >

[jira] [Updated] (SPARK-22491) union all can't execute parallel with group by

2017-11-10 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-22491: -- Description: test sql {code} create table temp.test_yuhai_group as select city_name

[jira] [Created] (SPARK-22491) union all can't execute parallel with group by

2017-11-10 Thread cen yuhai (JIRA)
cen yuhai created SPARK-22491: - Summary: union all can't execute parallel with group by Key: SPARK-22491 URL: https://issues.apache.org/jira/browse/SPARK-22491 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-22491) union all can't execute parallel with group by

2017-11-10 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-22491: -- Description: test sql {code} create table temp.test_yuhai_group as select city_name

[jira] [Created] (SPARK-22234) Distinct window functions are not supported

2017-10-10 Thread cen yuhai (JIRA)
cen yuhai created SPARK-22234: - Summary: Distinct window functions are not supported Key: SPARK-22234 URL: https://issues.apache.org/jira/browse/SPARK-22234 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2017-09-08 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158189#comment-16158189 ] cen yuhai commented on SPARK-18492: --- spark 2.1.1 also have this problem > GeneratedIterator grows

[jira] [Closed] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai closed SPARK-21796. - Resolution: Not A Problem > pyspark count failed in python3.5.2 > --- >

[jira] [Commented] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136189#comment-16136189 ] cen yuhai commented on SPARK-21796: --- It is env problem, I will reinstall system, close this issue. >

[jira] [Commented] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135048#comment-16135048 ] cen yuhai commented on SPARK-21796: --- I can execute the code by spark-submit or zeppelin > pyspark

[jira] [Commented] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134988#comment-16134988 ] cen yuhai commented on SPARK-21796: --- !screenshot-1.png! This is my script, I just change the version,

[jira] [Updated] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-21796: -- Attachment: screenshot-1.png > pyspark count failed in python3.5.2 >

[jira] [Comment Edited] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134958#comment-16134958 ] cen yuhai edited comment on SPARK-21796 at 8/21/17 9:44 AM: [~srowen] hi

[jira] [Comment Edited] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134958#comment-16134958 ] cen yuhai edited comment on SPARK-21796 at 8/21/17 9:44 AM: [~srowen] hi

[jira] [Commented] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134958#comment-16134958 ] cen yuhai commented on SPARK-21796: --- [~srowen] hi owean, I already set in spark-env.sh export

[jira] [Commented] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134951#comment-16134951 ] cen yuhai commented on SPARK-21796: --- [~hyukjin.kwon] I upload the file. Some machines will failed,

[jira] [Updated] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-21796: -- Attachment: user > pyspark count failed in python3.5.2 > --- > >

[jira] [Comment Edited] (SPARK-13330) PYTHONHASHSEED is not propgated to python worker

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134917#comment-16134917 ] cen yuhai edited comment on SPARK-13330 at 8/21/17 9:00 AM: [~zjffdu]

[jira] [Commented] (SPARK-13330) PYTHONHASHSEED is not propgated to python worker

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134917#comment-16134917 ] cen yuhai commented on SPARK-13330: --- [~zjffdu] can you also look at this issue:

[jira] [Updated] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-21796: -- Environment: Spark 2.1.1 Python 3.5.2 Anaconda3 4.2.0 (was: spark 2.1.1 Python 3.5.2 anaconda3

[jira] [Updated] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-21796: -- Environment: Python 3.5.2 Anaconda3 4.2.0 (was: Spark 2.1.1 Python 3.5.2 Anaconda3 4.2.0) >

[jira] [Updated] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-21796: -- Description: steps: {code} pyspark user_data =

[jira] [Created] (SPARK-21796) pyspark count failed in python3.5.2

2017-08-21 Thread cen yuhai (JIRA)
cen yuhai created SPARK-21796: - Summary: pyspark count failed in python3.5.2 Key: SPARK-21796 URL: https://issues.apache.org/jira/browse/SPARK-21796 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16188) Spark sql create a lot of small files

2017-08-18 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131796#comment-16131796 ] cen yuhai commented on SPARK-16188: --- [~xianlongZhang] yes, you are right, I has implemented this

[jira] [Commented] (SPARK-16188) Spark sql create a lot of small files

2017-08-16 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129815#comment-16129815 ] cen yuhai commented on SPARK-16188: --- [~xianlongZhang] You can use distribute by rand() or some other

[jira] [Commented] (SPARK-19490) Hive partition columns are case-sensitive

2017-07-28 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105979#comment-16105979 ] cen yuhai commented on SPARK-19490: --- I forgot the `fixed PR`, maybe that pr will not by backport to 2.1

[jira] [Commented] (SPARK-19471) [SQL]A confusing NullPointerException when creating table

2017-07-22 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097221#comment-16097221 ] cen yuhai commented on SPARK-19471: --- I try this pr, it works well. > [SQL]A confusing

[jira] [Commented] (SPARK-20295) when spark.sql.adaptive.enabled is enabled, have conflict with Exchange Resue

2017-06-22 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059273#comment-16059273 ] cen yuhai commented on SPARK-20295: --- I hit this bug... > when spark.sql.adaptive.enabled is enabled,

[jira] [Commented] (SPARK-19490) Hive partition columns are case-sensitive

2017-06-16 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052633#comment-16052633 ] cen yuhai commented on SPARK-19490: --- I don't know, what is your spark version? > Hive partition

[jira] [Updated] (SPARK-21055) Support grouping__id

2017-06-11 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-21055: -- Description: Now, spark doesn't support grouping__id, spark provide another function grouping_id() to

[jira] [Updated] (SPARK-21055) Support grouping__id

2017-06-11 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-21055: -- Description: Now, spark doesn't support grouping__id, spark provide another function grouping_id() to

[jira] [Updated] (SPARK-21055) Support grouping__id

2017-06-11 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-21055: -- Description: Now, spark doesn't support grouping__id, spark provide another function grouping_id() to

[jira] [Created] (SPARK-21055) Support grouping__id

2017-06-11 Thread cen yuhai (JIRA)
cen yuhai created SPARK-21055: - Summary: Support grouping__id Key: SPARK-21055 URL: https://issues.apache.org/jira/browse/SPARK-21055 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-28 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027743#comment-16027743 ] cen yuhai commented on SPARK-20882: --- [~zsxwing] It is my problem... It is none of spark's busniess. >

[jira] [Closed] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-26 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai closed SPARK-20882. - Resolution: Not A Problem > Executor is waiting for ShuffleBlockFetcherIterator >

[jira] [Comment Edited] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-26 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16026040#comment-16026040 ] cen yuhai edited comment on SPARK-20882 at 5/26/17 9:13 AM: [~zsxwing] yes.

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-26 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16026040#comment-16026040 ] cen yuhai commented on SPARK-20882: --- yes. {code} 17/05/26 12:04:14 INFO TransportResponseHandler: Still

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-26 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Description: This bug is like https://issues.apache.org/jira/browse/SPARK-19300. but I have updated my

[jira] [Comment Edited] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025581#comment-16025581 ] cen yuhai edited comment on SPARK-20882 at 5/26/17 12:08 AM: - I think the

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025581#comment-16025581 ] cen yuhai commented on SPARK-20882: --- I think the problem is when the connection to nodemanger:7337 is

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025496#comment-16025496 ] cen yuhai commented on SPARK-20882: --- Yes, I am using shuffle service > Executor is waiting for

[jira] [Comment Edited] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16025496#comment-16025496 ] cen yuhai edited comment on SPARK-20882 at 5/25/17 10:38 PM: - [~zsxwing]

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Description: This bug is like https://issues.apache.org/jira/browse/SPARK-19300. but I have updated my

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Description: This bug is like https://issues.apache.org/jira/browse/SPARK-19300. but I have updated my

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Description: This bug is like https://issues.apache.org/jira/browse/SPARK-19300. but I have updated my

[jira] [Commented] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024503#comment-16024503 ] cen yuhai commented on SPARK-20882: --- [~zsxwing] > Executor is waiting for ShuffleBlockFetcherIterator

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Attachment: screenshot-3.png > Executor is waiting for ShuffleBlockFetcherIterator >

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Attachment: executor_log > Executor is waiting for ShuffleBlockFetcherIterator >

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Attachment: screenshot-2.png > Executor is waiting for ShuffleBlockFetcherIterator >

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Attachment: screenshot-1.png > Executor is waiting for ShuffleBlockFetcherIterator >

[jira] [Updated] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20882: -- Attachment: executor_jstack > Executor is waiting for ShuffleBlockFetcherIterator >

[jira] [Created] (SPARK-20882) Executor is waiting for ShuffleBlockFetcherIterator

2017-05-25 Thread cen yuhai (JIRA)
cen yuhai created SPARK-20882: - Summary: Executor is waiting for ShuffleBlockFetcherIterator Key: SPARK-20882 URL: https://issues.apache.org/jira/browse/SPARK-20882 Project: Spark Issue Type:

[jira] [Updated] (SPARK-20476) Exception between "create table as" and "get_json_object"

2017-04-26 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20476: -- Description: I encounter this problem when I want to create a table as select , get_json_object

[jira] [Updated] (SPARK-20476) Exception between "create table as" and "get_json_object"

2017-04-26 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20476: -- Description: I encounter this problem when I want to create a table as select , get_json_object

[jira] [Updated] (SPARK-20476) Exception between "create table as" and "get_json_object"

2017-04-26 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-20476: -- Description: {code} create table spark_json_object as select get_json_object(deliver_geojson,'$.')

[jira] [Created] (SPARK-20476) Exception between "create table as" and "get_json_object"

2017-04-26 Thread cen yuhai (JIRA)
cen yuhai created SPARK-20476: - Summary: Exception between "create table as" and "get_json_object" Key: SPARK-20476 URL: https://issues.apache.org/jira/browse/SPARK-20476 Project: Spark Issue

[jira] [Commented] (SPARK-16599) java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask(BlockInfoManager.scala:343)

2017-04-17 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971081#comment-15971081 ] cen yuhai commented on SPARK-16599: --- [~srowen] I also encounter this problem >

[jira] [Created] (SPARK-20033) spark sql can not use hive permanent function

2017-03-20 Thread cen yuhai (JIRA)
cen yuhai created SPARK-20033: - Summary: spark sql can not use hive permanent function Key: SPARK-20033 URL: https://issues.apache.org/jira/browse/SPARK-20033 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-19490) Hive partition columns are case-sensitive

2017-02-22 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Description: The real partitions columns are lower case (year, month, day) {code} Caused by:

[jira] [Updated] (SPARK-19490) Hive partition columns are case-sensitive

2017-02-22 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Description: The real partitions columns are lower case (year, month, day) {code} Caused by:

[jira] [Updated] (SPARK-19490) Hive partition columns are case-sensitive

2017-02-22 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Description: The real partitions columns are lower case (year, month, day) {code} Caused by:

[jira] [Updated] (SPARK-19490) Hive partition columns are case-sensitive

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Summary: Hive partition columns are case-sensitive (was: Hive partition column are case-sensitive) >

[jira] [Updated] (SPARK-19490) Hive partition column are case-sensitive

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Summary: Hive partition column are case-sensitive (was: Hive partition column are case sensitivity)

[jira] [Updated] (SPARK-19490) Hive partition column are case sensitivity

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Description: The real partitions columns are lower case (year, month, day) {code} Caused by:

[jira] [Updated] (SPARK-19490) Hive partition column are case sensitivity

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Summary: Hive partition column are case sensitivity (was: Expected only partition pruning predicates)

[jira] [Updated] (SPARK-19490) Expected only partition pruning predicates

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Description: YEAR, MONTH, DAY are partition columns, the real partitions columns are lower case

[jira] [Updated] (SPARK-19490) Expected only partition pruning predicates

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Description: YEAR, MONTH, DAY are partition columns, the real partitions columns are lower case

[jira] [Comment Edited] (SPARK-19490) Expected only partition pruning predicates

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855783#comment-15855783 ] cen yuhai edited comment on SPARK-19490 at 2/7/17 11:11 AM: cc [~lian cheng]

[jira] [Commented] (SPARK-19490) Expected only partition pruning predicates

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855783#comment-15855783 ] cen yuhai commented on SPARK-19490: --- cc [~lian cheng]AN > Expected only partition pruning predicates >

[jira] [Updated] (SPARK-19490) Expected only partition pruning predicates

2017-02-07 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cen yuhai updated SPARK-19490: -- Description: YEAR, MONTH, DAY are partition columns {code} Caused by: java.lang.RuntimeException:

[jira] [Created] (SPARK-19490) Expected only partition pruning predicates

2017-02-07 Thread cen yuhai (JIRA)
cen yuhai created SPARK-19490: - Summary: Expected only partition pruning predicates Key: SPARK-19490 URL: https://issues.apache.org/jira/browse/SPARK-19490 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-16441) Spark application hang when dynamic allocation is enabled

2017-02-05 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15853474#comment-15853474 ] cen yuhai commented on SPARK-16441: --- I don't know, I will use spark 2.1, this may not be a problem. >

[jira] [Comment Edited] (SPARK-19328) using spark thrift server gets memory leak problem in `ExecutorsListener`

2017-01-22 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833925#comment-15833925 ] cen yuhai edited comment on SPARK-19328 at 1/23/17 5:27 AM: I think this is

[jira] [Commented] (SPARK-19328) using spark thrift server gets memory leak problem in `ExecutorsListener`

2017-01-22 Thread cen yuhai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15833925#comment-15833925 ] cen yuhai commented on SPARK-19328: --- I think this is fixed by SPARK-17406 > using spark thrift server

  1   2   3   >