[jira] [Commented] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's
[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627120#comment-15627120 ] Tejas Patil commented on SPARK-15867: - Yes. I am interested in this support. > TABLESAMPLE BUCKET semantics don't match Hive's > --- > > Key: SPARK-15867 > URL: https://issues.apache.org/jira/browse/SPARK-15867 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.0, 2.0.0 >Reporter: Andrew Or > > {code} > SELECT * FROM boxes TABLESAMPLE (BUCKET 3 OUT OF 16) > {code} > In Hive, this would select the 3rd bucket out of every 16 buckets there are > in the table. E.g. if the table was clustered by 32 buckets then this would > sample the 3rd and the 19th bucket. (See > https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Sampling) > In Spark, however, we simply sample 3/16 of the number of input rows. > Either we don't support it in Spark or do it in a way that's consistent with > Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's
[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15626947#comment-15626947 ] Reynold Xin commented on SPARK-15867: - cc [~tejasp] My guess is that your Facebook gang would be interested in this ... might be good to post a short design description before moving fully ahead with a patch on this one. > TABLESAMPLE BUCKET semantics don't match Hive's > --- > > Key: SPARK-15867 > URL: https://issues.apache.org/jira/browse/SPARK-15867 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.0, 2.0.0 >Reporter: Andrew Or > > {code} > SELECT * FROM boxes TABLESAMPLE (BUCKET 3 OUT OF 16) > {code} > In Hive, this would select the 3rd bucket out of every 16 buckets there are > in the table. E.g. if the table was clustered by 32 buckets then this would > sample the 3rd and the 19th bucket. (See > https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Sampling) > In Spark, however, we simply sample 3/16 of the number of input rows. > Either we don't support it in Spark or do it in a way that's consistent with > Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's
[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324960#comment-15324960 ] Andrew Or commented on SPARK-15867: --- I think we should fix it, though looks like it's been an issue for a while. I don't think it was ever documented so it's OK to change the behavior. > TABLESAMPLE BUCKET semantics don't match Hive's > --- > > Key: SPARK-15867 > URL: https://issues.apache.org/jira/browse/SPARK-15867 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.6.0, 2.0.0 >Reporter: Andrew Or > > {code} > SELECT * FROM boxes TABLESAMPLE (BUCKET 3 OUT OF 16) > {code} > In Hive, this would select the 3rd bucket out of every 16 buckets there are > in the table. E.g. if the table was clustered by 32 buckets then this would > sample the 3rd and the 19th bucket. (See > https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Sampling) > In Spark, however, we simply sample 3/16 of the number of input rows. > Either we don't support it in Spark or do it in a way that's consistent with > Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's
[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1532#comment-1532 ] Herman van Hovell commented on SPARK-15867: --- This would change the behavior in comparison with 1.6, see: https://github.com/apache/spark/blob/branch-1.6/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala#L1282-L1286 I don't think changing is much of a big deal though. > TABLESAMPLE BUCKET semantics don't match Hive's > --- > > Key: SPARK-15867 > URL: https://issues.apache.org/jira/browse/SPARK-15867 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.0.0 >Reporter: Andrew Or > > {code} > SELECT * FROM boxes TABLESAMPLE (BUCKET 3 OUT OF 16) > {code} > In Hive, this would select the 3rd bucket out of every 16 buckets there are > in the table. E.g. if the table was clustered by 32 buckets then this would > sample the 3rd and the 19th bucket. (See > https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Sampling) > In Spark, however, we simply sample 3/16 of the number of input rows. > Either we don't support it in Spark or do it in a way that's consistent with > Hive. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org