[jira] [Commented] (SPARK-27837) Running rand() in SQL with seed of column results in error (rand(col1))
[ https://issues.apache.org/jira/browse/SPARK-27837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849871#comment-16849871 ] Liang-Chi Hsieh commented on SPARK-27837: - Btw, I think this is not a bug but like an improvement or new feature. > Running rand() in SQL with seed of column results in error (rand(col1)) > --- > > Key: SPARK-27837 > URL: https://issues.apache.org/jira/browse/SPARK-27837 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Jason Ferrell >Priority: Major > > Running this sql: > with a as > ( > select 123 val1 > union all > select 123 val1 > union all > select 123 val1 > ) > select val1,rand(123),rand(val1) > from a > Results in error: org.apache.spark.sql.AnalysisException: Input argument to > rand must be an integer, long or null literal.; > It doesn't appear to recognize the value of the column as an int. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27837) Running rand() in SQL with seed of column results in error (rand(col1))
[ https://issues.apache.org/jira/browse/SPARK-27837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849836#comment-16849836 ] Liang-Chi Hsieh commented on SPARK-27837: - Ah, I see. MySQL disallows nonconstant argument in previous version. What you suggest is like nonconstant argument to rand function in MySQL. > Running rand() in SQL with seed of column results in error (rand(col1)) > --- > > Key: SPARK-27837 > URL: https://issues.apache.org/jira/browse/SPARK-27837 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Jason Ferrell >Priority: Major > > Running this sql: > with a as > ( > select 123 val1 > union all > select 123 val1 > union all > select 123 val1 > ) > select val1,rand(123),rand(val1) > from a > Results in error: org.apache.spark.sql.AnalysisException: Input argument to > rand must be an integer, long or null literal.; > It doesn't appear to recognize the value of the column as an int. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27837) Running rand() in SQL with seed of column results in error (rand(col1))
[ https://issues.apache.org/jira/browse/SPARK-27837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849794#comment-16849794 ] Jason Ferrell commented on SPARK-27837: --- Here is an example from Microsoft SQL Server. I'm pretty sure Oracle works the same way with dbms_random. with a as ( select 123 val1 union all select 123 val1 union all select 456 val1 union all select 789 val1 ) select val1,rand(val1) seededrand,rand() unseededrand from a |val1|seededrand|unseededrand| |123|0.715865|0.989062| |123|0.715865|0.989062| |456|0.72207|0.989062| |789|0.728275|0.989062| > Running rand() in SQL with seed of column results in error (rand(col1)) > --- > > Key: SPARK-27837 > URL: https://issues.apache.org/jira/browse/SPARK-27837 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Jason Ferrell >Priority: Major > > Running this sql: > with a as > ( > select 123 val1 > union all > select 123 val1 > union all > select 123 val1 > ) > select val1,rand(123),rand(val1) > from a > Results in error: org.apache.spark.sql.AnalysisException: Input argument to > rand must be an integer, long or null literal.; > It doesn't appear to recognize the value of the column as an int. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27837) Running rand() in SQL with seed of column results in error (rand(col1))
[ https://issues.apache.org/jira/browse/SPARK-27837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848752#comment-16848752 ] Liang-Chi Hsieh commented on SPARK-27837: - I don't see it makes sense. I checked few DBs, and didn't see rand function work like your way. > Running rand() in SQL with seed of column results in error (rand(col1)) > --- > > Key: SPARK-27837 > URL: https://issues.apache.org/jira/browse/SPARK-27837 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Jason Ferrell >Priority: Major > > Running this sql: > with a as > ( > select 123 val1 > union all > select 123 val1 > union all > select 123 val1 > ) > select val1,rand(123),rand(val1) > from a > Results in error: org.apache.spark.sql.AnalysisException: Input argument to > rand must be an integer, long or null literal.; > It doesn't appear to recognize the value of the column as an int. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27837) Running rand() in SQL with seed of column results in error (rand(col1))
[ https://issues.apache.org/jira/browse/SPARK-27837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848200#comment-16848200 ] Jason Ferrell commented on SPARK-27837: --- Is there any way to get this into an enhancement request? It doesn't seem quite functional as part of the SQL code that I would pass a literal to every single row of an operation. The behavior I think would be most advantageous would be to pass an int corresponding to a column within the row so that I can get a random value for each row, seeded by a value on that row. > Running rand() in SQL with seed of column results in error (rand(col1)) > --- > > Key: SPARK-27837 > URL: https://issues.apache.org/jira/browse/SPARK-27837 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Jason Ferrell >Priority: Major > > Running this sql: > with a as > ( > select 123 val1 > union all > select 123 val1 > union all > select 123 val1 > ) > select val1,rand(123),rand(val1) > from a > Results in error: org.apache.spark.sql.AnalysisException: Input argument to > rand must be an integer, long or null literal.; > It doesn't appear to recognize the value of the column as an int. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27837) Running rand() in SQL with seed of column results in error (rand(col1))
[ https://issues.apache.org/jira/browse/SPARK-27837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848125#comment-16848125 ] Liang-Chi Hsieh commented on SPARK-27837: - Please see the analysis exception: Input argument to rand must be an integer, long or null literal. > Running rand() in SQL with seed of column results in error (rand(col1)) > --- > > Key: SPARK-27837 > URL: https://issues.apache.org/jira/browse/SPARK-27837 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Jason Ferrell >Priority: Major > > Running this sql: > with a as > ( > select 123 val1 > union all > select 123 val1 > union all > select 123 val1 > ) > select val1,rand(123),rand(val1) > from a > Results in error: org.apache.spark.sql.AnalysisException: Input argument to > rand must be an integer, long or null literal.; > It doesn't appear to recognize the value of the column as an int. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27837) Running rand() in SQL with seed of column results in error (rand(col1))
[ https://issues.apache.org/jira/browse/SPARK-27837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16848123#comment-16848123 ] Liang-Chi Hsieh commented on SPARK-27837: - The problem isn't that val1 isn't an int, but it isn't a literal. Seed to rand must be a literal. > Running rand() in SQL with seed of column results in error (rand(col1)) > --- > > Key: SPARK-27837 > URL: https://issues.apache.org/jira/browse/SPARK-27837 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: Jason Ferrell >Priority: Major > > Running this sql: > with a as > ( > select 123 val1 > union all > select 123 val1 > union all > select 123 val1 > ) > select val1,rand(123),rand(val1) > from a > Results in error: org.apache.spark.sql.AnalysisException: Input argument to > rand must be an integer, long or null literal.; > It doesn't appear to recognize the value of the column as an int. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
