[jira] [Created] (SPARK-35835) Select filter query on table with struct complex type fails
Chetan Bhat created SPARK-35835: --- Summary: Select filter query on table with struct complex type fails Key: SPARK-35835 URL: https://issues.apache.org/jira/browse/SPARK-35835 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat [Steps]:- >From Spark beeline create a parquet or ORC table having complex type data. >Load data in the table and execute select filter query. 0: jdbc:hive2://vm2:22550/> create table Struct_com (CUST_ID string, YEAR int, MONTH int, AGE int, GENDER string, EDUCATED string, IS_MARRIED string, STRUCT_INT_DOUBLE_STRING_DATE struct,CARD_COUNT int,DEBIT_COUNT int, CREDIT_COUNT int, DEPOSIT double, HQ_DEPOSIT double) stored as parquet; +-+ | Result | +-+ +-+ No rows selected (0.161 seconds) 0: jdbc:hive2://vm2:22550/> LOAD DATA INPATH 'hdfs://hacluster/chetan/Struct.csv' OVERWRITE INTO TABLE Struct_com; +-+ | Result | +-+ +-+ No rows selected (1.09 seconds) 0: jdbc:hive2://vm2:22550/> SELECT struct_int_double_string_date.COUNTRY,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.CHECK_DATE,struct_in t_double_string_date.Country, SUM(struct_int_double_string_date.id) AS Sum FROM (select * from Struct_com) SUB_QRY WHERE struct_int_double_string_date.id > 5700 GRO UP BY struct_int_double_string_date.COUNTRY,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.Country ORDER BY struct_int_double_string_date.COUNTRY asc,struct_int_double_string_date.CHECK_DATE asc,struct_int_double_string_date.CHECK_DATE asc, struct_int_double_stri ng_date.Country asc; [Actual Issue] : - Select filter query on table with struct complex type fails 0: jdbc:hive2://vm2:22550/> SELECT struct_int_double_string_date.COUNTRY,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.CHECK_DATE,struct_in t_double_string_date.Country, SUM(struct_int_double_string_date.id) AS Sum FROM (select * from Struct_com) SUB_QRY WHERE struct_int_double_string_date.id > 5700 GRO UP BY struct_int_double_string_date.COUNTRY,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.Country ORDER BY struct_int_double_string_date.COUNTRY asc,struct_int_double_string_date.CHECK_DATE asc,struct_int_double_string_date.CHECK_DATE asc, struct_int_double_stri ng_date.Country asc; Error: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Exchange rangepartitioning(COUNTRY#139896 ASC NULLS FIRST, CHECK_DATE#139897 ASC NULLS FIRST, CHECK_DATE#139897 ASC NULLS FIRST, COUNTRY#139896 ASC NULLS FIRST, 200 ), ENSURE_REQUIREMENTS, [id=#17161] +- *(2) HashAggregate(keys=[_gen_alias_139928#139928, _gen_alias_139929#139929], functions=[sum(cast(_gen_alias_139931#139931 as bigint))], output=[COUNTRY#139896, CHECK_DATE#139897, CHECK_DATE#139898, Country#139899, Sum#139877L]) +- Exchange hashpartitioning(_gen_alias_139928#139928, _gen_alias_139929#139929, 200), ENSURE_REQUIREMENTS, [id=#17157] +- *(1) HashAggregate(keys=[_gen_alias_139928#139928, _gen_alias_139929#139929], functions=[partial_sum(cast(_gen_alias_139931#139931 as bigint))], output=[_g en_alias_139928#139928, _gen_alias_139929#139929, sum#139934L]) +- *(1) Project [STRUCT_INT_DOUBLE_STRING_DATE#139885.COUNTRY AS _gen_alias_139928#139928, STRUCT_INT_DOUBLE_STRING_DATE#139885.CHECK_DATE AS _gen_alias_13 9929#139929, STRUCT_INT_DOUBLE_STRING_DATE#139885.COUNTRY AS _gen_alias_139930#139930, STRUCT_INT_DOUBLE_STRING_DATE#139885.ID AS _gen_alias_139931#139931] +- *(1) Filter (isnotnull(STRUCT_INT_DOUBLE_STRING_DATE#139885) AND (STRUCT_INT_DOUBLE_STRING_DATE#139885.ID > 5700)) +- FileScan parquet default.struct_com[STRUCT_INT_DOUBLE_STRING_DATE#139885] Batched: false, DataFilters: [isnotnull(STRUCT_INT_DOUBLE_STRING_DATE#13 9885), (STRUCT_INT_DOUBLE_STRING_DATE#139885.ID > 5700)], Format: Parquet, Location: InMemoryFileIndex[hdfs://hacluster/user/hive/warehouse/struct_com], PartitionFi lters: [], PushedFilters: [IsNotNull(STRUCT_INT_DOUBLE_STRING_DATE), GreaterThan(STRUCT_INT_DOUBLE_STRING_DATE.ID,5700)], ReadSchema: struct> at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(Spar kExecuteStatementOperation.scala:396) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$3(SparkExecuteStatementOperation.scala:281) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) at org.apache.spark.sql
[jira] [Commented] (SPARK-35635) concurrent insert statements from multiple beeline fail with job aborted exception
[ https://issues.apache.org/jira/browse/SPARK-35635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17358356#comment-17358356 ] Chetan Bhat commented on SPARK-35635: - Yes thats the issue. That has to be taken care from the system during concurrent query execution. > concurrent insert statements from multiple beeline fail with job aborted > exception > -- > > Key: SPARK-35635 > URL: https://issues.apache.org/jira/browse/SPARK-35635 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.1 > Environment: Spark 3.1.1 >Reporter: Chetan Bhat >Priority: Minor > > Create tables - > CREATE TABLE J1_TBL ( > i integer, > j integer, > t string > ) USING parquet; > CREATE TABLE J2_TBL ( > i integer, > k integer > ) USING parquet; > From 4 concurrent beeline sessions execute the insert into select queries - > INSERT INTO J1_TBL VALUES (1, 4, 'one'); > INSERT INTO J1_TBL VALUES (2, 3, 'two'); > INSERT INTO J1_TBL VALUES (3, 2, 'three'); > INSERT INTO J1_TBL VALUES (4, 1, 'four'); > INSERT INTO J1_TBL VALUES (5, 0, 'five'); > INSERT INTO J1_TBL VALUES (6, 6, 'six'); > INSERT INTO J1_TBL VALUES (7, 7, 'seven'); > INSERT INTO J1_TBL VALUES (8, 8, 'eight'); > INSERT INTO J1_TBL VALUES (0, NULL, 'zero'); > INSERT INTO J1_TBL VALUES (NULL, NULL, 'null'); > INSERT INTO J1_TBL VALUES (NULL, 0, 'zero'); > INSERT INTO J2_TBL VALUES (1, -1); > INSERT INTO J2_TBL VALUES (2, 2); > INSERT INTO J2_TBL VALUES (3, -3); > INSERT INTO J2_TBL VALUES (2, 4); > INSERT INTO J2_TBL VALUES (5, -5); > INSERT INTO J2_TBL VALUES (5, -5); > INSERT INTO J2_TBL VALUES (0, NULL); > INSERT INTO J2_TBL VALUES (NULL, NULL); > INSERT INTO J2_TBL VALUES (NULL, 0); > > Issue : concurrent insert statements from multiple beeline fail with job > aborted exception. > 0: jdbc:hive2://10.19.89.222:23040/> INSERT INTO J1_TBL VALUES (8, 8, > 'eight'); > Error: org.apache.hive.service.cli.HiveSQLException: Error running query: > org.apache.spark.SparkException: Job aborted. > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:366) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1781/750578465.apply$mcV$sp(Unknown > Source) > at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:45) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.spark.SparkException: Job aborted. > at > org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:231) > at > org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:188) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:109) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:107) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:121) > at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:228) > at org.apache.spark.sql.Dataset$$Lambda$1650/1168893915.apply(Unknown Source) > at org.apache.spark.sql.Datas
[jira] [Closed] (SPARK-35631) java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 executed with set spark.sql.ansi.enabled=true
[ https://issues.apache.org/jira/browse/SPARK-35631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat closed SPARK-35631. --- this is not a issue. Closed without handling. > java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 > executed with set spark.sql.ansi.enabled=true > > > Key: SPARK-35631 > URL: https://issues.apache.org/jira/browse/SPARK-35631 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.1 > Environment: Spark 3.1.1 >Reporter: Chetan Bhat >Priority: Minor > > From Spark beeline the queries are executed > set spark.sql.ansi.enabled=true > SELECT 2147483647 + 1 > > Issue : The select query fails with java.lang.ArithmeticException: integer > overflow > 0: jdbc:hive2://10.20.253.239:23040/default> set spark.sql.ansi.enabled=true; > +-++ > | key | value | > +-++ > | spark.sql.ansi.enabled | true | > +-++ > 1 row selected (0.052 seconds) > 0: jdbc:hive2://10.20.253.239:23040/default> SELECT 2147483647 + 1; > Error: org.apache.hive.service.cli.HiveSQLException: Error running query: > java.lang.ArithmeticException: integer overflow > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:361) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1762/209207680.apply$mcV$sp(Unknown > Source) > at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ArithmeticException: integer overflow > at java.lang.Math.addExact(Math.java:790) > at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:95) > at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:94) > at > org.apache.spark.sql.catalyst.expressions.Add.nullSafeEval(arithmetic.scala:264) > at > org.apache.spark.sql.catalyst.expressions.BinaryExpression.eval(Expression.scala:567) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:66) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:54) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:317) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1613/79619382.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:73) > at > org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:317) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:322) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1615/1159662764.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:407) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1601/550689618.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterat
[jira] [Created] (SPARK-35635) concurrent insert statements from multiple beeline fail with job aborted exception
Chetan Bhat created SPARK-35635: --- Summary: concurrent insert statements from multiple beeline fail with job aborted exception Key: SPARK-35635 URL: https://issues.apache.org/jira/browse/SPARK-35635 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat Create tables - CREATE TABLE J1_TBL ( i integer, j integer, t string ) USING parquet; CREATE TABLE J2_TBL ( i integer, k integer ) USING parquet; >From 4 concurrent beeline sessions execute the insert into select queries - INSERT INTO J1_TBL VALUES (1, 4, 'one'); INSERT INTO J1_TBL VALUES (2, 3, 'two'); INSERT INTO J1_TBL VALUES (3, 2, 'three'); INSERT INTO J1_TBL VALUES (4, 1, 'four'); INSERT INTO J1_TBL VALUES (5, 0, 'five'); INSERT INTO J1_TBL VALUES (6, 6, 'six'); INSERT INTO J1_TBL VALUES (7, 7, 'seven'); INSERT INTO J1_TBL VALUES (8, 8, 'eight'); INSERT INTO J1_TBL VALUES (0, NULL, 'zero'); INSERT INTO J1_TBL VALUES (NULL, NULL, 'null'); INSERT INTO J1_TBL VALUES (NULL, 0, 'zero'); INSERT INTO J2_TBL VALUES (1, -1); INSERT INTO J2_TBL VALUES (2, 2); INSERT INTO J2_TBL VALUES (3, -3); INSERT INTO J2_TBL VALUES (2, 4); INSERT INTO J2_TBL VALUES (5, -5); INSERT INTO J2_TBL VALUES (5, -5); INSERT INTO J2_TBL VALUES (0, NULL); INSERT INTO J2_TBL VALUES (NULL, NULL); INSERT INTO J2_TBL VALUES (NULL, 0); Issue : concurrent insert statements from multiple beeline fail with job aborted exception. 0: jdbc:hive2://10.19.89.222:23040/> INSERT INTO J1_TBL VALUES (8, 8, 'eight'); Error: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.SparkException: Job aborted. at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:366) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1781/750578465.apply$mcV$sp(Unknown Source) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:45) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.spark.SparkException: Job aborted. at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:231) at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:188) at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:109) at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:107) at org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:121) at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:228) at org.apache.spark.sql.Dataset$$Lambda$1650/1168893915.apply(Unknown Source) at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3687) at org.apache.spark.sql.Dataset$$Lambda$1651/490917717.apply(Unknown Source) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103) at org.apache.spark.sql.execution.SQLExecution$$$Lambda$1659/232159932.apply(Unknown Source) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90) at org.apache.spark.sql.ex
[jira] [Created] (SPARK-35634) Select queries with union throw parsing exception
Chetan Bhat created SPARK-35634: --- Summary: Select queries with union throw parsing exception Key: SPARK-35634 URL: https://issues.apache.org/jira/browse/SPARK-35634 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat >From Spark beeline some select queries with union are executed. Issue - Some select union queries throw parsing exception. SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1; SELECT 1.1 AS three UNION SELECT 2 UNION SELECT 3 ORDER BY 1; 0: jdbc:hive2://10.19.89.222:23040/> SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1; Error: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.catalyst.parser.ParseException: mismatched input 'SELECT' expecting \{, ';'}(line 1, pos 47) == SQL == SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1 ---^^^ at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:366) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1781/750578465.apply$mcV$sp(Unknown Source) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:45) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.spark.sql.catalyst.parser.ParseException: mismatched input 'SELECT' expecting \{, ';'}(line 1, pos 47) == SQL == SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1 ---^^^ at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124) at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75) at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:613) at org.apache.spark.sql.SparkSession$$Lambda$1239/1044989659.apply(Unknown Source) at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:613) at org.apache.spark.sql.SparkSession$$Lambda$1236/744351354.apply(Unknown Source) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:777) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:610) at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:650) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:325) ... 17 more (state=,code=0) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-35633) Select nvl query throws parse exception
Chetan Bhat created SPARK-35633: --- Summary: Select nvl query throws parse exception Key: SPARK-35633 URL: https://issues.apache.org/jira/browse/SPARK-35633 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat >From spark beeline execute the queries . Select nvl query throws parse >exception 0: jdbc:hive2://10.20.253.239:23040/default> CREATE TEMPORARY VIEW tempView1 AS VALUES (1, NAMED_STRUCT('col1', 'gamma', 'col2', 'delta')) AS T(id, st); +-+ | Result | +-+ +-+ No rows selected (0.018 seconds) 0: jdbc:hive2://10.20.253.239:23040/default> SELECT nvl(st.col1, "value"), count(*) FROM from tempView1 GROUP BY nvl(st.col1, "value"); Error: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.catalyst.parser.ParseException: no viable alternative at input 'from'(line 1, pos 44) == SQL == SELECT nvl(st.col1, "value"), count(*) FROM from tempView1 GROUP BY nvl(st.col1, "value") ^^^ at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:361) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1762/209207680.apply$mcV$sp(Unknown Source) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.spark.sql.catalyst.parser.ParseException: no viable alternative at input 'from'(line 1, pos 44) == SQL == SELECT nvl(st.col1, "value"), count(*) FROM from tempView1 GROUP BY nvl(st.col1, "value") ^^^ at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124) at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49) at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75) at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:613) at org.apache.spark.sql.SparkSession$$Lambda$1224/1732187536.apply(Unknown Source) at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:613) at org.apache.spark.sql.SparkSession$$Lambda$1221/1720168341.apply(Unknown Source) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:772) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:610) at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:650) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:325) ... 17 more (state=,code=0) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-35632) SELECT rand(cast(3 / 7 AS int)) value gives different resultset in Spark 3.1 as compared to Spark 2.4.5
Chetan Bhat created SPARK-35632: --- Summary: SELECT rand(cast(3 / 7 AS int)) value gives different resultset in Spark 3.1 as compared to Spark 2.4.5 Key: SPARK-35632 URL: https://issues.apache.org/jira/browse/SPARK-35632 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat Issue - SELECT rand(cast(3 / 7 AS int)) value gives different resultset in Spark 3.1 as compared to Spark 2.4.5 Queries are executed in Spark beeline. Output in Spark 3.1 0: jdbc:hive2://10.20.253.239:23040/default> SELECT rand(cast(3 / 7 AS int)); ++ | rand(CAST((CAST(3 AS DOUBLE) / CAST(7 AS DOUBLE)) AS INT)) | ++ | 0.7604953758285915 | ++ 1 row selected (0.042 seconds) Output in Spark 2.4.5 0: jdbc:hive2://linux-63:22550/> SELECT rand(cast(3 / 7 AS int)); +-+ | rand(CAST((CAST(3 AS DOUBLE) / CAST(7 AS DOUBLE)) AS INT)) | +-+ | 0.8446490682263027 | +-+ 1 row selected (6.358 seconds) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-35631) java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 executed with set spark.sql.ansi.enabled=true
[ https://issues.apache.org/jira/browse/SPARK-35631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat resolved SPARK-35631. - Resolution: Invalid As per the information provided in [https://spark.apache.org/docs/3.1.2/sql-ref-ansi-compliance.html#content] link this is not a issue. > java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 > executed with set spark.sql.ansi.enabled=true > > > Key: SPARK-35631 > URL: https://issues.apache.org/jira/browse/SPARK-35631 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.1 > Environment: Spark 3.1.1 >Reporter: Chetan Bhat >Priority: Minor > > From Spark beeline the queries are executed > set spark.sql.ansi.enabled=true > SELECT 2147483647 + 1 > > Issue : The select query fails with java.lang.ArithmeticException: integer > overflow > 0: jdbc:hive2://10.20.253.239:23040/default> set spark.sql.ansi.enabled=true; > +-++ > | key | value | > +-++ > | spark.sql.ansi.enabled | true | > +-++ > 1 row selected (0.052 seconds) > 0: jdbc:hive2://10.20.253.239:23040/default> SELECT 2147483647 + 1; > Error: org.apache.hive.service.cli.HiveSQLException: Error running query: > java.lang.ArithmeticException: integer overflow > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:361) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1762/209207680.apply$mcV$sp(Unknown > Source) > at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ArithmeticException: integer overflow > at java.lang.Math.addExact(Math.java:790) > at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:95) > at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:94) > at > org.apache.spark.sql.catalyst.expressions.Add.nullSafeEval(arithmetic.scala:264) > at > org.apache.spark.sql.catalyst.expressions.BinaryExpression.eval(Expression.scala:567) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:66) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:54) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:317) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1613/79619382.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:73) > at > org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:317) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:322) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1615/1159662764.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:407) > at > org.apache.spark.sql.catalyst.trees.TreeNode
[jira] [Created] (SPARK-35631) java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 executed with set spark.sql.ansi.enabled=true
Chetan Bhat created SPARK-35631: --- Summary: java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 executed with set spark.sql.ansi.enabled=true Key: SPARK-35631 URL: https://issues.apache.org/jira/browse/SPARK-35631 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat >From Spark beeline the queries are executed set spark.sql.ansi.enabled=true SELECT 2147483647 + 1 Issue : The select query fails with java.lang.ArithmeticException: integer overflow 0: jdbc:hive2://10.20.253.239:23040/default> set spark.sql.ansi.enabled=true; +-++ | key | value | +-++ | spark.sql.ansi.enabled | true | +-++ 1 row selected (0.052 seconds) 0: jdbc:hive2://10.20.253.239:23040/default> SELECT 2147483647 + 1; Error: org.apache.hive.service.cli.HiveSQLException: Error running query: java.lang.ArithmeticException: integer overflow at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:361) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1762/209207680.apply$mcV$sp(Unknown Source) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArithmeticException: integer overflow at java.lang.Math.addExact(Math.java:790) at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:95) at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:94) at org.apache.spark.sql.catalyst.expressions.Add.nullSafeEval(arithmetic.scala:264) at org.apache.spark.sql.catalyst.expressions.BinaryExpression.eval(Expression.scala:567) at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:66) at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:54) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:317) at org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1613/79619382.apply(Unknown Source) at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:73) at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:317) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:322) at org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1615/1159662764.apply(Unknown Source) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:407) at org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1601/550689618.apply(Unknown Source) at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:243) at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:405) at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:358) at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:322) at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$transformExpressionsDown$1(QueryPlan.scala:94) at org.apache.spark.sql.catalyst.plans.QueryPlan$$Lambda$1315/1031179320.apply(Unknown Source
[jira] [Created] (SPARK-34673) Select queries fail with Error: java.lang.IllegalArgumentException: Error: name expected at the position 10 of 'decimal(2,-2)' but '-' is found. (state=,code=0)
Chetan Bhat created SPARK-34673: --- Summary: Select queries fail with Error: java.lang.IllegalArgumentException: Error: name expected at the position 10 of 'decimal(2,-2)' but '-' is found. (state=,code=0) Key: SPARK-34673 URL: https://issues.apache.org/jira/browse/SPARK-34673 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.4.5 Environment: Spark 2.4.5 Reporter: Chetan Bhat Temporary views are created Select filter queries are executed on the Temporary views. [Actual Issue] : - Select queries fail with Error: java.lang.IllegalArgumentException: Error: name expected at the position 10 of 'decimal(2,-2)' but '-' is found. (state=,code=0) [Expected Result] :- Select queries should be success on Temporary views. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-34179) examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working
[ https://issues.apache.org/jira/browse/SPARK-34179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-34179: Description: *Issue 1 :* *array_sort examples provided in [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not working.* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); –> *this example when executed in spark-sql fails with below error* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 48) == SQL == SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end) ^^^ spark-sql> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end); --> *This example when executed fails with below error* spark-sql> > > > > > > > > > > > > > > > > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left > is null and right is null then 0 when left is null then -1 when right is > null then 1 when left < right then 1 when left > right then -1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 57) == SQL == SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end) -^^^ spark-sql> *Issue 2 :-* *Examples for overlay functions are not working in link - https://spark.apache.org/docs/latest/api/sql/index.html* spark-sql> SELECT overlay('Spark SQL' PLACING '_' FROM 6); Error in query: mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 27) == SQL == SELECT overlay('Spark SQL' PLACING '_' FROM 6) ---^^^ spark-sql> SELECT overlay('Spark SQL' PLACING 'CORE' FROM 7); Error in query: mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 27) == SQL == SELECT overlay('Spark SQL' PLACING 'CORE' FROM 7) ---^^^ spark-sql> SELECT overlay('Spark SQL' PLACING 'ANSI ' FROM 7 FOR 0); Error in query: mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 27) == SQL == SELECT overlay('Spark SQL' PLACING 'ANSI ' FROM 7 FOR 0) ---^^^ spark-sql> SELECT overlay('Spark SQL' PLACING 'tructured' FROM 2 FOR 4); Error in query: mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 27) == SQL == SELECT overlay('Spark SQL' PLACING 'tructured' FROM 2 FOR 4) ---^^^ spark-sql> SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('_', 'utf-8') FROM 6); Error in query: mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 44) == SQL == SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('_', 'utf-8') FROM 6) ^^^ spark-sql> SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('CORE', 'utf-8') FROM 7); Error in query: mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 44) == SQL == SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('CORE', 'utf-8') FROM 7) ^^^ spark-sql> SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('ANSI ', 'utf-8') FROM 7 FOR 0); Error in query: mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 44) == SQL == SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('ANSI ', 'utf-8') FROM 7 FOR 0) ^^^ spark-sql> SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('tructured', 'utf-8') FROM 2 FOR 4); Error in query: mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 44) == SQL == SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('tructured', 'utf-8') FROM 2 FOR 4) ^^^ was: *array_sort examples provided in [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not working.* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); –> *this example when executed in spark-sql fails with below error* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 48) == SQL == SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0
[jira] [Updated] (SPARK-34179) examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working
[ https://issues.apache.org/jira/browse/SPARK-34179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-34179: Description: *array_sort examples provided in [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not working.* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); –> *this example when executed in spark-sql fails with below error* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 48) == SQL == SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end) ^^^ spark-sql> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end); --> *This example when executed fails with below error* spark-sql> > > > > > > > > > > > > > > > > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left > is null and right is null then 0 when left is null then -1 when right is > null then 1 when left < right then 1 when left > right then -1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 57) == SQL == SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end) -^^^ spark-sql> was: *array_sort examples provided in [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not working.* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); –> *this example when executed in spark-sql fails with below error* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 48) == SQL == SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end) ^^^ spark-sql> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end); --> *This example when executed fails with below error* spark-sql> > > > > > > > > > > > > > > > > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left > is null and right is null then 0 when left is null then -1 when right is > null then 1 when left < right then 1 when left > right then -1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 57) == SQL == SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end) -^^^ spark-sql> Issue 2 :- > examples provided in https://spark.apache.org/docs/latest/api/sql/index.html > link not working > --- > > Key: SPARK-34179 > URL: https://issues.apache.org/jira/browse/SPARK-34179 > Project: Spark > Issue Type: Bug > Components: docs >Affects Versions: 2.4.5 > Environment: Spark 2.4.5 >Reporter: Chetan Bhat >Priority: Minor > > *array_sort examples provided in > [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not > working.* > > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end); –> *this example when executed > in spark-sql fails with below error* > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end); > Error in query: > extraneous input '->' expecting \{')', ','}(line 1, pos 48) > == SQL == > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end) > ^^^ > spark-sql> > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is > null and right i
[jira] [Updated] (SPARK-34179) examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working
[ https://issues.apache.org/jira/browse/SPARK-34179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-34179: Description: *array_sort examples provided in [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not working.* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); –> *this example when executed in spark-sql fails with below error* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 48) == SQL == SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end) ^^^ spark-sql> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end); --> *This example when executed fails with below error* spark-sql> > > > > > > > > > > > > > > > > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left > is null and right is null then 0 when left is null then -1 when right is > null then 1 when left < right then 1 when left > right then -1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 57) == SQL == SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end) -^^^ spark-sql> Issue 2 :- was: *array_sort examples provided in https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not working.* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); –> *this example when executed in spark-sql fails with below error* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 48) == SQL == SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end) ^^^ spark-sql> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end); --> *This example when executed fails with below error* spark-sql> > > > > > > > > > > > > > > > > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left > is null and right is null then 0 when left is null then -1 when right is > null then 1 when left < right then 1 when left > right then -1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 57) == SQL == SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end) -^^^ spark-sql> > examples provided in https://spark.apache.org/docs/latest/api/sql/index.html > link not working > --- > > Key: SPARK-34179 > URL: https://issues.apache.org/jira/browse/SPARK-34179 > Project: Spark > Issue Type: Bug > Components: docs >Affects Versions: 2.4.5 > Environment: Spark 2.4.5 >Reporter: Chetan Bhat >Priority: Minor > > *array_sort examples provided in > [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not > working.* > > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end); –> *this example when executed > in spark-sql fails with below error* > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end); > Error in query: > extraneous input '->' expecting \{')', ','}(line 1, pos 48) > == SQL == > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end) > ^^^ > spark-sql> > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is > null and right is null the
[jira] [Updated] (SPARK-34179) examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working
[ https://issues.apache.org/jira/browse/SPARK-34179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-34179: Summary: examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working (was: array_sort examples provided in https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not working) > examples provided in https://spark.apache.org/docs/latest/api/sql/index.html > link not working > --- > > Key: SPARK-34179 > URL: https://issues.apache.org/jira/browse/SPARK-34179 > Project: Spark > Issue Type: Bug > Components: docs >Affects Versions: 2.4.5 > Environment: Spark 2.4.5 >Reporter: Chetan Bhat >Priority: Minor > > *array_sort examples provided in > https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not > working.* > > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end); –> *this example when executed > in spark-sql fails with below error* > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end); > Error in query: > extraneous input '->' expecting \{')', ','}(line 1, pos 48) > == SQL == > SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right > then -1 when left > right then 1 else 0 end) > ^^^ > spark-sql> > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is > null and right is null then 0 when left is null then -1 when right is null > then 1 when left < right then 1 when left > right then -1 else 0 end); --> > *This example when executed fails with below error* > > spark-sql> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left > is null and right is null then 0 when left is null then -1 when right is null > then 1 when left < right then 1 when left > right then -1 else 0 end); > Error in query: > extraneous input '->' expecting \{')', ','}(line 1, pos 57) > == SQL == > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is > null and right is null then 0 when left is null then -1 when right is null > then 1 when left < right then 1 when left > right then -1 else 0 end) > -^^^ > spark-sql> > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-34179) array_sort examples provided in https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not working
Chetan Bhat created SPARK-34179: --- Summary: array_sort examples provided in https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not working Key: SPARK-34179 URL: https://issues.apache.org/jira/browse/SPARK-34179 Project: Spark Issue Type: Bug Components: docs Affects Versions: 2.4.5 Environment: Spark 2.4.5 Reporter: Chetan Bhat *array_sort examples provided in https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not working.* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); –> *this example when executed in spark-sql fails with below error* SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 48) == SQL == SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then -1 when left > right then 1 else 0 end) ^^^ spark-sql> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end); --> *This example when executed fails with below error* spark-sql> > > > > > > > > > > > > > > > > > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left > is null and right is null then 0 when left is null then -1 when right is > null then 1 when left < right then 1 when left > right then -1 else 0 end); Error in query: extraneous input '->' expecting \{')', ','}(line 1, pos 57) == SQL == SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is null and right is null then 0 when left is null then -1 when right is null then 1 when left < right then 1 when left > right then -1 else 0 end) -^^^ spark-sql> -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32887) Example command in https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html to be changed
[ https://issues.apache.org/jira/browse/SPARK-32887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-32887: Description: In the link [https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html] the below command example mentioned is wrong. SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1'); Complete example executed throws below error. CREATE TABLE employee(name STRING)PARTITIONED BY (grade int) stored as parquet; INSERT INTO employee PARTITION (grade = 1) VALUES ('sam'); INSERT INTO employee PARTITION (grade = 2) VALUES ('suj'); spark-sql> SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1'); **Error in query:** ``` mismatched input ''grade=1'' expecting \{'ADD', 'AFTER', 'ALL', 'ALTER', 'ANALYZE', 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT', 'AUTHORIZATION', 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE', 'CASCADE', 'CASE', 'CAST', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER', 'CLUSTERED', 'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT', 'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE', 'CONSTRAINT', 'COST', 'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE', 'CURRENT_TIME', 'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DATA', 'DATABASE', DATABASES, 'DAY', 'DBPROPERTIES', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC', 'DESCRIBE', 'DFS', 'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE', 'DROP', 'ELSE', 'END', 'ESCAPE', 'ESCAPED', 'EXCEPT', 'EXCHANGE', 'EXISTS', 'EXPLAIN', 'EXPORT', 'EXTENDED', 'EXTERNAL', 'EXTRACT', 'FALSE', 'FETCH', 'FIELDS', 'FILTER', 'FILEFORMAT', 'FIRST', 'FOLLOWING', 'FOR', 'FOREIGN', 'FORMAT', 'FORMATTED', 'FROM', 'FULL', 'FUNCTION', 'FUNCTIONS', 'GLOBAL', 'GRANT', 'GROUP', 'GROUPING', 'HAVING', 'HOUR', 'IF', 'IGNORE', 'IMPORT', 'IN', 'INDEX', 'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT', 'INSERT', 'INTERSECT', 'INTERVAL', 'INTO', 'IS', 'ITEMS', 'JOIN', 'KEYS', 'LAST', 'LATERAL', 'LAZY', 'LEADING', 'LEFT', 'LIKE', 'LIMIT', 'LINES', 'LIST', 'LOAD', 'LOCAL', 'LOCATION', 'LOCK', 'LOCKS', 'LOGICAL', 'MACRO', 'MAP', 'MATCHED', 'MERGE', 'MINUTE', 'MONTH', 'MSCK', 'NAMESPACE', 'NAMESPACES', 'NATURAL', 'NO', NOT, 'NULL', 'NULLS', 'OF', 'ON', 'ONLY', 'OPTION', 'OPTIONS', 'OR', 'ORDER', 'OUT', 'OUTER', 'OUTPUTFORMAT', 'OVER', 'OVERLAPS', 'OVERLAY', 'OVERWRITE', 'PARTITION', 'PARTITIONED', 'PARTITIONS', 'PERCENT', 'PIVOT', 'PLACING', 'POSITION', 'PRECEDING', 'PRIMARY', 'PRINCIPALS', 'PROPERTIES', 'PURGE', 'QUERY', 'RANGE', 'RECORDREADER', 'RECORDWRITER', 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH', 'RENAME', 'REPAIR', 'REPLACE', 'RESET', 'RESTRICT', 'REVOKE', 'RIGHT', RLIKE, 'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP', 'ROW', 'ROWS', 'SCHEMA', 'SECOND', 'SELECT', 'SEMI', 'SEPARATED', 'SERDE', 'SERDEPROPERTIES', 'SESSION_USER', 'SET', 'MINUS', 'SETS', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START', 'STATISTICS', 'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'TABLE', 'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES', TEMPORARY, 'TERMINATED', 'THEN', 'TO', 'TOUCH', 'TRAILING', 'TRANSACTION', 'TRANSACTIONS', 'TRANSFORM', 'TRIM', 'TRUE', 'TRUNCATE', 'TYPE', 'UNARCHIVE', 'UNBOUNDED', 'UNCACHE', 'UNION', 'UNIQUE', 'UNKNOWN', 'UNLOCK', 'UNSET', 'UPDATE', 'USE', 'USER', 'USING', 'VALUES', 'VIEW', 'VIEWS', 'WHEN', 'WHERE', 'WINDOW', 'WITH', 'YEAR', 'DIV', IDENTIFIER, BACKQUOTED_IDENTIFIER}(line 1, pos 59) == SQL == SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1') ---^^^ ``` Expected : - If that partition value is string we can give like this grade ='abc' was: In the link [https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html] the below command example mentioned is wrong. SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1'); CREATE TABLE employee(name STRING)PARTITIONED BY (grade int) stored as parquet; INSERT INTO employee PARTITION (grade = 1) VALUES ('sam'); INSERT INTO employee PARTITION (grade = 2) VALUES ('suj'); spark-sql> SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1'); **Error in query:** ``` mismatched input ''grade=1'' expecting \{'ADD', 'AFTER', 'ALL', 'ALTER', 'ANALYZE', 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT', 'AUTHORIZATION', 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE', 'CASCADE', 'CASE', 'CAST', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER', 'CLUSTERED', 'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT', 'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE', 'CONSTRAINT', 'COST', 'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE', 'CURRENT_TIME', 'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DATA', 'DATABASE', DATABASES, 'DAY', 'DBPROPERTIES', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC', 'DESCRIBE', 'DFS', 'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE', 'DROP', 'EL
[jira] [Created] (SPARK-32887) Example command in https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html to be changed
Chetan Bhat created SPARK-32887: --- Summary: Example command in https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html to be changed Key: SPARK-32887 URL: https://issues.apache.org/jira/browse/SPARK-32887 Project: Spark Issue Type: Bug Components: docs Affects Versions: 3.0.0 Environment: Spark 2.4.5, Spark 3.0.0 Reporter: Chetan Bhat In the link [https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html] the below command example mentioned is wrong. SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1'); CREATE TABLE employee(name STRING)PARTITIONED BY (grade int) stored as parquet; INSERT INTO employee PARTITION (grade = 1) VALUES ('sam'); INSERT INTO employee PARTITION (grade = 2) VALUES ('suj'); spark-sql> SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1'); **Error in query:** ``` mismatched input ''grade=1'' expecting \{'ADD', 'AFTER', 'ALL', 'ALTER', 'ANALYZE', 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT', 'AUTHORIZATION', 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE', 'CASCADE', 'CASE', 'CAST', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER', 'CLUSTERED', 'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT', 'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE', 'CONSTRAINT', 'COST', 'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE', 'CURRENT_TIME', 'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DATA', 'DATABASE', DATABASES, 'DAY', 'DBPROPERTIES', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC', 'DESCRIBE', 'DFS', 'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE', 'DROP', 'ELSE', 'END', 'ESCAPE', 'ESCAPED', 'EXCEPT', 'EXCHANGE', 'EXISTS', 'EXPLAIN', 'EXPORT', 'EXTENDED', 'EXTERNAL', 'EXTRACT', 'FALSE', 'FETCH', 'FIELDS', 'FILTER', 'FILEFORMAT', 'FIRST', 'FOLLOWING', 'FOR', 'FOREIGN', 'FORMAT', 'FORMATTED', 'FROM', 'FULL', 'FUNCTION', 'FUNCTIONS', 'GLOBAL', 'GRANT', 'GROUP', 'GROUPING', 'HAVING', 'HOUR', 'IF', 'IGNORE', 'IMPORT', 'IN', 'INDEX', 'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT', 'INSERT', 'INTERSECT', 'INTERVAL', 'INTO', 'IS', 'ITEMS', 'JOIN', 'KEYS', 'LAST', 'LATERAL', 'LAZY', 'LEADING', 'LEFT', 'LIKE', 'LIMIT', 'LINES', 'LIST', 'LOAD', 'LOCAL', 'LOCATION', 'LOCK', 'LOCKS', 'LOGICAL', 'MACRO', 'MAP', 'MATCHED', 'MERGE', 'MINUTE', 'MONTH', 'MSCK', 'NAMESPACE', 'NAMESPACES', 'NATURAL', 'NO', NOT, 'NULL', 'NULLS', 'OF', 'ON', 'ONLY', 'OPTION', 'OPTIONS', 'OR', 'ORDER', 'OUT', 'OUTER', 'OUTPUTFORMAT', 'OVER', 'OVERLAPS', 'OVERLAY', 'OVERWRITE', 'PARTITION', 'PARTITIONED', 'PARTITIONS', 'PERCENT', 'PIVOT', 'PLACING', 'POSITION', 'PRECEDING', 'PRIMARY', 'PRINCIPALS', 'PROPERTIES', 'PURGE', 'QUERY', 'RANGE', 'RECORDREADER', 'RECORDWRITER', 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH', 'RENAME', 'REPAIR', 'REPLACE', 'RESET', 'RESTRICT', 'REVOKE', 'RIGHT', RLIKE, 'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP', 'ROW', 'ROWS', 'SCHEMA', 'SECOND', 'SELECT', 'SEMI', 'SEPARATED', 'SERDE', 'SERDEPROPERTIES', 'SESSION_USER', 'SET', 'MINUS', 'SETS', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START', 'STATISTICS', 'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'TABLE', 'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES', TEMPORARY, 'TERMINATED', 'THEN', 'TO', 'TOUCH', 'TRAILING', 'TRANSACTION', 'TRANSACTIONS', 'TRANSFORM', 'TRIM', 'TRUE', 'TRUNCATE', 'TYPE', 'UNARCHIVE', 'UNBOUNDED', 'UNCACHE', 'UNION', 'UNIQUE', 'UNKNOWN', 'UNLOCK', 'UNSET', 'UPDATE', 'USE', 'USER', 'USING', 'VALUES', 'VIEW', 'VIEWS', 'WHEN', 'WHERE', 'WINDOW', 'WITH', 'YEAR', 'DIV', IDENTIFIER, BACKQUOTED_IDENTIFIER}(line 1, pos 59) == SQL == SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1') ---^^^ ``` Expected : - If that partition value is string we can give like this grade ='abc' -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-32700) select from table TABLESAMPLE gives wrong resultset.
Chetan Bhat created SPARK-32700: --- Summary: select from table TABLESAMPLE gives wrong resultset. Key: SPARK-32700 URL: https://issues.apache.org/jira/browse/SPARK-32700 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.0.0 Environment: Spark 3.0.0 Reporter: Chetan Bhat create table test(id int,name string) stored as parquet; insert into test values (5,'Alex'),(8,'Lucy'),(2,'Mary'),(4,'Fred'),(1,'Lisa'),(9,'Eric'),(10,'Adam'),(6,'Mark'),(7,'Lily'),(3,'Evan'); SELECT * FROM test TABLESAMPLE (50 PERCENT); --> output is giving only 3 rows. spark-sql> SELECT * FROM test TABLESAMPLE (50 PERCENT); 5 Alex 10 Adam 4 Fred Expected as per the link is 5 rows -->[https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-sampling.html] Also the bucket parameter for select from table TABLESAMPLE gives wrong resultset. spark-sql> SELECT * FROM test TABLESAMPLE (BUCKET 4 OUT OF 10); 5 Alex 8 Lucy 9 Eric 1 Lisa 3 Evan Expected is 4 records. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-32697) Direct Date and timestamp format data insertion fails
Chetan Bhat created SPARK-32697: --- Summary: Direct Date and timestamp format data insertion fails Key: SPARK-32697 URL: https://issues.apache.org/jira/browse/SPARK-32697 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.0.0 Environment: Spark 3.0.0 Reporter: Chetan Bhat Direct Date and timestamp format data tried to be inserted but the data insertion fails as shown below. spark-sql> create table test(no timestamp) stored as parquet; Time taken: 0.561 seconds spark-sql> insert into test select '1979-04-27 00:00:00'; Error in query: Cannot write incompatible data to table '`default`.`test`': - Cannot safely cast 'no': string to timestamp; -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-22859) Permission of created table and database folder are not correct
Chetan Bhat created SPARK-22859: --- Summary: Permission of created table and database folder are not correct Key: SPARK-22859 URL: https://issues.apache.org/jira/browse/SPARK-22859 Project: Spark Issue Type: Bug Components: Spark Submit Affects Versions: 2.1.0 Reporter: Chetan Bhat Steps : In spark Beeline user executes the following queries. drop database if exists test_db1 cascade; create database test_db1 location 'hdfs://hacluster/user/test1'; use test_db1; create table if not exists ORDERS(O_ORDERDATE string,O_ORDERPRIORITY string,O_ORDERSTATUS string,O_ORDERKEY string,O_CUSTKEY string,O_TOTALPRICE double,O_CLERK string,O_SHIPPRIORITY int,O_COMMENT string) STORED BY 'org.apache.carbondata.format'TBLPROPERTIES ('table_blocksize'='128'); User checks the permission of the created database and table in carbon store using the bin/hadoop fs -getfacl command. Issue : The Permission of created table and database folder are not correct. i.e file: /user/test1/orders owner: anonymous group: users user::rwx group::r-x other::r-x Expected : Correct permissions for the created table and database folder in carbon store should be file: /user/test1/orders owner: anonymous group: users user::rwx group::--- other::--- -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Reopened] (SPARK-22841) Select regexp_extract from table with where clause having is null throws indexoutofbounds exception
[ https://issues.apache.org/jira/browse/SPARK-22841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat reopened SPARK-22841: - Exception should not be thrown in any scenario. Instead proper error message can be displayed. > Select regexp_extract from table with where clause having is null throws > indexoutofbounds exception > --- > > Key: SPARK-22841 > URL: https://issues.apache.org/jira/browse/SPARK-22841 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.1.0 >Reporter: Chetan Bhat > > Steps : > Thrift server is started using the command - bin/spark-submit --master > yarn-client --executor-memory 10G --executor-cores 5 --driver-memory 5G > --num-executors 3 --class > org.apache.carbondata.spark.thriftserver.CarbonThriftServer > /srv/spark2.2Bigdata/install/spark/sparkJdbc/carbonlib/carbondata_2.11-1.3.0-SNAPSHOT-shade-hadoop2.7.2.jar > "hdfs://hacluster/user/sparkhive/warehouse" > Spark shell is launched using the command - bin/spark-shell --master > yarn-client --executor-memory 10G --executor-cores 5 --driver-memory 5G > --num-executors 3 --jars > /srv/spark2.2Bigdata/install/spark/sparkJdbc/carbonlib/carbondata_2.11-1.3.0-SNAPSHOT-shade-hadoop2.7.2.jar > From Spark shell the streaming table is created and data is loaded to the > streaming table. > import java.io. > {File, PrintWriter} > import java.net.ServerSocket > import org.apache.spark.sql. > {CarbonEnv, SparkSession} > import org.apache.spark.sql.hive.CarbonRelation > import org.apache.spark.sql.streaming. > {ProcessingTime, StreamingQuery} > import org.apache.carbondata.core.constants.CarbonCommonConstants > import org.apache.carbondata.core.util.CarbonProperties > import org.apache.carbondata.core.util.path. > {CarbonStorePath, CarbonTablePath} > CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT, > "/MM/dd") > import org.apache.spark.sql.CarbonSession._ > val carbonSession = SparkSession. > builder(). > appName("StreamExample"). > getOrCreateCarbonSession("hdfs://hacluster/user/hive/warehouse/carbon.store") > carbonSession.sparkContext.setLogLevel("INFO") > def sql(sql: String) = carbonSession.sql(sql) > def writeSocket(serverSocket: ServerSocket): Thread = { > val thread = new Thread() { > override def run(): Unit = { > // wait for client to connection request and accept > val clientSocket = serverSocket.accept() > val socketWriter = new PrintWriter(clientSocket.getOutputStream()) > var index = 0 > for (_ <- 1 to 1000) { > // write 5 records per iteration > for (_ <- 0 to 100) > { index = index + 1 socketWriter.println(index.toString + ",name_" + index + > ",city_" + index + "," + (index * 1.00).toString + ",school_" + index + > ":school_" + index + index + "$" + index) } > socketWriter.flush() > Thread.sleep(2000) > } > socketWriter.close() > System.out.println("Socket closed") > } > } > thread.start() > thread > } > def startStreaming(spark: SparkSession, tablePath: CarbonTablePath, > tableName: String, port: Int): Thread = { > val thread = new Thread() { > override def run(): Unit = { > var qry: StreamingQuery = null > try > { val readSocketDF = spark.readStream .format("socket") .option("host", > "10.18.98.34") .option("port", port) .load() qry = readSocketDF.writeStream > .format("carbondata") .trigger(ProcessingTime("5 seconds")) > .option("checkpointLocation", tablePath.getStreamingCheckpointDir) > .option("tablePath", tablePath.getPath).option("tableName", tableName) > .start() qry.awaitTermination() } > catch > { case ex: Throwable => ex.printStackTrace() println("Done reading and > writing streaming data") } > finally > { qry.stop() } > } > } > thread.start() > thread > } > val streamTableName = "uniqdata" > sql(s"CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION > string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 > bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 > decimal(36,36),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 > int) STORED BY 'org.apache.carbondata.format' > TBLPROPERTIES('streaming'='true')") > sql(s"LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table > uniqdata OPTIONS( > 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')") > val carbonTable = CarbonEnv.getInstance(carbonSession).carbonMetastore. > lookupRelation(Some("default"), > streamTableName)(carbonSession).asInstanceOf[CarbonRelation].carbonTable > val tablePath = > CarbonStorePath.getCarbonTablePath(carbonTable.getAbsoluteTableIdentifier) > val port = 8006 > val serverSocket = new Serve
[jira] [Created] (SPARK-22841) Select regexp_extract from table with where clause having is null throws indexoutofbounds exception
Chetan Bhat created SPARK-22841: --- Summary: Select regexp_extract from table with where clause having is null throws indexoutofbounds exception Key: SPARK-22841 URL: https://issues.apache.org/jira/browse/SPARK-22841 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.1.0 Reporter: Chetan Bhat Steps : Thrift server is started using the command - bin/spark-submit --master yarn-client --executor-memory 10G --executor-cores 5 --driver-memory 5G --num-executors 3 --class org.apache.carbondata.spark.thriftserver.CarbonThriftServer /srv/spark2.2Bigdata/install/spark/sparkJdbc/carbonlib/carbondata_2.11-1.3.0-SNAPSHOT-shade-hadoop2.7.2.jar "hdfs://hacluster/user/sparkhive/warehouse" Spark shell is launched using the command - bin/spark-shell --master yarn-client --executor-memory 10G --executor-cores 5 --driver-memory 5G --num-executors 3 --jars /srv/spark2.2Bigdata/install/spark/sparkJdbc/carbonlib/carbondata_2.11-1.3.0-SNAPSHOT-shade-hadoop2.7.2.jar >From Spark shell the streaming table is created and data is loaded to the >streaming table. import java.io. {File, PrintWriter} import java.net.ServerSocket import org.apache.spark.sql. {CarbonEnv, SparkSession} import org.apache.spark.sql.hive.CarbonRelation import org.apache.spark.sql.streaming. {ProcessingTime, StreamingQuery} import org.apache.carbondata.core.constants.CarbonCommonConstants import org.apache.carbondata.core.util.CarbonProperties import org.apache.carbondata.core.util.path. {CarbonStorePath, CarbonTablePath} CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT, "/MM/dd") import org.apache.spark.sql.CarbonSession._ val carbonSession = SparkSession. builder(). appName("StreamExample"). getOrCreateCarbonSession("hdfs://hacluster/user/hive/warehouse/carbon.store") carbonSession.sparkContext.setLogLevel("INFO") def sql(sql: String) = carbonSession.sql(sql) def writeSocket(serverSocket: ServerSocket): Thread = { val thread = new Thread() { override def run(): Unit = { // wait for client to connection request and accept val clientSocket = serverSocket.accept() val socketWriter = new PrintWriter(clientSocket.getOutputStream()) var index = 0 for (_ <- 1 to 1000) { // write 5 records per iteration for (_ <- 0 to 100) { index = index + 1 socketWriter.println(index.toString + ",name_" + index + ",city_" + index + "," + (index * 1.00).toString + ",school_" + index + ":school_" + index + index + "$" + index) } socketWriter.flush() Thread.sleep(2000) } socketWriter.close() System.out.println("Socket closed") } } thread.start() thread } def startStreaming(spark: SparkSession, tablePath: CarbonTablePath, tableName: String, port: Int): Thread = { val thread = new Thread() { override def run(): Unit = { var qry: StreamingQuery = null try { val readSocketDF = spark.readStream .format("socket") .option("host", "10.18.98.34") .option("port", port) .load() qry = readSocketDF.writeStream .format("carbondata") .trigger(ProcessingTime("5 seconds")) .option("checkpointLocation", tablePath.getStreamingCheckpointDir) .option("tablePath", tablePath.getPath).option("tableName", tableName) .start() qry.awaitTermination() } catch { case ex: Throwable => ex.printStackTrace() println("Done reading and writing streaming data") } finally { qry.stop() } } } thread.start() thread } val streamTableName = "uniqdata" sql(s"CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,36),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('streaming'='true')") sql(s"LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table uniqdata OPTIONS( 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')") val carbonTable = CarbonEnv.getInstance(carbonSession).carbonMetastore. lookupRelation(Some("default"), streamTableName)(carbonSession).asInstanceOf[CarbonRelation].carbonTable val tablePath = CarbonStorePath.getCarbonTablePath(carbonTable.getAbsoluteTableIdentifier) val port = 8006 val serverSocket = new ServerSocket(port) val socketThread = writeSocket(serverSocket) val streamingThread = startStreaming(carbonSession, tablePath, streamTableName, port) >From Beeline user executes the query select regexp_extract(CUST_NAME,'a',1)from uniqdata where regexp_extract(CUST_NAME,'a',1) IS NULL or regexp_extract(DOB,'b',2) is NULL; Issue : Select regexp_extract from table with where clause having is null throws indexoutofbounds exception 0: jdbc:hive2://10.18.98.34:23040> select regexp_ext
[jira] [Updated] (SPARK-14197) Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#333
[ https://issues.apache.org/jira/browse/SPARK-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-14197: Fix Version/s: (was: 1.5.1) > Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid > call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = > 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) > -- > > Key: SPARK-14197 > URL: https://issues.apache.org/jira/browse/SPARK-14197 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 >Reporter: Chetan Bhat >Priority: Minor > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > In Beeline when the SQL is executed using Spark the following error occurs. > select count(*) from babu1 where imei in ( select if (imei = > '1AA10007',imei,NULL) from babu1); > Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid > call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = > 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-14197) Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#333
[ https://issues.apache.org/jira/browse/SPARK-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-14197: Affects Version/s: 1.5.1 > Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid > call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = > 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) > -- > > Key: SPARK-14197 > URL: https://issues.apache.org/jira/browse/SPARK-14197 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 >Reporter: Chetan Bhat >Priority: Minor > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > In Beeline when the SQL is executed using Spark the following error occurs. > select count(*) from babu1 where imei in ( select if (imei = > '1AA10007',imei,NULL) from babu1); > Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid > call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = > 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14199) Error: org.apache.spark.sql.hive.huawei.rules.analyze.UnrelatedSubqueryConversion$MultiValueException$: subquery expression must return single column! (state=,code=0)
Chetan Bhat created SPARK-14199: --- Summary: Error: org.apache.spark.sql.hive.huawei.rules.analyze.UnrelatedSubqueryConversion$MultiValueException$: subquery expression must return single column! (state=,code=0) Key: SPARK-14199 URL: https://issues.apache.org/jira/browse/SPARK-14199 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.1 Reporter: Chetan Bhat Priority: Minor In Beeline when the SQL is executed using Spark error is displayed. select imei,a, (select MAC from babu8 a where a.imei = '1AA10007' ) from ( select imei,count(contractNumber)+1 a from testolap group by imei) a where imei in ( select imei from testolap where imei like '%007%') group by imei, a; Error: org.apache.spark.sql.hive.huawei.rules.analyze.UnrelatedSubqueryConversion$MultiValueException$: subquery expression must return single column! (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14198) Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '' '' '' in subquery source; line 1 pos 86 (state=,code=0)
Chetan Bhat created SPARK-14198: --- Summary: Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '' '' '' in subquery source; line 1 pos 86 (state=,code=0) Key: SPARK-14198 URL: https://issues.apache.org/jira/browse/SPARK-14198 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.1 Reporter: Chetan Bhat Priority: Minor In Beeline when the SQL is executed using Spark the error occurs. select count(*) from ( select case when imei like '%007%' then imei end from babu8); Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '' '' '' in subquery source; line 1 pos 86 (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-14196) Error: org.apache.spark.sql.AnalysisException: cannot resolve 'query_alias_fix_conflicts_0._c0'; (state=,code=0)
[ https://issues.apache.org/jira/browse/SPARK-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-14196: Component/s: SQL > Error: org.apache.spark.sql.AnalysisException: cannot resolve > 'query_alias_fix_conflicts_0._c0'; (state=,code=0) > > > Key: SPARK-14196 > URL: https://issues.apache.org/jira/browse/SPARK-14196 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 >Reporter: Chetan Bhat >Priority: Minor > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > In Beeline when the SQL is executed using Spark the following error is > displayed. > select count(*) from testolap where imei in ( select case when imei like > '%007%' then imei end from testolap); > Error: org.apache.spark.sql.AnalysisException: cannot resolve > 'query_alias_fix_conflicts_0._c0'; (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14197) Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#333
Chetan Bhat created SPARK-14197: --- Summary: Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) Key: SPARK-14197 URL: https://issues.apache.org/jira/browse/SPARK-14197 Project: Spark Issue Type: Bug Components: SQL Reporter: Chetan Bhat Priority: Minor Fix For: 1.5.1 In Beeline when the SQL is executed using Spark the following error occurs. select count(*) from babu1 where imei in ( select if (imei = '1AA10007',imei,NULL) from babu1); Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14196) Error: org.apache.spark.sql.AnalysisException: cannot resolve 'query_alias_fix_conflicts_0._c0'; (state=,code=0)
Chetan Bhat created SPARK-14196: --- Summary: Error: org.apache.spark.sql.AnalysisException: cannot resolve 'query_alias_fix_conflicts_0._c0'; (state=,code=0) Key: SPARK-14196 URL: https://issues.apache.org/jira/browse/SPARK-14196 Project: Spark Issue Type: Bug Affects Versions: 1.5.1 Reporter: Chetan Bhat Priority: Minor In Beeline when the SQL is executed using Spark the following error is displayed. select count(*) from testolap where imei in ( select case when imei like '%007%' then imei end from testolap); Error: org.apache.spark.sql.AnalysisException: cannot resolve 'query_alias_fix_conflicts_0._c0'; (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-14195) Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'select' 'MAC' 'from' in expression specification; line 1 pos 16 (state=,code=0)
[ https://issues.apache.org/jira/browse/SPARK-14195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-14195: Environment: SUSE 11 Remaining Estimate: 0.05h Original Estimate: 0.05h Description: In Beeline when the SQL is executed using Spark the following error is displayed. select a.imei, (select MAC from testolap where imei like '%AA%' 1) MAC from testolap a limit 10; Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'select' 'MAC' 'from' in expression specification; line 1 pos 16 (state=,code=0) > Error: org.apache.spark.sql.AnalysisException: cannot recognize input near > 'select' 'MAC' 'from' in expression specification; line 1 pos 16 > (state=,code=0) > --- > > Key: SPARK-14195 > URL: https://issues.apache.org/jira/browse/SPARK-14195 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 > Environment: SUSE 11 >Reporter: Chetan Bhat >Priority: Minor > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > In Beeline when the SQL is executed using Spark the following error is > displayed. > select a.imei, (select MAC from testolap where imei like '%AA%' 1) MAC from > testolap a limit 10; > Error: org.apache.spark.sql.AnalysisException: cannot recognize input near > 'select' 'MAC' 'from' in expression specification; line 1 pos 16 > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14195) Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'select' 'MAC' 'from' in expression specification; line 1 pos 16 (state=,code=0)
Chetan Bhat created SPARK-14195: --- Summary: Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'select' 'MAC' 'from' in expression specification; line 1 pos 16 (state=,code=0) Key: SPARK-14195 URL: https://issues.apache.org/jira/browse/SPARK-14195 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.1 Reporter: Chetan Bhat Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org