[jira] [Created] (SPARK-35835) Select filter query on table with struct complex type fails
Chetan Bhat created SPARK-35835: --- Summary: Select filter query on table with struct complex type fails Key: SPARK-35835 URL: https://issues.apache.org/jira/browse/SPARK-35835 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat [Steps]:- >From Spark beeline create a parquet or ORC table having complex type data. >Load data in the table and execute select filter query. 0: jdbc:hive2://vm2:22550/> create table Struct_com (CUST_ID string, YEAR int, MONTH int, AGE int, GENDER string, EDUCATED string, IS_MARRIED string, STRUCT_INT_DOUBLE_STRING_DATE struct,CARD_COUNT int,DEBIT_COUNT int, CREDIT_COUNT int, DEPOSIT double, HQ_DEPOSIT double) stored as parquet; +-+ | Result | +-+ +-+ No rows selected (0.161 seconds) 0: jdbc:hive2://vm2:22550/> LOAD DATA INPATH 'hdfs://hacluster/chetan/Struct.csv' OVERWRITE INTO TABLE Struct_com; +-+ | Result | +-+ +-+ No rows selected (1.09 seconds) 0: jdbc:hive2://vm2:22550/> SELECT struct_int_double_string_date.COUNTRY,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.CHECK_DATE,struct_in t_double_string_date.Country, SUM(struct_int_double_string_date.id) AS Sum FROM (select * from Struct_com) SUB_QRY WHERE struct_int_double_string_date.id > 5700 GRO UP BY struct_int_double_string_date.COUNTRY,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.Country ORDER BY struct_int_double_string_date.COUNTRY asc,struct_int_double_string_date.CHECK_DATE asc,struct_int_double_string_date.CHECK_DATE asc, struct_int_double_stri ng_date.Country asc; [Actual Issue] : - Select filter query on table with struct complex type fails 0: jdbc:hive2://vm2:22550/> SELECT struct_int_double_string_date.COUNTRY,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.CHECK_DATE,struct_in t_double_string_date.Country, SUM(struct_int_double_string_date.id) AS Sum FROM (select * from Struct_com) SUB_QRY WHERE struct_int_double_string_date.id > 5700 GRO UP BY struct_int_double_string_date.COUNTRY,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.CHECK_DATE,struct_int_double_string_date.Country ORDER BY struct_int_double_string_date.COUNTRY asc,struct_int_double_string_date.CHECK_DATE asc,struct_int_double_string_date.CHECK_DATE asc, struct_int_double_stri ng_date.Country asc; Error: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: execute, tree: Exchange rangepartitioning(COUNTRY#139896 ASC NULLS FIRST, CHECK_DATE#139897 ASC NULLS FIRST, CHECK_DATE#139897 ASC NULLS FIRST, COUNTRY#139896 ASC NULLS FIRST, 200 ), ENSURE_REQUIREMENTS, [id=#17161] +- *(2) HashAggregate(keys=[_gen_alias_139928#139928, _gen_alias_139929#139929], functions=[sum(cast(_gen_alias_139931#139931 as bigint))], output=[COUNTRY#139896, CHECK_DATE#139897, CHECK_DATE#139898, Country#139899, Sum#139877L]) +- Exchange hashpartitioning(_gen_alias_139928#139928, _gen_alias_139929#139929, 200), ENSURE_REQUIREMENTS, [id=#17157] +- *(1) HashAggregate(keys=[_gen_alias_139928#139928, _gen_alias_139929#139929], functions=[partial_sum(cast(_gen_alias_139931#139931 as bigint))], output=[_g en_alias_139928#139928, _gen_alias_139929#139929, sum#139934L]) +- *(1) Project [STRUCT_INT_DOUBLE_STRING_DATE#139885.COUNTRY AS _gen_alias_139928#139928, STRUCT_INT_DOUBLE_STRING_DATE#139885.CHECK_DATE AS _gen_alias_13 9929#139929, STRUCT_INT_DOUBLE_STRING_DATE#139885.COUNTRY AS _gen_alias_139930#139930, STRUCT_INT_DOUBLE_STRING_DATE#139885.ID AS _gen_alias_139931#139931] +- *(1) Filter (isnotnull(STRUCT_INT_DOUBLE_STRING_DATE#139885) AND (STRUCT_INT_DOUBLE_STRING_DATE#139885.ID > 5700)) +- FileScan parquet default.struct_com[STRUCT_INT_DOUBLE_STRING_DATE#139885] Batched: false, DataFilters: [isnotnull(STRUCT_INT_DOUBLE_STRING_DATE#13 9885), (STRUCT_INT_DOUBLE_STRING_DATE#139885.ID > 5700)], Format: Parquet, Location: InMemoryFileIndex[hdfs://hacluster/user/hive/warehouse/struct_com], PartitionFi lters: [], PushedFilters: [IsNotNull(STRUCT_INT_DOUBLE_STRING_DATE), GreaterThan(STRUCT_INT_DOUBLE_STRING_DATE.ID,5700)], ReadSchema: struct> at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(Spar kExecuteStatementOperation.scala:396) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$3(SparkExecuteStatementOperation.scala:281) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) at
[jira] [Commented] (SPARK-35635) concurrent insert statements from multiple beeline fail with job aborted exception
[ https://issues.apache.org/jira/browse/SPARK-35635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358356#comment-17358356 ] Chetan Bhat commented on SPARK-35635: - Yes thats the issue. That has to be taken care from the system during concurrent query execution. > concurrent insert statements from multiple beeline fail with job aborted > exception > -- > > Key: SPARK-35635 > URL: https://issues.apache.org/jira/browse/SPARK-35635 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.1 > Environment: Spark 3.1.1 >Reporter: Chetan Bhat >Priority: Minor > > Create tables - > CREATE TABLE J1_TBL ( > i integer, > j integer, > t string > ) USING parquet; > CREATE TABLE J2_TBL ( > i integer, > k integer > ) USING parquet; > From 4 concurrent beeline sessions execute the insert into select queries - > INSERT INTO J1_TBL VALUES (1, 4, 'one'); > INSERT INTO J1_TBL VALUES (2, 3, 'two'); > INSERT INTO J1_TBL VALUES (3, 2, 'three'); > INSERT INTO J1_TBL VALUES (4, 1, 'four'); > INSERT INTO J1_TBL VALUES (5, 0, 'five'); > INSERT INTO J1_TBL VALUES (6, 6, 'six'); > INSERT INTO J1_TBL VALUES (7, 7, 'seven'); > INSERT INTO J1_TBL VALUES (8, 8, 'eight'); > INSERT INTO J1_TBL VALUES (0, NULL, 'zero'); > INSERT INTO J1_TBL VALUES (NULL, NULL, 'null'); > INSERT INTO J1_TBL VALUES (NULL, 0, 'zero'); > INSERT INTO J2_TBL VALUES (1, -1); > INSERT INTO J2_TBL VALUES (2, 2); > INSERT INTO J2_TBL VALUES (3, -3); > INSERT INTO J2_TBL VALUES (2, 4); > INSERT INTO J2_TBL VALUES (5, -5); > INSERT INTO J2_TBL VALUES (5, -5); > INSERT INTO J2_TBL VALUES (0, NULL); > INSERT INTO J2_TBL VALUES (NULL, NULL); > INSERT INTO J2_TBL VALUES (NULL, 0); > > Issue : concurrent insert statements from multiple beeline fail with job > aborted exception. > 0: jdbc:hive2://10.19.89.222:23040/> INSERT INTO J1_TBL VALUES (8, 8, > 'eight'); > Error: org.apache.hive.service.cli.HiveSQLException: Error running query: > org.apache.spark.SparkException: Job aborted. > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:366) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1781/750578465.apply$mcV$sp(Unknown > Source) > at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:45) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.spark.SparkException: Job aborted. > at > org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:231) > at > org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:188) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:109) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:107) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:121) > at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:228) > at org.apache.spark.sql.Dataset$$Lambda$1650/1168893915.apply(Unknown Source) > at
[jira] [Closed] (SPARK-35631) java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 executed with set spark.sql.ansi.enabled=true
[ https://issues.apache.org/jira/browse/SPARK-35631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat closed SPARK-35631. --- this is not a issue. Closed without handling. > java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 > executed with set spark.sql.ansi.enabled=true > > > Key: SPARK-35631 > URL: https://issues.apache.org/jira/browse/SPARK-35631 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.1 > Environment: Spark 3.1.1 >Reporter: Chetan Bhat >Priority: Minor > > From Spark beeline the queries are executed > set spark.sql.ansi.enabled=true > SELECT 2147483647 + 1 > > Issue : The select query fails with java.lang.ArithmeticException: integer > overflow > 0: jdbc:hive2://10.20.253.239:23040/default> set spark.sql.ansi.enabled=true; > +-++ > | key | value | > +-++ > | spark.sql.ansi.enabled | true | > +-++ > 1 row selected (0.052 seconds) > 0: jdbc:hive2://10.20.253.239:23040/default> SELECT 2147483647 + 1; > Error: org.apache.hive.service.cli.HiveSQLException: Error running query: > java.lang.ArithmeticException: integer overflow > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:361) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1762/209207680.apply$mcV$sp(Unknown > Source) > at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ArithmeticException: integer overflow > at java.lang.Math.addExact(Math.java:790) > at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:95) > at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:94) > at > org.apache.spark.sql.catalyst.expressions.Add.nullSafeEval(arithmetic.scala:264) > at > org.apache.spark.sql.catalyst.expressions.BinaryExpression.eval(Expression.scala:567) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:66) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:54) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:317) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1613/79619382.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:73) > at > org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:317) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:322) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1615/1159662764.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:407) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1601/550689618.apply(Unknown > Source) > at >
[jira] [Created] (SPARK-35635) concurrent insert statements from multiple beeline fail with job aborted exception
Chetan Bhat created SPARK-35635: --- Summary: concurrent insert statements from multiple beeline fail with job aborted exception Key: SPARK-35635 URL: https://issues.apache.org/jira/browse/SPARK-35635 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat Create tables - CREATE TABLE J1_TBL ( i integer, j integer, t string ) USING parquet; CREATE TABLE J2_TBL ( i integer, k integer ) USING parquet; >From 4 concurrent beeline sessions execute the insert into select queries - INSERT INTO J1_TBL VALUES (1, 4, 'one'); INSERT INTO J1_TBL VALUES (2, 3, 'two'); INSERT INTO J1_TBL VALUES (3, 2, 'three'); INSERT INTO J1_TBL VALUES (4, 1, 'four'); INSERT INTO J1_TBL VALUES (5, 0, 'five'); INSERT INTO J1_TBL VALUES (6, 6, 'six'); INSERT INTO J1_TBL VALUES (7, 7, 'seven'); INSERT INTO J1_TBL VALUES (8, 8, 'eight'); INSERT INTO J1_TBL VALUES (0, NULL, 'zero'); INSERT INTO J1_TBL VALUES (NULL, NULL, 'null'); INSERT INTO J1_TBL VALUES (NULL, 0, 'zero'); INSERT INTO J2_TBL VALUES (1, -1); INSERT INTO J2_TBL VALUES (2, 2); INSERT INTO J2_TBL VALUES (3, -3); INSERT INTO J2_TBL VALUES (2, 4); INSERT INTO J2_TBL VALUES (5, -5); INSERT INTO J2_TBL VALUES (5, -5); INSERT INTO J2_TBL VALUES (0, NULL); INSERT INTO J2_TBL VALUES (NULL, NULL); INSERT INTO J2_TBL VALUES (NULL, 0); Issue : concurrent insert statements from multiple beeline fail with job aborted exception. 0: jdbc:hive2://10.19.89.222:23040/> INSERT INTO J1_TBL VALUES (8, 8, 'eight'); Error: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.SparkException: Job aborted. at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:366) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1781/750578465.apply$mcV$sp(Unknown Source) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:45) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.spark.SparkException: Job aborted. at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:231) at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:188) at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:109) at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:107) at org.apache.spark.sql.execution.command.DataWritingCommandExec.executeCollect(commands.scala:121) at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:228) at org.apache.spark.sql.Dataset$$Lambda$1650/1168893915.apply(Unknown Source) at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3687) at org.apache.spark.sql.Dataset$$Lambda$1651/490917717.apply(Unknown Source) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:103) at org.apache.spark.sql.execution.SQLExecution$$$Lambda$1659/232159932.apply(Unknown Source) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:163) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:90) at
[jira] [Created] (SPARK-35634) Select queries with union throw parsing exception
Chetan Bhat created SPARK-35634:
---
Summary: Select queries with union throw parsing exception
Key: SPARK-35634
URL: https://issues.apache.org/jira/browse/SPARK-35634
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.1
Environment: Spark 3.1.1
Reporter: Chetan Bhat
>From Spark beeline some select queries with union are executed.
Issue - Some select union queries throw parsing exception.
SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1;
SELECT 1.1 AS three UNION SELECT 2 UNION SELECT 3 ORDER BY 1;
0: jdbc:hive2://10.19.89.222:23040/> SELECT double(1.1) AS two UNION SELECT 2
UNION SELECT double(2.0) ORDER BY 1;
Error: org.apache.hive.service.cli.HiveSQLException: Error running query:
org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'SELECT' expecting \{, ';'}(line 1, pos 47)
== SQL ==
SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1
---^^^
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:366)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1781/750578465.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78)
at
org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:45)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'SELECT' expecting \{, ';'}(line 1, pos 47)
== SQL ==
SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1
---^^^
at
org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
at
org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
at
org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:613)
at org.apache.spark.sql.SparkSession$$Lambda$1239/1044989659.apply(Unknown
Source)
at
org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:613)
at org.apache.spark.sql.SparkSession$$Lambda$1236/744351354.apply(Unknown
Source)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:777)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:610)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:650)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:325)
... 17 more (state=,code=0)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-35633) Select nvl query throws parse exception
Chetan Bhat created SPARK-35633:
---
Summary: Select nvl query throws parse exception
Key: SPARK-35633
URL: https://issues.apache.org/jira/browse/SPARK-35633
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.1
Environment: Spark 3.1.1
Reporter: Chetan Bhat
>From spark beeline execute the queries . Select nvl query throws parse
>exception
0: jdbc:hive2://10.20.253.239:23040/default> CREATE TEMPORARY VIEW tempView1 AS
VALUES (1, NAMED_STRUCT('col1', 'gamma', 'col2', 'delta')) AS T(id, st);
+-+
| Result |
+-+
+-+
No rows selected (0.018 seconds)
0: jdbc:hive2://10.20.253.239:23040/default> SELECT nvl(st.col1, "value"),
count(*) FROM from tempView1 GROUP BY nvl(st.col1, "value");
Error: org.apache.hive.service.cli.HiveSQLException: Error running query:
org.apache.spark.sql.catalyst.parser.ParseException:
no viable alternative at input 'from'(line 1, pos 44)
== SQL ==
SELECT nvl(st.col1, "value"), count(*) FROM from tempView1 GROUP BY
nvl(st.col1, "value")
^^^
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:361)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1762/209207680.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78)
at
org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.spark.sql.catalyst.parser.ParseException:
no viable alternative at input 'from'(line 1, pos 44)
== SQL ==
SELECT nvl(st.col1, "value"), count(*) FROM from tempView1 GROUP BY
nvl(st.col1, "value")
^^^
at
org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
at
org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
at
org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:613)
at org.apache.spark.sql.SparkSession$$Lambda$1224/1732187536.apply(Unknown
Source)
at
org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:613)
at org.apache.spark.sql.SparkSession$$Lambda$1221/1720168341.apply(Unknown
Source)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:772)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:610)
at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:650)
at
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:325)
... 17 more (state=,code=0)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-35632) SELECT rand(cast(3 / 7 AS int)) value gives different resultset in Spark 3.1 as compared to Spark 2.4.5
Chetan Bhat created SPARK-35632: --- Summary: SELECT rand(cast(3 / 7 AS int)) value gives different resultset in Spark 3.1 as compared to Spark 2.4.5 Key: SPARK-35632 URL: https://issues.apache.org/jira/browse/SPARK-35632 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat Issue - SELECT rand(cast(3 / 7 AS int)) value gives different resultset in Spark 3.1 as compared to Spark 2.4.5 Queries are executed in Spark beeline. Output in Spark 3.1 0: jdbc:hive2://10.20.253.239:23040/default> SELECT rand(cast(3 / 7 AS int)); ++ | rand(CAST((CAST(3 AS DOUBLE) / CAST(7 AS DOUBLE)) AS INT)) | ++ | 0.7604953758285915 | ++ 1 row selected (0.042 seconds) Output in Spark 2.4.5 0: jdbc:hive2://linux-63:22550/> SELECT rand(cast(3 / 7 AS int)); +-+ | rand(CAST((CAST(3 AS DOUBLE) / CAST(7 AS DOUBLE)) AS INT)) | +-+ | 0.8446490682263027 | +-+ 1 row selected (6.358 seconds) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-35631) java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 executed with set spark.sql.ansi.enabled=true
[ https://issues.apache.org/jira/browse/SPARK-35631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat resolved SPARK-35631. - Resolution: Invalid As per the information provided in [https://spark.apache.org/docs/3.1.2/sql-ref-ansi-compliance.html#content] link this is not a issue. > java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 > executed with set spark.sql.ansi.enabled=true > > > Key: SPARK-35631 > URL: https://issues.apache.org/jira/browse/SPARK-35631 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.1.1 > Environment: Spark 3.1.1 >Reporter: Chetan Bhat >Priority: Minor > > From Spark beeline the queries are executed > set spark.sql.ansi.enabled=true > SELECT 2147483647 + 1 > > Issue : The select query fails with java.lang.ArithmeticException: integer > overflow > 0: jdbc:hive2://10.20.253.239:23040/default> set spark.sql.ansi.enabled=true; > +-++ > | key | value | > +-++ > | spark.sql.ansi.enabled | true | > +-++ > 1 row selected (0.052 seconds) > 0: jdbc:hive2://10.20.253.239:23040/default> SELECT 2147483647 + 1; > Error: org.apache.hive.service.cli.HiveSQLException: Error running query: > java.lang.ArithmeticException: integer overflow > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:361) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1762/209207680.apply$mcV$sp(Unknown > Source) > at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) > at > org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ArithmeticException: integer overflow > at java.lang.Math.addExact(Math.java:790) > at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:95) > at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:94) > at > org.apache.spark.sql.catalyst.expressions.Add.nullSafeEval(arithmetic.scala:264) > at > org.apache.spark.sql.catalyst.expressions.BinaryExpression.eval(Expression.scala:567) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:66) > at > org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:54) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:317) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1613/79619382.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:73) > at > org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:317) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:322) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1615/1159662764.apply(Unknown > Source) > at > org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:407) > at >
[jira] [Created] (SPARK-35631) java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 executed with set spark.sql.ansi.enabled=true
Chetan Bhat created SPARK-35631: --- Summary: java.lang.ArithmeticException: integer overflow when SELECT 2147483647 + 1 executed with set spark.sql.ansi.enabled=true Key: SPARK-35631 URL: https://issues.apache.org/jira/browse/SPARK-35631 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.1.1 Environment: Spark 3.1.1 Reporter: Chetan Bhat >From Spark beeline the queries are executed set spark.sql.ansi.enabled=true SELECT 2147483647 + 1 Issue : The select query fails with java.lang.ArithmeticException: integer overflow 0: jdbc:hive2://10.20.253.239:23040/default> set spark.sql.ansi.enabled=true; +-++ | key | value | +-++ | spark.sql.ansi.enabled | true | +-++ 1 row selected (0.052 seconds) 0: jdbc:hive2://10.20.253.239:23040/default> SELECT 2147483647 + 1; Error: org.apache.hive.service.cli.HiveSQLException: Error running query: java.lang.ArithmeticException: integer overflow at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(SparkExecuteStatementOperation.scala:361) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3$$Lambda$1762/209207680.apply$mcV$sp(Unknown Source) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78) at org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ArithmeticException: integer overflow at java.lang.Math.addExact(Math.java:790) at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:95) at org.apache.spark.sql.types.IntegerExactNumeric$.plus(numerics.scala:94) at org.apache.spark.sql.catalyst.expressions.Add.nullSafeEval(arithmetic.scala:264) at org.apache.spark.sql.catalyst.expressions.BinaryExpression.eval(Expression.scala:567) at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:66) at org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$1$$anonfun$applyOrElse$1.applyOrElse(expressions.scala:54) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$1(TreeNode.scala:317) at org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1613/79619382.apply(Unknown Source) at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:73) at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:317) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDown$3(TreeNode.scala:322) at org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1615/1159662764.apply(Unknown Source) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$mapChildren$1(TreeNode.scala:407) at org.apache.spark.sql.catalyst.trees.TreeNode$$Lambda$1601/550689618.apply(Unknown Source) at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:243) at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:405) at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:358) at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:322) at org.apache.spark.sql.catalyst.plans.QueryPlan.$anonfun$transformExpressionsDown$1(QueryPlan.scala:94) at org.apache.spark.sql.catalyst.plans.QueryPlan$$Lambda$1315/1031179320.apply(Unknown
[jira] [Created] (SPARK-34673) Select queries fail with Error: java.lang.IllegalArgumentException: Error: name expected at the position 10 of 'decimal(2,-2)' but '-' is found. (state=,code=0)
Chetan Bhat created SPARK-34673: --- Summary: Select queries fail with Error: java.lang.IllegalArgumentException: Error: name expected at the position 10 of 'decimal(2,-2)' but '-' is found. (state=,code=0) Key: SPARK-34673 URL: https://issues.apache.org/jira/browse/SPARK-34673 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.4.5 Environment: Spark 2.4.5 Reporter: Chetan Bhat Temporary views are created Select filter queries are executed on the Temporary views. [Actual Issue] : - Select queries fail with Error: java.lang.IllegalArgumentException: Error: name expected at the position 10 of 'decimal(2,-2)' but '-' is found. (state=,code=0) [Expected Result] :- Select queries should be success on Temporary views. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-34179) examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working
[
https://issues.apache.org/jira/browse/SPARK-34179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Bhat updated SPARK-34179:
Description:
*Issue 1 :*
*array_sort examples provided in
[https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not
working.*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end); –> *this example when executed in
spark-sql fails with below error*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 48)
== SQL ==
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end)
^^^
spark-sql>
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end); --> *This
example when executed fails with below error*
spark-sql>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left
> is null and right is null then 0 when left is null then -1 when right is
> null then 1 when left < right then 1 when left > right then -1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 57)
== SQL ==
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end)
-^^^
spark-sql>
*Issue 2 :-*
*Examples for overlay functions are not working in link -
https://spark.apache.org/docs/latest/api/sql/index.html*
spark-sql> SELECT overlay('Spark SQL' PLACING '_' FROM 6);
Error in query:
mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 27)
== SQL ==
SELECT overlay('Spark SQL' PLACING '_' FROM 6)
---^^^
spark-sql> SELECT overlay('Spark SQL' PLACING 'CORE' FROM 7);
Error in query:
mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 27)
== SQL ==
SELECT overlay('Spark SQL' PLACING 'CORE' FROM 7)
---^^^
spark-sql> SELECT overlay('Spark SQL' PLACING 'ANSI ' FROM 7 FOR 0);
Error in query:
mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 27)
== SQL ==
SELECT overlay('Spark SQL' PLACING 'ANSI ' FROM 7 FOR 0)
---^^^
spark-sql> SELECT overlay('Spark SQL' PLACING 'tructured' FROM 2 FOR 4);
Error in query:
mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 27)
== SQL ==
SELECT overlay('Spark SQL' PLACING 'tructured' FROM 2 FOR 4)
---^^^
spark-sql> SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('_',
'utf-8') FROM 6);
Error in query:
mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 44)
== SQL ==
SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('_', 'utf-8') FROM 6)
^^^
spark-sql> SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('CORE',
'utf-8') FROM 7);
Error in query:
mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 44)
== SQL ==
SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('CORE', 'utf-8')
FROM 7)
^^^
spark-sql> SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('ANSI ',
'utf-8') FROM 7 FOR 0);
Error in query:
mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 44)
== SQL ==
SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('ANSI ', 'utf-8')
FROM 7 FOR 0)
^^^
spark-sql> SELECT overlay(encode('Spark SQL', 'utf-8') PLACING
encode('tructured', 'utf-8') FROM 2 FOR 4);
Error in query:
mismatched input 'PLACING' expecting \{')', ','}(line 1, pos 44)
== SQL ==
SELECT overlay(encode('Spark SQL', 'utf-8') PLACING encode('tructured',
'utf-8') FROM 2 FOR 4)
^^^
was:
*array_sort examples provided in
[https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not
working.*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end); –> *this example when executed in
spark-sql fails with below error*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 48)
== SQL ==
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else
[jira] [Updated] (SPARK-34179) examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working
[
https://issues.apache.org/jira/browse/SPARK-34179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Bhat updated SPARK-34179:
Description:
*array_sort examples provided in
[https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not
working.*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end); –> *this example when executed in
spark-sql fails with below error*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 48)
== SQL ==
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end)
^^^
spark-sql>
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end); --> *This
example when executed fails with below error*
spark-sql>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left
> is null and right is null then 0 when left is null then -1 when right is
> null then 1 when left < right then 1 when left > right then -1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 57)
== SQL ==
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end)
-^^^
spark-sql>
was:
*array_sort examples provided in
[https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not
working.*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end); –> *this example when executed in
spark-sql fails with below error*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 48)
== SQL ==
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end)
^^^
spark-sql>
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end); --> *This
example when executed fails with below error*
spark-sql>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left
> is null and right is null then 0 when left is null then -1 when right is
> null then 1 when left < right then 1 when left > right then -1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 57)
== SQL ==
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end)
-^^^
spark-sql>
Issue 2 :-
> examples provided in https://spark.apache.org/docs/latest/api/sql/index.html
> link not working
> ---
>
> Key: SPARK-34179
> URL: https://issues.apache.org/jira/browse/SPARK-34179
> Project: Spark
> Issue Type: Bug
> Components: docs
>Affects Versions: 2.4.5
> Environment: Spark 2.4.5
>Reporter: Chetan Bhat
>Priority: Minor
>
> *array_sort examples provided in
> [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not
> working.*
>
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end); –> *this example when executed
> in spark-sql fails with below error*
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end);
> Error in query:
> extraneous input '->' expecting \{')', ','}(line 1, pos 48)
> == SQL ==
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end)
> ^^^
> spark-sql>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
> null and right
[jira] [Updated] (SPARK-34179) examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working
[
https://issues.apache.org/jira/browse/SPARK-34179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Bhat updated SPARK-34179:
Description:
*array_sort examples provided in
[https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not
working.*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end); –> *this example when executed in
spark-sql fails with below error*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 48)
== SQL ==
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end)
^^^
spark-sql>
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end); --> *This
example when executed fails with below error*
spark-sql>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left
> is null and right is null then 0 when left is null then -1 when right is
> null then 1 when left < right then 1 when left > right then -1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 57)
== SQL ==
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end)
-^^^
spark-sql>
Issue 2 :-
was:
*array_sort examples provided in
https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not
working.*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end); –> *this example when executed in
spark-sql fails with below error*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 48)
== SQL ==
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end)
^^^
spark-sql>
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end); --> *This
example when executed fails with below error*
spark-sql>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left
> is null and right is null then 0 when left is null then -1 when right is
> null then 1 when left < right then 1 when left > right then -1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 57)
== SQL ==
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end)
-^^^
spark-sql>
> examples provided in https://spark.apache.org/docs/latest/api/sql/index.html
> link not working
> ---
>
> Key: SPARK-34179
> URL: https://issues.apache.org/jira/browse/SPARK-34179
> Project: Spark
> Issue Type: Bug
> Components: docs
>Affects Versions: 2.4.5
> Environment: Spark 2.4.5
>Reporter: Chetan Bhat
>Priority: Minor
>
> *array_sort examples provided in
> [https://spark.apache.org/docs/latest/api/sql/index.html#array_sort] link not
> working.*
>
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end); –> *this example when executed
> in spark-sql fails with below error*
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end);
> Error in query:
> extraneous input '->' expecting \{')', ','}(line 1, pos 48)
> == SQL ==
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end)
> ^^^
> spark-sql>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
> null and right is null
[jira] [Updated] (SPARK-34179) examples provided in https://spark.apache.org/docs/latest/api/sql/index.html link not working
[
https://issues.apache.org/jira/browse/SPARK-34179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Bhat updated SPARK-34179:
Summary: examples provided in
https://spark.apache.org/docs/latest/api/sql/index.html link not working
(was: array_sort examples provided in
https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not
working)
> examples provided in https://spark.apache.org/docs/latest/api/sql/index.html
> link not working
> ---
>
> Key: SPARK-34179
> URL: https://issues.apache.org/jira/browse/SPARK-34179
> Project: Spark
> Issue Type: Bug
> Components: docs
>Affects Versions: 2.4.5
> Environment: Spark 2.4.5
>Reporter: Chetan Bhat
>Priority: Minor
>
> *array_sort examples provided in
> https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not
> working.*
>
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end); –> *this example when executed
> in spark-sql fails with below error*
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end);
> Error in query:
> extraneous input '->' expecting \{')', ','}(line 1, pos 48)
> == SQL ==
> SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right
> then -1 when left > right then 1 else 0 end)
> ^^^
> spark-sql>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
> null and right is null then 0 when left is null then -1 when right is null
> then 1 when left < right then 1 when left > right then -1 else 0 end); -->
> *This example when executed fails with below error*
>
> spark-sql>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left
> is null and right is null then 0 when left is null then -1 when right is null
> then 1 when left < right then 1 when left > right then -1 else 0 end);
> Error in query:
> extraneous input '->' expecting \{')', ','}(line 1, pos 57)
> == SQL ==
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
> null and right is null then 0 when left is null then -1 when right is null
> then 1 when left < right then 1 when left > right then -1 else 0 end)
> -^^^
> spark-sql>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-34179) array_sort examples provided in https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not working
Chetan Bhat created SPARK-34179:
---
Summary: array_sort examples provided in
https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not
working
Key: SPARK-34179
URL: https://issues.apache.org/jira/browse/SPARK-34179
Project: Spark
Issue Type: Bug
Components: docs
Affects Versions: 2.4.5
Environment: Spark 2.4.5
Reporter: Chetan Bhat
*array_sort examples provided in
https://spark.apache.org/docs/latest/api/sql/index.html#array_sort link not
working.*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end); –> *this example when executed in
spark-sql fails with below error*
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 48)
== SQL ==
SELECT array_sort(array(5, 6, 1), (left, right) -> case when left < right then
-1 when left > right then 1 else 0 end)
^^^
spark-sql>
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end); --> *This
example when executed fails with below error*
spark-sql>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left
> is null and right is null then 0 when left is null then -1 when right is
> null then 1 when left < right then 1 when left > right then -1 else 0 end);
Error in query:
extraneous input '->' expecting \{')', ','}(line 1, pos 57)
== SQL ==
SELECT array_sort(array('bc', 'ab', 'dc'), (left, right) -> case when left is
null and right is null then 0 when left is null then -1 when right is null then
1 when left < right then 1 when left > right then -1 else 0 end)
-^^^
spark-sql>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32887) Example command in https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html to be changed
[
https://issues.apache.org/jira/browse/SPARK-32887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Bhat updated SPARK-32887:
Description:
In the link
[https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html] the
below command example mentioned is wrong.
SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1');
Complete example executed throws below error.
CREATE TABLE employee(name STRING)PARTITIONED BY (grade int) stored as parquet;
INSERT INTO employee PARTITION (grade = 1) VALUES ('sam');
INSERT INTO employee PARTITION (grade = 2) VALUES ('suj');
spark-sql> SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1');
**Error in query:**
```
mismatched input ''grade=1'' expecting \{'ADD', 'AFTER', 'ALL', 'ALTER',
'ANALYZE', 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT',
'AUTHORIZATION', 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE',
'CASCADE', 'CASE', 'CAST', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER', 'CLUSTERED',
'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT',
'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE', 'CONSTRAINT', 'COST',
'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE', 'CURRENT_TIME',
'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DATA', 'DATABASE', DATABASES, 'DAY',
'DBPROPERTIES', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC', 'DESCRIBE', 'DFS',
'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE', 'DROP', 'ELSE', 'END',
'ESCAPE', 'ESCAPED', 'EXCEPT', 'EXCHANGE', 'EXISTS', 'EXPLAIN', 'EXPORT',
'EXTENDED', 'EXTERNAL', 'EXTRACT', 'FALSE', 'FETCH', 'FIELDS', 'FILTER',
'FILEFORMAT', 'FIRST', 'FOLLOWING', 'FOR', 'FOREIGN', 'FORMAT', 'FORMATTED',
'FROM', 'FULL', 'FUNCTION', 'FUNCTIONS', 'GLOBAL', 'GRANT', 'GROUP',
'GROUPING', 'HAVING', 'HOUR', 'IF', 'IGNORE', 'IMPORT', 'IN', 'INDEX',
'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT', 'INSERT', 'INTERSECT', 'INTERVAL',
'INTO', 'IS', 'ITEMS', 'JOIN', 'KEYS', 'LAST', 'LATERAL', 'LAZY', 'LEADING',
'LEFT', 'LIKE', 'LIMIT', 'LINES', 'LIST', 'LOAD', 'LOCAL', 'LOCATION', 'LOCK',
'LOCKS', 'LOGICAL', 'MACRO', 'MAP', 'MATCHED', 'MERGE', 'MINUTE', 'MONTH',
'MSCK', 'NAMESPACE', 'NAMESPACES', 'NATURAL', 'NO', NOT, 'NULL', 'NULLS', 'OF',
'ON', 'ONLY', 'OPTION', 'OPTIONS', 'OR', 'ORDER', 'OUT', 'OUTER',
'OUTPUTFORMAT', 'OVER', 'OVERLAPS', 'OVERLAY', 'OVERWRITE', 'PARTITION',
'PARTITIONED', 'PARTITIONS', 'PERCENT', 'PIVOT', 'PLACING', 'POSITION',
'PRECEDING', 'PRIMARY', 'PRINCIPALS', 'PROPERTIES', 'PURGE', 'QUERY', 'RANGE',
'RECORDREADER', 'RECORDWRITER', 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH',
'RENAME', 'REPAIR', 'REPLACE', 'RESET', 'RESTRICT', 'REVOKE', 'RIGHT', RLIKE,
'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP', 'ROW', 'ROWS', 'SCHEMA', 'SECOND',
'SELECT', 'SEMI', 'SEPARATED', 'SERDE', 'SERDEPROPERTIES', 'SESSION_USER',
'SET', 'MINUS', 'SETS', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START',
'STATISTICS', 'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'TABLE',
'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES', TEMPORARY, 'TERMINATED', 'THEN',
'TO', 'TOUCH', 'TRAILING', 'TRANSACTION', 'TRANSACTIONS', 'TRANSFORM', 'TRIM',
'TRUE', 'TRUNCATE', 'TYPE', 'UNARCHIVE', 'UNBOUNDED', 'UNCACHE', 'UNION',
'UNIQUE', 'UNKNOWN', 'UNLOCK', 'UNSET', 'UPDATE', 'USE', 'USER', 'USING',
'VALUES', 'VIEW', 'VIEWS', 'WHEN', 'WHERE', 'WINDOW', 'WITH', 'YEAR', 'DIV',
IDENTIFIER, BACKQUOTED_IDENTIFIER}(line 1, pos 59)
== SQL ==
SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1')
---^^^
```
Expected : - If that partition value is string we can give like this grade
='abc'
was:
In the link
[https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html] the
below command example mentioned is wrong.
SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1');
CREATE TABLE employee(name STRING)PARTITIONED BY (grade int) stored as parquet;
INSERT INTO employee PARTITION (grade = 1) VALUES ('sam');
INSERT INTO employee PARTITION (grade = 2) VALUES ('suj');
spark-sql> SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1');
**Error in query:**
```
mismatched input ''grade=1'' expecting \{'ADD', 'AFTER', 'ALL', 'ALTER',
'ANALYZE', 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT',
'AUTHORIZATION', 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE',
'CASCADE', 'CASE', 'CAST', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER', 'CLUSTERED',
'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT',
'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE', 'CONSTRAINT', 'COST',
'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE', 'CURRENT_TIME',
'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DATA', 'DATABASE', DATABASES, 'DAY',
'DBPROPERTIES', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC', 'DESCRIBE', 'DFS',
'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE', 'DROP',
[jira] [Created] (SPARK-32887) Example command in https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html to be changed
Chetan Bhat created SPARK-32887:
---
Summary: Example command in
https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html to be
changed
Key: SPARK-32887
URL: https://issues.apache.org/jira/browse/SPARK-32887
Project: Spark
Issue Type: Bug
Components: docs
Affects Versions: 3.0.0
Environment: Spark 2.4.5, Spark 3.0.0
Reporter: Chetan Bhat
In the link
[https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-table.html] the
below command example mentioned is wrong.
SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1');
CREATE TABLE employee(name STRING)PARTITIONED BY (grade int) stored as parquet;
INSERT INTO employee PARTITION (grade = 1) VALUES ('sam');
INSERT INTO employee PARTITION (grade = 2) VALUES ('suj');
spark-sql> SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1');
**Error in query:**
```
mismatched input ''grade=1'' expecting \{'ADD', 'AFTER', 'ALL', 'ALTER',
'ANALYZE', 'AND', 'ANTI', 'ANY', 'ARCHIVE', 'ARRAY', 'AS', 'ASC', 'AT',
'AUTHORIZATION', 'BETWEEN', 'BOTH', 'BUCKET', 'BUCKETS', 'BY', 'CACHE',
'CASCADE', 'CASE', 'CAST', 'CHANGE', 'CHECK', 'CLEAR', 'CLUSTER', 'CLUSTERED',
'CODEGEN', 'COLLATE', 'COLLECTION', 'COLUMN', 'COLUMNS', 'COMMENT', 'COMMIT',
'COMPACT', 'COMPACTIONS', 'COMPUTE', 'CONCATENATE', 'CONSTRAINT', 'COST',
'CREATE', 'CROSS', 'CUBE', 'CURRENT', 'CURRENT_DATE', 'CURRENT_TIME',
'CURRENT_TIMESTAMP', 'CURRENT_USER', 'DATA', 'DATABASE', DATABASES, 'DAY',
'DBPROPERTIES', 'DEFINED', 'DELETE', 'DELIMITED', 'DESC', 'DESCRIBE', 'DFS',
'DIRECTORIES', 'DIRECTORY', 'DISTINCT', 'DISTRIBUTE', 'DROP', 'ELSE', 'END',
'ESCAPE', 'ESCAPED', 'EXCEPT', 'EXCHANGE', 'EXISTS', 'EXPLAIN', 'EXPORT',
'EXTENDED', 'EXTERNAL', 'EXTRACT', 'FALSE', 'FETCH', 'FIELDS', 'FILTER',
'FILEFORMAT', 'FIRST', 'FOLLOWING', 'FOR', 'FOREIGN', 'FORMAT', 'FORMATTED',
'FROM', 'FULL', 'FUNCTION', 'FUNCTIONS', 'GLOBAL', 'GRANT', 'GROUP',
'GROUPING', 'HAVING', 'HOUR', 'IF', 'IGNORE', 'IMPORT', 'IN', 'INDEX',
'INDEXES', 'INNER', 'INPATH', 'INPUTFORMAT', 'INSERT', 'INTERSECT', 'INTERVAL',
'INTO', 'IS', 'ITEMS', 'JOIN', 'KEYS', 'LAST', 'LATERAL', 'LAZY', 'LEADING',
'LEFT', 'LIKE', 'LIMIT', 'LINES', 'LIST', 'LOAD', 'LOCAL', 'LOCATION', 'LOCK',
'LOCKS', 'LOGICAL', 'MACRO', 'MAP', 'MATCHED', 'MERGE', 'MINUTE', 'MONTH',
'MSCK', 'NAMESPACE', 'NAMESPACES', 'NATURAL', 'NO', NOT, 'NULL', 'NULLS', 'OF',
'ON', 'ONLY', 'OPTION', 'OPTIONS', 'OR', 'ORDER', 'OUT', 'OUTER',
'OUTPUTFORMAT', 'OVER', 'OVERLAPS', 'OVERLAY', 'OVERWRITE', 'PARTITION',
'PARTITIONED', 'PARTITIONS', 'PERCENT', 'PIVOT', 'PLACING', 'POSITION',
'PRECEDING', 'PRIMARY', 'PRINCIPALS', 'PROPERTIES', 'PURGE', 'QUERY', 'RANGE',
'RECORDREADER', 'RECORDWRITER', 'RECOVER', 'REDUCE', 'REFERENCES', 'REFRESH',
'RENAME', 'REPAIR', 'REPLACE', 'RESET', 'RESTRICT', 'REVOKE', 'RIGHT', RLIKE,
'ROLE', 'ROLES', 'ROLLBACK', 'ROLLUP', 'ROW', 'ROWS', 'SCHEMA', 'SECOND',
'SELECT', 'SEMI', 'SEPARATED', 'SERDE', 'SERDEPROPERTIES', 'SESSION_USER',
'SET', 'MINUS', 'SETS', 'SHOW', 'SKEWED', 'SOME', 'SORT', 'SORTED', 'START',
'STATISTICS', 'STORED', 'STRATIFY', 'STRUCT', 'SUBSTR', 'SUBSTRING', 'TABLE',
'TABLES', 'TABLESAMPLE', 'TBLPROPERTIES', TEMPORARY, 'TERMINATED', 'THEN',
'TO', 'TOUCH', 'TRAILING', 'TRANSACTION', 'TRANSACTIONS', 'TRANSFORM', 'TRIM',
'TRUE', 'TRUNCATE', 'TYPE', 'UNARCHIVE', 'UNBOUNDED', 'UNCACHE', 'UNION',
'UNIQUE', 'UNKNOWN', 'UNLOCK', 'UNSET', 'UPDATE', 'USE', 'USER', 'USING',
'VALUES', 'VIEW', 'VIEWS', 'WHEN', 'WHERE', 'WINDOW', 'WITH', 'YEAR', 'DIV',
IDENTIFIER, BACKQUOTED_IDENTIFIER}(line 1, pos 59)
== SQL ==
SHOW TABLE EXTENDED IN default LIKE 'employee' PARTITION ('grade=1')
---^^^
```
Expected : - If that partition value is string we can give like this grade
='abc'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-32700) select from table TABLESAMPLE gives wrong resultset.
Chetan Bhat created SPARK-32700: --- Summary: select from table TABLESAMPLE gives wrong resultset. Key: SPARK-32700 URL: https://issues.apache.org/jira/browse/SPARK-32700 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.0.0 Environment: Spark 3.0.0 Reporter: Chetan Bhat create table test(id int,name string) stored as parquet; insert into test values (5,'Alex'),(8,'Lucy'),(2,'Mary'),(4,'Fred'),(1,'Lisa'),(9,'Eric'),(10,'Adam'),(6,'Mark'),(7,'Lily'),(3,'Evan'); SELECT * FROM test TABLESAMPLE (50 PERCENT); --> output is giving only 3 rows. spark-sql> SELECT * FROM test TABLESAMPLE (50 PERCENT); 5 Alex 10 Adam 4 Fred Expected as per the link is 5 rows -->[https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-sampling.html] Also the bucket parameter for select from table TABLESAMPLE gives wrong resultset. spark-sql> SELECT * FROM test TABLESAMPLE (BUCKET 4 OUT OF 10); 5 Alex 8 Lucy 9 Eric 1 Lisa 3 Evan Expected is 4 records. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-32697) Direct Date and timestamp format data insertion fails
Chetan Bhat created SPARK-32697: --- Summary: Direct Date and timestamp format data insertion fails Key: SPARK-32697 URL: https://issues.apache.org/jira/browse/SPARK-32697 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 3.0.0 Environment: Spark 3.0.0 Reporter: Chetan Bhat Direct Date and timestamp format data tried to be inserted but the data insertion fails as shown below. spark-sql> create table test(no timestamp) stored as parquet; Time taken: 0.561 seconds spark-sql> insert into test select '1979-04-27 00:00:00'; Error in query: Cannot write incompatible data to table '`default`.`test`': - Cannot safely cast 'no': string to timestamp; -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-22859) Permission of created table and database folder are not correct
Chetan Bhat created SPARK-22859:
---
Summary: Permission of created table and database folder are not
correct
Key: SPARK-22859
URL: https://issues.apache.org/jira/browse/SPARK-22859
Project: Spark
Issue Type: Bug
Components: Spark Submit
Affects Versions: 2.1.0
Reporter: Chetan Bhat
Steps :
In spark Beeline user executes the following queries.
drop database if exists test_db1 cascade;
create database test_db1 location 'hdfs://hacluster/user/test1';
use test_db1;
create table if not exists ORDERS(O_ORDERDATE string,O_ORDERPRIORITY
string,O_ORDERSTATUS string,O_ORDERKEY string,O_CUSTKEY string,O_TOTALPRICE
double,O_CLERK string,O_SHIPPRIORITY int,O_COMMENT string) STORED BY
'org.apache.carbondata.format'TBLPROPERTIES ('table_blocksize'='128');
User checks the permission of the created database and table in carbon store
using the bin/hadoop fs -getfacl command.
Issue : The Permission of created table and database folder are not correct. i.e
file: /user/test1/orders
owner: anonymous
group: users
user::rwx
group::r-x
other::r-x
Expected : Correct permissions for the created table and database folder in
carbon store should be
file: /user/test1/orders
owner: anonymous
group: users
user::rwx
group::---
other::---
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Reopened] (SPARK-22841) Select regexp_extract from table with where clause having is null throws indexoutofbounds exception
[
https://issues.apache.org/jira/browse/SPARK-22841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Bhat reopened SPARK-22841:
-
Exception should not be thrown in any scenario. Instead proper error message
can be displayed.
> Select regexp_extract from table with where clause having is null throws
> indexoutofbounds exception
> ---
>
> Key: SPARK-22841
> URL: https://issues.apache.org/jira/browse/SPARK-22841
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 2.1.0
>Reporter: Chetan Bhat
>
> Steps :
> Thrift server is started using the command - bin/spark-submit --master
> yarn-client --executor-memory 10G --executor-cores 5 --driver-memory 5G
> --num-executors 3 --class
> org.apache.carbondata.spark.thriftserver.CarbonThriftServer
> /srv/spark2.2Bigdata/install/spark/sparkJdbc/carbonlib/carbondata_2.11-1.3.0-SNAPSHOT-shade-hadoop2.7.2.jar
> "hdfs://hacluster/user/sparkhive/warehouse"
> Spark shell is launched using the command - bin/spark-shell --master
> yarn-client --executor-memory 10G --executor-cores 5 --driver-memory 5G
> --num-executors 3 --jars
> /srv/spark2.2Bigdata/install/spark/sparkJdbc/carbonlib/carbondata_2.11-1.3.0-SNAPSHOT-shade-hadoop2.7.2.jar
> From Spark shell the streaming table is created and data is loaded to the
> streaming table.
> import java.io.
> {File, PrintWriter}
> import java.net.ServerSocket
> import org.apache.spark.sql.
> {CarbonEnv, SparkSession}
> import org.apache.spark.sql.hive.CarbonRelation
> import org.apache.spark.sql.streaming.
> {ProcessingTime, StreamingQuery}
> import org.apache.carbondata.core.constants.CarbonCommonConstants
> import org.apache.carbondata.core.util.CarbonProperties
> import org.apache.carbondata.core.util.path.
> {CarbonStorePath, CarbonTablePath}
> CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT,
> "/MM/dd")
> import org.apache.spark.sql.CarbonSession._
> val carbonSession = SparkSession.
> builder().
> appName("StreamExample").
> getOrCreateCarbonSession("hdfs://hacluster/user/hive/warehouse/carbon.store")
> carbonSession.sparkContext.setLogLevel("INFO")
> def sql(sql: String) = carbonSession.sql(sql)
> def writeSocket(serverSocket: ServerSocket): Thread = {
> val thread = new Thread() {
> override def run(): Unit = {
> // wait for client to connection request and accept
> val clientSocket = serverSocket.accept()
> val socketWriter = new PrintWriter(clientSocket.getOutputStream())
> var index = 0
> for (_ <- 1 to 1000) {
> // write 5 records per iteration
> for (_ <- 0 to 100)
> { index = index + 1 socketWriter.println(index.toString + ",name_" + index +
> ",city_" + index + "," + (index * 1.00).toString + ",school_" + index +
> ":school_" + index + index + "$" + index) }
> socketWriter.flush()
> Thread.sleep(2000)
> }
> socketWriter.close()
> System.out.println("Socket closed")
> }
> }
> thread.start()
> thread
> }
> def startStreaming(spark: SparkSession, tablePath: CarbonTablePath,
> tableName: String, port: Int): Thread = {
> val thread = new Thread() {
> override def run(): Unit = {
> var qry: StreamingQuery = null
> try
> { val readSocketDF = spark.readStream .format("socket") .option("host",
> "10.18.98.34") .option("port", port) .load() qry = readSocketDF.writeStream
> .format("carbondata") .trigger(ProcessingTime("5 seconds"))
> .option("checkpointLocation", tablePath.getStreamingCheckpointDir)
> .option("tablePath", tablePath.getPath).option("tableName", tableName)
> .start() qry.awaitTermination() }
> catch
> { case ex: Throwable => ex.printStackTrace() println("Done reading and
> writing streaming data") }
> finally
> { qry.stop() }
> }
> }
> thread.start()
> thread
> }
> val streamTableName = "uniqdata"
> sql(s"CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
> decimal(36,36),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
> int) STORED BY 'org.apache.carbondata.format'
> TBLPROPERTIES('streaming'='true')")
> sql(s"LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table
> uniqdata OPTIONS(
> 'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')")
> val carbonTable = CarbonEnv.getInstance(carbonSession).carbonMetastore.
> lookupRelation(Some("default"),
> streamTableName)(carbonSession).asInstanceOf[CarbonRelation].carbonTable
> val tablePath =
> CarbonStorePath.getCarbonTablePath(carbonTable.getAbsoluteTableIdentifier)
> val port = 8006
> val serverSocket = new
[jira] [Created] (SPARK-22841) Select regexp_extract from table with where clause having is null throws indexoutofbounds exception
Chetan Bhat created SPARK-22841:
---
Summary: Select regexp_extract from table with where clause having
is null throws indexoutofbounds exception
Key: SPARK-22841
URL: https://issues.apache.org/jira/browse/SPARK-22841
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.1.0
Reporter: Chetan Bhat
Steps :
Thrift server is started using the command - bin/spark-submit --master
yarn-client --executor-memory 10G --executor-cores 5 --driver-memory 5G
--num-executors 3 --class
org.apache.carbondata.spark.thriftserver.CarbonThriftServer
/srv/spark2.2Bigdata/install/spark/sparkJdbc/carbonlib/carbondata_2.11-1.3.0-SNAPSHOT-shade-hadoop2.7.2.jar
"hdfs://hacluster/user/sparkhive/warehouse"
Spark shell is launched using the command - bin/spark-shell --master
yarn-client --executor-memory 10G --executor-cores 5 --driver-memory 5G
--num-executors 3 --jars
/srv/spark2.2Bigdata/install/spark/sparkJdbc/carbonlib/carbondata_2.11-1.3.0-SNAPSHOT-shade-hadoop2.7.2.jar
>From Spark shell the streaming table is created and data is loaded to the
>streaming table.
import java.io.
{File, PrintWriter}
import java.net.ServerSocket
import org.apache.spark.sql.
{CarbonEnv, SparkSession}
import org.apache.spark.sql.hive.CarbonRelation
import org.apache.spark.sql.streaming.
{ProcessingTime, StreamingQuery}
import org.apache.carbondata.core.constants.CarbonCommonConstants
import org.apache.carbondata.core.util.CarbonProperties
import org.apache.carbondata.core.util.path.
{CarbonStorePath, CarbonTablePath}
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT,
"/MM/dd")
import org.apache.spark.sql.CarbonSession._
val carbonSession = SparkSession.
builder().
appName("StreamExample").
getOrCreateCarbonSession("hdfs://hacluster/user/hive/warehouse/carbon.store")
carbonSession.sparkContext.setLogLevel("INFO")
def sql(sql: String) = carbonSession.sql(sql)
def writeSocket(serverSocket: ServerSocket): Thread = {
val thread = new Thread() {
override def run(): Unit = {
// wait for client to connection request and accept
val clientSocket = serverSocket.accept()
val socketWriter = new PrintWriter(clientSocket.getOutputStream())
var index = 0
for (_ <- 1 to 1000) {
// write 5 records per iteration
for (_ <- 0 to 100)
{ index = index + 1 socketWriter.println(index.toString + ",name_" + index +
",city_" + index + "," + (index * 1.00).toString + ",school_" + index +
":school_" + index + index + "$" + index) }
socketWriter.flush()
Thread.sleep(2000)
}
socketWriter.close()
System.out.println("Socket closed")
}
}
thread.start()
thread
}
def startStreaming(spark: SparkSession, tablePath: CarbonTablePath, tableName:
String, port: Int): Thread = {
val thread = new Thread() {
override def run(): Unit = {
var qry: StreamingQuery = null
try
{ val readSocketDF = spark.readStream .format("socket") .option("host",
"10.18.98.34") .option("port", port) .load() qry = readSocketDF.writeStream
.format("carbondata") .trigger(ProcessingTime("5 seconds"))
.option("checkpointLocation", tablePath.getStreamingCheckpointDir)
.option("tablePath", tablePath.getPath).option("tableName", tableName) .start()
qry.awaitTermination() }
catch
{ case ex: Throwable => ex.printStackTrace() println("Done reading and writing
streaming data") }
finally
{ qry.stop() }
}
}
thread.start()
thread
}
val streamTableName = "uniqdata"
sql(s"CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION
string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2
decimal(36,36),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1
int) STORED BY 'org.apache.carbondata.format'
TBLPROPERTIES('streaming'='true')")
sql(s"LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table
uniqdata OPTIONS(
'BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1')")
val carbonTable = CarbonEnv.getInstance(carbonSession).carbonMetastore.
lookupRelation(Some("default"),
streamTableName)(carbonSession).asInstanceOf[CarbonRelation].carbonTable
val tablePath =
CarbonStorePath.getCarbonTablePath(carbonTable.getAbsoluteTableIdentifier)
val port = 8006
val serverSocket = new ServerSocket(port)
val socketThread = writeSocket(serverSocket)
val streamingThread = startStreaming(carbonSession, tablePath, streamTableName,
port)
>From Beeline user executes the query
select regexp_extract(CUST_NAME,'a',1)from uniqdata where
regexp_extract(CUST_NAME,'a',1) IS NULL or regexp_extract(DOB,'b',2) is NULL;
Issue : Select regexp_extract from table with where clause having is null
throws indexoutofbounds exception
0: jdbc:hive2://10.18.98.34:23040> select
[jira] [Updated] (SPARK-14197) Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#333
[ https://issues.apache.org/jira/browse/SPARK-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-14197: Fix Version/s: (was: 1.5.1) > Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid > call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = > 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) > -- > > Key: SPARK-14197 > URL: https://issues.apache.org/jira/browse/SPARK-14197 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 >Reporter: Chetan Bhat >Priority: Minor > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > In Beeline when the SQL is executed using Spark the following error occurs. > select count(*) from babu1 where imei in ( select if (imei = > '1AA10007',imei,NULL) from babu1); > Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid > call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = > 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-14197) Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#333
[ https://issues.apache.org/jira/browse/SPARK-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-14197: Affects Version/s: 1.5.1 > Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid > call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = > 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) > -- > > Key: SPARK-14197 > URL: https://issues.apache.org/jira/browse/SPARK-14197 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 >Reporter: Chetan Bhat >Priority: Minor > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > In Beeline when the SQL is executed using Spark the following error occurs. > select count(*) from babu1 where imei in ( select if (imei = > '1AA10007',imei,NULL) from babu1); > Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid > call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = > 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14199) Error: org.apache.spark.sql.hive.huawei.rules.analyze.UnrelatedSubqueryConversion$MultiValueException$: subquery expression must return single column! (state=,code=0)
Chetan Bhat created SPARK-14199: --- Summary: Error: org.apache.spark.sql.hive.huawei.rules.analyze.UnrelatedSubqueryConversion$MultiValueException$: subquery expression must return single column! (state=,code=0) Key: SPARK-14199 URL: https://issues.apache.org/jira/browse/SPARK-14199 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.1 Reporter: Chetan Bhat Priority: Minor In Beeline when the SQL is executed using Spark error is displayed. select imei,a, (select MAC from babu8 a where a.imei = '1AA10007' ) from ( select imei,count(contractNumber)+1 a from testolap group by imei) a where imei in ( select imei from testolap where imei like '%007%') group by imei, a; Error: org.apache.spark.sql.hive.huawei.rules.analyze.UnrelatedSubqueryConversion$MultiValueException$: subquery expression must return single column! (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14198) Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '' '' '' in subquery source; line 1 pos 86 (state=,code=0)
Chetan Bhat created SPARK-14198: --- Summary: Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '' '' '' in subquery source; line 1 pos 86 (state=,code=0) Key: SPARK-14198 URL: https://issues.apache.org/jira/browse/SPARK-14198 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.1 Reporter: Chetan Bhat Priority: Minor In Beeline when the SQL is executed using Spark the error occurs. select count(*) from ( select case when imei like '%007%' then imei end from babu8); Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '' '' '' in subquery source; line 1 pos 86 (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-14196) Error: org.apache.spark.sql.AnalysisException: cannot resolve 'query_alias_fix_conflicts_0._c0'; (state=,code=0)
[ https://issues.apache.org/jira/browse/SPARK-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-14196: Component/s: SQL > Error: org.apache.spark.sql.AnalysisException: cannot resolve > 'query_alias_fix_conflicts_0._c0'; (state=,code=0) > > > Key: SPARK-14196 > URL: https://issues.apache.org/jira/browse/SPARK-14196 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 >Reporter: Chetan Bhat >Priority: Minor > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > In Beeline when the SQL is executed using Spark the following error is > displayed. > select count(*) from testolap where imei in ( select case when imei like > '%007%' then imei end from testolap); > Error: org.apache.spark.sql.AnalysisException: cannot resolve > 'query_alias_fix_conflicts_0._c0'; (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14197) Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#333
Chetan Bhat created SPARK-14197: --- Summary: Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) Key: SPARK-14197 URL: https://issues.apache.org/jira/browse/SPARK-14197 Project: Spark Issue Type: Bug Components: SQL Reporter: Chetan Bhat Priority: Minor Fix For: 1.5.1 In Beeline when the SQL is executed using Spark the following error occurs. select count(*) from babu1 where imei in ( select if (imei = '1AA10007',imei,NULL) from babu1); Error: org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to name on unresolved object, tree: unresolvedalias(if ((imei#33365 = 1AA10007)) imei#33365 else cast(null as string)) (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14196) Error: org.apache.spark.sql.AnalysisException: cannot resolve 'query_alias_fix_conflicts_0._c0'; (state=,code=0)
Chetan Bhat created SPARK-14196: --- Summary: Error: org.apache.spark.sql.AnalysisException: cannot resolve 'query_alias_fix_conflicts_0._c0'; (state=,code=0) Key: SPARK-14196 URL: https://issues.apache.org/jira/browse/SPARK-14196 Project: Spark Issue Type: Bug Affects Versions: 1.5.1 Reporter: Chetan Bhat Priority: Minor In Beeline when the SQL is executed using Spark the following error is displayed. select count(*) from testolap where imei in ( select case when imei like '%007%' then imei end from testolap); Error: org.apache.spark.sql.AnalysisException: cannot resolve 'query_alias_fix_conflicts_0._c0'; (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-14195) Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'select' 'MAC' 'from' in expression specification; line 1 pos 16 (state=,code=0)
[ https://issues.apache.org/jira/browse/SPARK-14195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Bhat updated SPARK-14195: Environment: SUSE 11 Remaining Estimate: 0.05h Original Estimate: 0.05h Description: In Beeline when the SQL is executed using Spark the following error is displayed. select a.imei, (select MAC from testolap where imei like '%AA%' 1) MAC from testolap a limit 10; Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'select' 'MAC' 'from' in expression specification; line 1 pos 16 (state=,code=0) > Error: org.apache.spark.sql.AnalysisException: cannot recognize input near > 'select' 'MAC' 'from' in expression specification; line 1 pos 16 > (state=,code=0) > --- > > Key: SPARK-14195 > URL: https://issues.apache.org/jira/browse/SPARK-14195 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 1.5.1 > Environment: SUSE 11 >Reporter: Chetan Bhat >Priority: Minor > Original Estimate: 0.05h > Remaining Estimate: 0.05h > > In Beeline when the SQL is executed using Spark the following error is > displayed. > select a.imei, (select MAC from testolap where imei like '%AA%' 1) MAC from > testolap a limit 10; > Error: org.apache.spark.sql.AnalysisException: cannot recognize input near > 'select' 'MAC' 'from' in expression specification; line 1 pos 16 > (state=,code=0) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-14195) Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'select' 'MAC' 'from' in expression specification; line 1 pos 16 (state=,code=0)
Chetan Bhat created SPARK-14195: --- Summary: Error: org.apache.spark.sql.AnalysisException: cannot recognize input near 'select' 'MAC' 'from' in expression specification; line 1 pos 16 (state=,code=0) Key: SPARK-14195 URL: https://issues.apache.org/jira/browse/SPARK-14195 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.1 Reporter: Chetan Bhat Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
