Re: [Spark SQL]: Is it possible that spark SQL appends "SELECT 1 " to the query

Weird, does Teradata not support LIMIT n? looking at the Spark source code suggests it won't. The syntax is "SELECT TOP"? I wonder if that's why the generic query that seems to test existence loses the LIMIT. But, that "SELECT 1" test seems to be used for MySQL, Postgres, so I'm still not sure

Re: [Spark SQL]: Is it possible that spark SQL appends "SELECT 1 " to the query

Hm, the existence queries even in 2.4.x had LIMIT 1. Are you sure nothing else is generating or changing those queries? On Thu, Nov 17, 2022 at 11:20 AM Ramakrishna Rayudu < ramakrishna560.ray...@gmail.com> wrote: > We are using spark 2.4.4 version. > I can see two types of queries in DB logs. >

[Spark SQL]: Is it possible that spark SQL appends "SELECT 1 " to the query

Hi Team, I am facing one issue. Can you please help me on this. 1. We are connecting Tera data from spark SQL with below API Dataset jdbcDF = spark.read().jdbc(connectionUrl, tableQuery,

Re: [Spark SQL]: Is it possible that spark SQL appends "SELECT 1 " to the query

This is a query to check the existence of the table upfront. It is nearly a no-op query; can it have a perf impact? On Thu, Nov 17, 2022 at 10:42 AM Ramakrishna Rayudu < ramakrishna560.ray...@gmail.com> wrote: > Hi Team, > > I am facing one issue. Can you please help me on this. > >

Re: [Spark SQL]: Is it possible that spark SQL appends "SELECT 1 " to the query

We are using spark 2.4.4 version. I can see two types of queries in DB logs. SELECT 1 FROM (INPUT_QUERY) SPARK_GEN_SUB_0 SELECT * FROM (INPUT_QUERY) SPARK_GEN_SUB_0 WHERE 1=0 When we see `SELECT *` which ending up with `Where 1=0` but query starts with `SELECT 1` there is no where condition.

Re: [Spark SQL]: Is it possible that spark SQL appends "SELECT 1 " to the query

Hm, actually that doesn't look like the queries that Spark uses to test existence, which will be "SELECT 1 ... LIMIT 1" or "SELECT * ... WHERE 1=0" depending on the dialect. What version, and are you sure something else is not sending those queries? On Thu, Nov 17, 2022 at 11:02 AM Ramakrishna