Hi, thanks for the reply. Let’s imagine we have a parquet based table called parquet_table, now I want to insert it into a new JDBC table, all using pure SQL.
If the JDBC table already exists, it’s easy, we do CREATE TABLE USING JDBC and then we do INSERT INTO that table. If the table doesn’t exist, is there a way to create it using Spark SQL only? I don’t want to use DataFrames API, I know that I can use .write() for that, but I want to keep it in pure SQL, since that is more comprehendible for data analysts. > On 2 Feb 2023, at 02:08, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > > Hi, > > It is not very clear your statement below: > > ".. If the table existed, I would create a table using JDBC in spark SQL and > then insert into it, but I can't create a table if it doesn't exist in JDBC > database..." > > If the table exists in your JDBC database, why do you need to create it? > > How do you verify if it exists? Can you share the code and the doc link? > > HTH > > > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > https://en.everybodywiki.com/Mich_Talebzadeh > > > Disclaimer: Use it at your own risk. Any and all responsibility for any loss, > damage or destruction of data or any other property which may arise from > relying on this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such > loss, damage or destruction. > > > > On Wed, 1 Feb 2023 at 19:33, Harut Martirosyan <harut.martiros...@gmail.com > <mailto:harut.martiros...@gmail.com>> wrote: >> I have a resultset (defined in SQL), and I want to insert it into my JDBC >> database using only SQL, not dataframes API. >> >> If the table existed, I would create a table using JDBC in spark SQL and >> then insert into it, but I can't create a table if it doesn't exist in JDBC >> database. >> >> How to do that using pure SQL (no python/scala/java)? >> >> I am trying to use INSERT OVERWRITE DIRECTORY with JDBC file format >> (according to the documentation) but as expected this functionality is >> available only for File-based storage systems. >> >> -- >> RGRDZ Harut
