PS: we supported `where_condition` in jdbc. Please refer https://seatunnel.apache.org/docs/2.3.7/connector-v2/source/Jdbc
Jia Fan <[email protected]> 于2024年8月30日周五 14:49写道： > I have two ideas. > 1. Why not create a new virtual table type named `SQL`? > We can define `SQL` not only containing the `Where` clause, but also > can support the `Join` statement. > 2. Add a new option in the source step only in the SeaTunnel Web named > `Where` clause. > > Arshad, Mohammad <[email protected]> 于2024年8月30日周五 14:41写道： > >> >> https://github.com/apache/seatunnel-web/blob/30ed42fdac21a0edc6a9cb06ed00f81967b64717/seatunnel-server/seatunnel-app/src/main/java/org/apache/seatunnel/app/thirdparty/datasource/impl/BaseJdbcDataSourceConfigSwitcher.java#L217 >> Above generateSql method generates the query for all JDBC data sources. >> It does not support adding a WHERE clause because through UI there is not >> enough information collected to generate WHERE clause. >> Then, How does seatunnel-web support the WHERE clause for JDBC data >> sources? >> >> >> Thanks & Regards >> Mohammad Arshad >> >> >> From: Jia Fan <[email protected]> >> Date: Friday, 30 August 2024 at 8:00 AM >> To: [email protected] <[email protected]> >> Subject: Re: How to create a seatunnel job in seatunnel-web with >> conditional query >> Hi Mohammad: >> We can support default value in SeaTunnel Web just like we did in >> >> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fseatunnel.apache.org%2Fdocs%2Fconcept%2Fsink-options-placeholders%23placeholder&data=05%7C02%7Cmohammad.arshad%40visa.com%7C8d6c1c18066943c7320e08dcc89bb6eb%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C638605818343670075%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=B932PqEdwNObTnpy%2FufQS2Ao9Xkh%2BjuCzFktG6K4qko%3D&reserved=0 >> < >> https://seatunnel.apache.org/docs/concept/sink-options-placeholders#placeholder >> > >> eg: >> SELECT name, age FROM test.test_table WHERE age > ${ageParam:25}. Users >> should define the default value when creating a source. >> WDYT? >> >> Mohammad Arshad <[email protected]> 于2024年8月30日周五 00:22写道： >> >> > Thank you Jia Fan for your reply >> > >> > Yes, the variable substitution feature is available in SeaTunnel >> Engine. If >> > the job were created manually, there would be no issues. However, the >> > problem arises when a job is created through SeaTunnel Web. >> > >> > In SeaTunnel Web, how can I create a source step with a query like >> "SELECT >> > name, age FROM test.test_table WHERE age > ${ageParam}"? Do you think >> this >> > is possible through the SeaTunnel Web? >> > >> > Thanks & Regards >> > Mohammad Arshad >> > >> > >> > >> > On Thu, Aug 29, 2024 at 7:32 AM Jia Fan <[email protected]> wrote: >> > >> > > One way is to use our config variable feature in >> > > >> > > >> > >> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fseatunnel.apache.org%2Fdocs%2F2.3.7%2Fconcept%2Fconfig%23config-variable-substitution&data=05%7C02%7Cmohammad.arshad%40visa.com%7C8d6c1c18066943c7320e08dcc89bb6eb%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C638605818343681883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=uJsdtWVHs9MpBqANn85ogfg1Yiw6QtGMz1c8Z4qj2Xw%3D&reserved=0 >> < >> https://seatunnel.apache.org/docs/2.3.7/concept/config#config-variable-substitution >> > >> > > . >> > > >> > > Arshad, Mohammad <[email protected]> 于2024年8月28日周三 >> > 19:27写道： >> > > >> > > > Hi All, >> > > > I'm interested in creating a job that includes a conditional query, >> yet >> > > it >> > > > appears that this option is currently unavailable in SeaTunnel-Web. >> > > > To illustrate, suppose we have a test_table with fields name and >> age. >> > My >> > > > objective is to transfer all records from test_table to another >> table, >> > > but >> > > > only for those entries where age is greater than 25. >> > > > >> > > > At present, SeaTunnel-Web enables the creation of jobs with queries >> > such >> > > > as “SELECT `name`, `age` FROM `test`.`test_table`”. However, it does >> > not >> > > > provide an option for creating queries like >> > > > “SELECT `name`, `age` FROM `test`.`test_table` where `age` > 25” or >> > > > “SELECT `name`, `age` FROM `test`.`test_table` where `age` >$ >> > > {ageParam}” >> > > > >> > > > Could there be something I'm overlooking? Are there any alternative >> > > > methods to accomplish this? >> > > > >> > > > An attempt is made to add support for this feature at execution >> time as >> > > > part of the pull request at >> > > > >> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fseatunnel-web%2Fpull%2F193&data=05%7C02%7Cmohammad.arshad%40visa.com%7C8d6c1c18066943c7320e08dcc89bb6eb%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C638605818343689686%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=7PJFUDmyXQ6ob1ZW0FTWRMhe4huget2RHiGE5GwCmyw%3D&reserved=0 >> <https://github.com/apache/seatunnel-web/pull/193>. >> > > > However, it might be more efficient if the conditional query could >> be >> > > > supported directly at the job creation stage. Only some parameters >> to >> > be >> > > > changed at execution time, not the whole query. >> > > > >> > > > Does anyone have any ideas or suggestions on how to implement this >> > > feature? >> > > > >> > > > Thanks & Regards >> > > > Mohammad Arshad >> > > > >> > > > >> > > >> > >> >
