Hello again, regarding the embedded SQL I just saw that we are using HSQLDB for some of the tests for JDBC operators. See for example here: https://github.com/apache/incubator-wayang/blob/main/wayang-platforms/wayang-jdbc-template/src/test/java/org/apache/wayang/jdbc/test/HsqldbFilterOperator.java
So, Harry, I don't think you need to add any new functionality regarding tests for the JDBC sink. Best -- Zoi On 2025/10/21 20:24:27 "Gavriilidis, Haralampos" wrote: > And thanks also to Kaustubh and Zoi for their input, indeed a generic JDBC > sink would be something I can definitely look into while I am at it 😉 > > > ________________________________ > From: Gavriilidis, Haralampos > Sent: Tuesday, October 21, 2025 10:21 PM > To: [email protected] > Subject: Re: PostgreSQL Sink > > > Hi Mirko, > > > yes, the embedded PostgreSQL idea is only for testing purposes, and not > related to the Sink operator. Most of the other platforms can be used as > libraries (e.g., Spark), which makes testing easy. However, for PostgreSQL > the tests require a separate instance running. Actually, there seem to be > some PostgreSQL-related tests, but they are not run during the build, are > they? > > > In this case embedding would mean that we use one of the mentioned projects, > to include PostgreSQL as a Maven dependency, only for the tests. Then we > could also run PostgreSQL tests during the build etc. Not sure if these > embedded PostgreSQL projects work well, but would be willing to try it out. > Or is there an other standard way of testing the PostgreSQL platform within > Wayang. > > > The PostgreSQL Sink operator is a separate thing, that I want to bring from > an older project. > > > Best, > > Harry > > ________________________________ > From: Mirko Kämpf <[email protected]> > Sent: Tuesday, October 21, 2025 9:00:51 PM > To: [email protected] > Subject: Re: PostgreSQL Sink > > Hello Wayang fans ;-) > Hello Harry, > > this embedded Postgres approach sounds interesting. It is also used by a > project called "pixeltable" I have tested in the past. > For testing and also for "one-shot processing" this could be reasonable. > > I do not yet get my head around the "embedded PostgresDB" for the specific > Wayang Operator - but for testing, this could be beneficial. > > For me, embedding means, we bundle a part we need and which is not easily > available - in case of Wayang Testing, I can envision an embedded > PostrgesDB > per Operator, so that we can verify the flow into and from the embedded DB. > Is this what you are looking for? > > Cheers, > Mirko > > > > Am Di., 21. Okt. 2025 um 18:39 Uhr schrieb Gavriilidis, Haralampos > <[email protected]>: > > > Dear all, > > > > > > I am currently preparing a PostgreSQL sink operator for Wayang. Is there > > already a ticket for this, or should I create one? > > > > While working on it, I recalled that testing PostgreSQL-specific > > functionality requires a separately managed PostgreSQL instance. For that > > reason, I am currently exploring embedded PostgreSQL options such as > > https://github.com/zonkyio/embedded-postgres and > > https://github.com/opentable/otj-pg-embedded to simplify and enhance the > > testing pipelines. Would this be something interesting for the project? > > > > > > Best, > > > > Harry > > > > > -- > Dr. Mirko Kämpf > *Gründer & Coach * > *maindset.ACADEMY* > > Müchelner Str. 23 > 06259 Frankleben >
