Re: How to test new datasource without "in memory/mock" implementation
I agree with Julian. Not everybody has the option to run docker locally. Perhaps execute tests against embedded (or fake) instance as well as a real one (docker or externally managed) ? Like it is done in MongoDatabasePolicy.java [1] Currently tests for Elasticsearch, Geode and Mongo are all run against embedded test instances but one should be able to execute them against a real one as well. https://github.com/apache/calcite/blob/master/mongodb/src/test/java/org/apache/calcite/adapter/mongodb/MongoDatabasePolicy.java#L70 [1] On Sun, Oct 18, 2020 at 3:30 AM Tugdual Grall wrote: > Ok > > I will look more into it > > On Sun, Oct 18, 2020 at 8:54 AM Julian Hyde > wrote: > > > This would be inconvenient for me personally. My company does not allow > me > > to run Docker on my work computer. > > > > Julian > > > > > On Oct 17, 2020, at 11:31 PM, Tugdual Grall wrote: > > > > > > Hi > > > > > > I have created this PR to share with the community what could be done: > > > * https://github.com/apache/calcite/pull/2223 > > > > > > And associated issue > https://issues.apache.org/jira/browse/CALCITE-4344 > > > > > > I guess we will continue the discussion on the issue itself > > > > > > Tug > > > > > >> On Sat, Oct 17, 2020 at 11:05 PM Vladimir Sitnikov < > > >> sitnikov.vladi...@gmail.com> wrote: > > >> > > >> Tugdual>Are we "allowed" to use containers in the tests, for example > > >> using TestContainers > > >> > > >> I believe Testcontainers-based tests are more than welcome, however, > > >> nobody implemented that yet. > > >> It would be great if you could contribute that. > > >> > > >> Vladimir > > >> > > >> > > >
Re: How to test new datasource without "in memory/mock" implementation
Ok I will look more into it On Sun, Oct 18, 2020 at 8:54 AM Julian Hyde wrote: > This would be inconvenient for me personally. My company does not allow me > to run Docker on my work computer. > > Julian > > > On Oct 17, 2020, at 11:31 PM, Tugdual Grall wrote: > > > > Hi > > > > I have created this PR to share with the community what could be done: > > * https://github.com/apache/calcite/pull/2223 > > > > And associated issue https://issues.apache.org/jira/browse/CALCITE-4344 > > > > I guess we will continue the discussion on the issue itself > > > > Tug > > > >> On Sat, Oct 17, 2020 at 11:05 PM Vladimir Sitnikov < > >> sitnikov.vladi...@gmail.com> wrote: > >> > >> Tugdual>Are we "allowed" to use containers in the tests, for example > >> using TestContainers > >> > >> I believe Testcontainers-based tests are more than welcome, however, > >> nobody implemented that yet. > >> It would be great if you could contribute that. > >> > >> Vladimir > >> > >> >
Re: How to test new datasource without "in memory/mock" implementation
This would be inconvenient for me personally. My company does not allow me to run Docker on my work computer. Julian > On Oct 17, 2020, at 11:31 PM, Tugdual Grall wrote: > > Hi > > I have created this PR to share with the community what could be done: > * https://github.com/apache/calcite/pull/2223 > > And associated issue https://issues.apache.org/jira/browse/CALCITE-4344 > > I guess we will continue the discussion on the issue itself > > Tug > >> On Sat, Oct 17, 2020 at 11:05 PM Vladimir Sitnikov < >> sitnikov.vladi...@gmail.com> wrote: >> >> Tugdual>Are we "allowed" to use containers in the tests, for example >> using TestContainers >> >> I believe Testcontainers-based tests are more than welcome, however, >> nobody implemented that yet. >> It would be great if you could contribute that. >> >> Vladimir >> >>
Re: How to test new datasource without "in memory/mock" implementation
Hi I have created this PR to share with the community what could be done: * https://github.com/apache/calcite/pull/2223 And associated issue https://issues.apache.org/jira/browse/CALCITE-4344 I guess we will continue the discussion on the issue itself Tug On Sat, Oct 17, 2020 at 11:05 PM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > Tugdual>Are we "allowed" to use containers in the tests, for example > using TestContainers > > I believe Testcontainers-based tests are more than welcome, however, > nobody implemented that yet. > It would be great if you could contribute that. > > Vladimir > >
Re: How to test new datasource without "in memory/mock" implementation
Tugdual>Are we "allowed" to use containers in the tests, for example using TestContainers I believe Testcontainers-based tests are more than welcome, however, nobody implemented that yet. It would be great if you could contribute that. Vladimir
Re: How to test new datasource without "in memory/mock" implementation
We need a solution that everyone can use easily. Ideally, contributors to the rest of Calcite - say Calcite core - should not have to spend extra effort, or experience any extra delay or flakiness, in order to verify that they have not broken the Redis adapter. The nearest example we have today is the Druid adapter. Stamatis spent considerable time recently [1] to set up a container for Druid, document how to use it, and enable it in CI. Someone could do the same for Redis, preferably using the same container technology. Julian [1] https://issues.apache.org/jira/browse/CALCITE-4278 On Sat, Oct 17, 2020 at 12:46 AM Tugdual Grall wrote: > > Hello, > > I am new to the Calcite (at least trying to contribute - I did use it > indirectly a lot in Apache Drill ) > > I would like to enrich the Redis adapter (or create a new one) to support > RediSearch ( http://redisearch.io ) since it provides richer features such > as secondary indexing, querying and aggregations on data located in Redis. > > What is the best way to run tests? > I see that the current Redis tests are using a fake implementation, same > for Mongo... > > RediSearch does not have such an implementation (and same for other modules > Graph, JSON, ..) > > Are we "allowed" to use containers in the tests, for example using > TestContainers ( https://www.testcontainers.org/ ) ? > > If not can you point me to the proper way to run tests when a mock server > does not exist? > > Thanks > Tug
How to test new datasource without "in memory/mock" implementation
Hello, I am new to the Calcite (at least trying to contribute - I did use it indirectly a lot in Apache Drill ) I would like to enrich the Redis adapter (or create a new one) to support RediSearch ( http://redisearch.io ) since it provides richer features such as secondary indexing, querying and aggregations on data located in Redis. What is the best way to run tests? I see that the current Redis tests are using a fake implementation, same for Mongo... RediSearch does not have such an implementation (and same for other modules Graph, JSON, ..) Are we "allowed" to use containers in the tests, for example using TestContainers ( https://www.testcontainers.org/ ) ? If not can you point me to the proper way to run tests when a mock server does not exist? Thanks Tug
