[Geotools-devel] Compiling geotools fails on jjtree-javacce missing plugin descriptor
Hello, when I run mvn clean compile on a fresh checkout of geotools, I get this error: [INFO] EPSG Authority Service using WKT file .. SUCCESS [ 0.093 s] [INFO] OGC CQL to Filter parser ... FAILURE [ 0.001 s] ... [ERROR] Failed to parse plugin descriptor for org.geotools.maven:jjtree-javacc:29-SNAPSHOT (/home/hans/external/geotools/build/maven/jjtree-javacc/target/classes): No plugin descriptor found at META-INF/maven/plugin.xml -> [Help 1] I've tried without succes rebuilding with: rm -rf ~/.m2/repository/ git checkout tags/28.0 Anyone knows what's wrong? I run maven 3.6.3 on JDK17 Hans Yperman Department IT [cid:47214f21-5865-4bcf-9e26-d197e372fb8a] Vlaams Instituut voor de Zee vzw InnovOcean Campus, Jacobsenstraat 1 8400 Oostende, België ☎+32 (0) 59 33 61 13 📧 hans.yper...@vliz.be<mailto:hans.yper...@vliz.be> www.vliz.be<http://www.vliz.be> ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] Release artefacts available for testing
Jody, On windows, classpath entries are separated by semicolon ';' but on all other OSes (mac, Linux, BSD) they are separated by colon ':' . This is the same behaviour as the PATH environment variable, presumably because windows uses colon for drive letters. There is a bug in recent java man pages that forgets to document this: https://bugs.openjdk.org/browse/JDK-8262004 So I assume this is normal. Hans Yperman Department IT [cid:b1a442ea-d6a5-4990-a69f-c354ac9c2c39] Vlaams Instituut voor de Zee vzw InnovOcean Campus, Jacobsenstraat 1 8400 Oostende, België ☎+32 (0) 59 33 61 13 📧 hans.yper...@vliz.be<mailto:hans.yper...@vliz.be> www.vliz.be<http://www.vliz.be> Van: Jody Garnett Verzonden: zondag 22 januari 2023 15:46 Aan: Ian Turton CC: Geotools-Devel list Onderwerp: Re: [Geotools-devel] Release artefacts available for testing Ian: Going to asume the issue is on my end; prior releases are behaving this way also. Managed to get it to startup using: java -cp "lib/*:bin" org.geotools.tutorial.quickstart.Quickstart Which does not make a lot of sense to me. -- Jody Garnett On Sun, Jan 22, 2023 at 3:26 PM Jody Garnett mailto:jody.garn...@gmail.com>> wrote: Testing bin download: 1) README.html links are good 2) Bin works java -cp "lib/*" org.geotools.util.factory.GeoTools 3) The quickstart compiles javac -cp "lib/*" -d bin src/org/geotools/tutorial/quickstart/Quickstart.java However it does not run java -cp "lib/*;bin" org.geotools.tutorial.quickstart.Quickstart Error: Could not find or load main class org.geotools.tutorial.quickstart.Quickstart Caused by: java.lang.ClassNotFoundException: org.geotools.tutorial.quickstart.Quickstart Changing to Java 8 again did not help ... -- Jody Garnett On Thu, Jan 19, 2023 at 11:15 AM Ian Turton mailto:ijtur...@gmail.com>> wrote: Please try out the new 28.1 release - https://build.geoserver.org/view/release/job/geotools-release/73/artifact/build/release/distribution/28.1/geotools-28.1-bin.zip Ian -- Ian Turton ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net<mailto:GeoTools-Devel@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/geotools-devel ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
Re: [Geotools-devel] geotools mongodb bugs in filtering
Dear Marco, I made a pull request for this issue https://github.com/geotools/geotools/pull/3965 . The code change is OK, but I made some mistakes while formatting the patch and ran out of time yesterday. If not acceptable, I’ll retry somewhere in the near future. * My test now runs on the real mongodb. * ECQL fixed all function bugs. * Fixing the bug surfaced 3 broken unit tests. I checked them, the behaviour encoded in them seems incorrect to me, so I adapted these too. * I did not fix the other minor troubles. From: Marco Volpini Sent: dinsdag 12 juli 2022 11:16 To: Hans Yperman Cc: geotools-devel@lists.sourceforge.net Subject: Re: [Geotools-devel] geotools mongodb bugs in filtering Dear Hans, Find listed below some replies to your questions: * Tests: when running online tests geotools makes usage of a property file with connection params to a mongodb instance that should be present in your a directory ".geotools" in your user directory. If the directory with the .properties is not present then geotools will create it along with an example .properties file to fill with proper connection parameters in order to run the tests. I would avoid then the change you proposed and try to run the tests with the properties file correctly configured. * You could even test the splitting logic avoiding online test: make the splitting class instead of anonymous a package default class and then execute test on the visit methods by using a FilterFactory<https://github.com/geotools/geotools/blob/f66689b5305eaf381e21dc2feec8011138d4cb24/modules/library/opengis/src/main/java/org/opengis/filter/FilterFactory2.java#L48> to create the various visatable filters that you want to test. * The JsonSelect function is supposed to be used for an integration of AppSchema with MongoDB doesn't have a real usage in normal CQL use cases. However the filter you provide to the compiler seems fine thus if you are receiving exception might be a bug. Try to use XCQL.toFilter or ECQL.toFilter methods and see if you reproduce it. The CQL compiler is more strict on the filters types and might be the reason of the compilation failure. * I would not delete the MongoFilterSplitter class. Although it's behaviour is faulty in the visiting of Binary ops and Like filters it is extended and overrided in GeoServer and its deletion will cause issues. Instead you might do you code changes to that one instead of to the anonymous class in the MongoFeatureSource and use it as a base to be extended for it. * Regarding the Temporal operator test with Before: I think it don't have much sense to test it in the splitting process as it is currently not supported by the plugin. The FilterToMongo class will indeed throw an exception if found. The currently supported time filters are in the form dateTimeProp >= '2021-06-01T09:42:47.000+02:00' using the comparison operators <>=. * Regarding the way to open pull requests: usually a pull request should have a single focus on a bug or a functionality. So if all of your modifications are for the faulty splitting logic I guess you can just open one pr. Regards, Marco Volpini == GeoServer Professional Services from the experts! Visit http://bit.ly/gs-services-us for more information. == Marco Volpini Software Engineer GeoSolutions Group phone: +39 0584 962313 fax: +39 0584 1660272 https://www.geosolutionsgroup.com/ http://twitter.com/geosolutions_it --- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail. On Fri, Jul 8, 2022 at 1:35 PM Hans Yperman mailto:hans.yper...@vliz.be>> wrote: Hi Marco, Thanks for your quick reaction, In this mail I’ll concentrate on the Filtering issue, as it is the most blocking for us. The schemaless plugin is interesting and I’ll check it out. Fixing the filtering bug causes the dates to partially work. So I’ll see what happens after the core bug is fixed. I’ve i
Re: [Geotools-devel] geotools mongodb bugs in filtering
Marco, Thanks for your answers. I’ll modify the code this evening * Tests: I’ll install mongo on my dev machine and have a try. * Tests without mongo: I need more than just the splitter, I also need the filter capabilities so I have to instantiate a MongoDataStore object without making a connection somehow. I can add an extra non-public constructor that grants access to the necessary fields if you prefer that. * I have classcastexceptions with every function I try, not only JSonSelect. I’ll try XCQL this evening. * I’ll not delete the MongoFiterSplitter class. Do I understand you that other modules import the Mongo driver module and write code on top of it? If yes, I’d better test them too. * The Before operation is there exactly because mongo does not support it. I need something that gets split of to force the splitter to do a split, and I can’t get functions to work, so there is not much left. From: Marco Volpini Sent: dinsdag 12 juli 2022 11:16 To: Hans Yperman Cc: geotools-devel@lists.sourceforge.net Subject: Re: [Geotools-devel] geotools mongodb bugs in filtering Dear Hans, Find listed below some replies to your questions: * Tests: when running online tests geotools makes usage of a property file with connection params to a mongodb instance that should be present in your a directory ".geotools" in your user directory. If the directory with the .properties is not present then geotools will create it along with an example .properties file to fill with proper connection parameters in order to run the tests. I would avoid then the change you proposed and try to run the tests with the properties file correctly configured. * You could even test the splitting logic avoiding online test: make the splitting class instead of anonymous a package default class and then execute test on the visit methods by using a FilterFactory<https://github.com/geotools/geotools/blob/f66689b5305eaf381e21dc2feec8011138d4cb24/modules/library/opengis/src/main/java/org/opengis/filter/FilterFactory2.java#L48> to create the various visatable filters that you want to test. * The JsonSelect function is supposed to be used for an integration of AppSchema with MongoDB doesn't have a real usage in normal CQL use cases. However the filter you provide to the compiler seems fine thus if you are receiving exception might be a bug. Try to use XCQL.toFilter or ECQL.toFilter methods and see if you reproduce it. The CQL compiler is more strict on the filters types and might be the reason of the compilation failure. * I would not delete the MongoFilterSplitter class. Although it's behaviour is faulty in the visiting of Binary ops and Like filters it is extended and overrided in GeoServer and its deletion will cause issues. Instead you might do you code changes to that one instead of to the anonymous class in the MongoFeatureSource and use it as a base to be extended for it. * Regarding the Temporal operator test with Before: I think it don't have much sense to test it in the splitting process as it is currently not supported by the plugin. The FilterToMongo class will indeed throw an exception if found. The currently supported time filters are in the form dateTimeProp >= '2021-06-01T09:42:47.000+02:00' using the comparison operators <>=. * Regarding the way to open pull requests: usually a pull request should have a single focus on a bug or a functionality. So if all of your modifications are for the faulty splitting logic I guess you can just open one pr. Regards, Marco Volpini == GeoServer Professional Services from the experts! Visit http://bit.ly/gs-services-us for more information. == Marco Volpini Software Engineer GeoSolutions Group phone: +39 0584 962313 fax: +39 0584 1660272 https://www.geosolutionsgroup.com/ http://twitter.com/geosolutions_it --- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-ma
Re: [Geotools-devel] geotools mongodb bugs in filtering
o their parent implementation. * But: What if e.g. there is a JSonSelectFunction but no Literal. The behaviour from super.visit…() needs to be verified for this function, it might not be as intended. But I need clarification on the 2nd bonus bug to write some unit tests that demonstrate correct behaviour before If I suppose optimistically that you agree with all the above and fill in the 2nd bonus bug, I can provide a PR. Do you want 1 PR containing everything, or would you prefer me to split it up in multiple PRs for ease of code review? Hans From: Marco Volpini Sent: vrijdag 8 juli 2022 9:14 To: Hans Yperman Cc: geotools-devel@lists.sourceforge.net Subject: Re: [Geotools-devel] geotools mongodb bugs in filtering Dear Hans, all improvements and bug fixes are welcome on both GeoTools and GeoServer. You can either provide the bug fix/improvement by yourself by opening a pull request on the GeoServer or Geotools repo, either by having someone else doing it for you (see the commercial support page<https://geoserver.org/support/>). In terms of help that can be provided to fix the issues that you have found, I can give you some code pointers (see below) and review the pull requests once they are opened. Please see my replies below to your points: ·MongoDB is schemaless, but it seems geoserver needs 2 schema definitions: 1 defining the attributes for layers, a 2nd specific for mongodb cached on disk and invisible if you don't know about its existence. If an attribute is added to mongodb, we'll need to delete/adapt the second, and restart the geoserver. This is due the fact that GeoTools and GeoServer are schema driven based on the gml standard that always needs a schema upfront in order to properly handle the data. Moreover the gt mongodb data store manages SimpleFeatures. Simple Feature specs doesn't allow nested objects and arrays to be encoded. The addition internal JSON schema is indeed also used to flattenize any nested structure to make it SimpleFeature compatible. However if you are interested in serving feature only as WFS GeoJSON and as WMS you might think to try the MongoDB schemaless plugin<https://docs.geoserver.org/latest/en/user/community/schemaless-features/schemaless-mongo/index.html> that instead will serve mongodb document as features as they are in the db, without flattening properties and without the need to provide a schema upfront. Mind that GML output will not be available in this case. ·The mongodb password needs to be hardcoded in the URL. This means geoserver can't hide/encrypt it. It is readable for everyone who can see the datastore. Yes currently the mongodb pwd can be only provided in the connection string input field in plain text, although GeoServer allows to externalize the connection string using environment properties<https://docs.geoserver.org/master/en/user/datadirectory/configtemplate.html>. Feel free to improve the mongodb connection string configuration of the MongoDB DataStore. Here some code pointers that you might find usefull: MongoDataStoreFactory<https://github.com/geotools/geotools/blob/1fed12f88ab0c9d4ef92aa9cf73af8f94d5c1291/modules/plugin/mongodb/src/main/java/org/geotools/data/mongodb/MongoDataStoreFactory.java#L30> and MongoDataStore<https://github.com/geotools/geotools/blob/1e036c73a4e420beeeb86c381c797a898a68c8a7/modules/plugin/mongodb/src/main/java/org/geotools/data/mongodb/MongoDataStore.java#L74>. ·I can't get Filtering to work. e.g. I add cql_filter=datatype%3D'C' which adds a filter on an attribute datatype='C'. Depending on the attribute, I get the whole mongodb collection or nothing. I debugged it to a function splitFilter that basically just throws my filter away. Issue GEOT-5911 has the same conclusions as I have and a partial fix, but it seems dead. This indeed is a bug. I've reproduced it with the mongo db data store (while on the mongodb schemaless plugin the same filter works fine). I gave just a quick look so I'm not 100% sure but to fix it I believe that at the end of this visit method in the PostPreFilter Splitter<https://github.com/geotools/geotools/blob/1e036c73a4e420beeeb86c381c797a898a68c8a7/modules/plugin/mongodb/src/main/java/org/geotools/data/mongodb/MongoFeatureSource.java#L340> in an else statement a call to the superclass visit(BinaryComparisonOperator filter) should be added. · I have the impression filtering on dates/times is either not implemented (throw new UnsupportedOperationException() ) or done by geoserver instead of mongo (costing me the mongodb indexes). Can't test this because of the issue above so I might be wrong here. Currently you can filter by dates/times only if the date time value is stored as a string in ISO format on MongoDB. In a cql filter then you can specify a time filter using >,<,>=,<= operato
[Geotools-devel] geotools mongodb bugs in filtering
Hello everybody, and especially Marco Volpini , We are testing the geoserver/geotools/mongodb combination. We have a number of filtering defects that block us from continuing our tests. There are a few issues open for this, e.g., GEOT-5911 GEOT-6570 GEOT-6348 GEOT-6810 . We debugged the underlying cause, and we think we have enough knowledge to fix this bug. We want to know your position as maintainer: * Do we agree that there is a problem related to filtering worth fixing. * Can you provide necessary help(e.g., guidance) to fix this, if resources available. * Are you willing to accept a pull request , after aligning with your coding standards? Unfortunately, we already sent 2 messages to the geoserver and geotools user list and did not receive any reaction on either. See: https://sourceforge.net/p/geoserver/mailman/geoserver-users/thread/DB9PR05MB898397B5B85AAF69FA48213A9CB59%40DB9PR05MB8983.eurprd05.prod.outlook.com/#msg37671813 https://sourceforge.net/p/geotools/mailman/geotools-gt2-users/thread/DB9PR05MB89834EE49A41E3744010B51D9CBA9%40DB9PR05MB8983.eurprd05.prod.outlook.com/#msg37674928 Kind regards, Hans ___ GeoTools-Devel mailing list GeoTools-Devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geotools-devel
