Re: [JPP-Devel] Metric distances for lat long coordinates
Hi, Sounds useful to me. But i am not sure its so easy given that the length of a degree is changing, but there may be some formulas out there (even web pages) that calculate great circle distances ... mhm... i think somewhere i even used such, once. Cheers, Stefan On Tue, Feb 19, 2019, 15:49 Nicolas Ribot Hi, > > I was thinking about adding an option when measuring distance, to compute > distance in meters (or km) if map units are latitude/longitude. > > For instance, here: a distance measure along a 1 degree segment: > > [image: Screenshot 2019-02-19 at 19.45.41.png] > > Option in Measurement would allow to activate this feature and choose > between m and km (other units too ?) > > Do you think this feature would be useful ? > > Cheers > Nicolas > ___ > Jump-pilot-devel mailing list > Jump-pilot-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Slow parsing of date field from Spatialite database
Hi, In r6130 the building of layer list from Geopackage datastore works (thanks Nicolas) and parsing fields with the date/datetime data is very fast (thanks Ede). There seems to be still something wrong with adding Geopackage tables into map through the layer list. Error gives a hint that building spatial index query fails but the same error happens also if I drop the spatial index from the geopackage db. java.lang.NullPointerException at com.vividsolutions.jump.datastore.spatialite.SpatialiteSQLBuilder.buildSpatialIndexFilter(SpatialiteSQLBuilder.java:157) at com.vividsolutions.jump.datastore.spatialite.SpatialiteSQLBuilder.buildBoxFilter(SpatialiteSQLBuilder.java:116) at com.vividsolutions.jump.datastore.spatialite.SpatialiteSQLBuilder.getSQL(SpatialiteSQLBuilder.java:39) at com.vividsolutions.jump.workbench.ui.plugin.datastore.DataStoreDataSource.createFeatureCollection(DataStoreDataSource.java:165) at com.vividsolutions.jump.workbench.ui.plugin.datastore.DataStoreDataSource$1.executeQuery(DataStoreDataSource.java:101) at com.vividsolutions.jump.workbench.ui.plugin.datastore.DataStoreDataSource$1.executeQuery(DataStoreDataSource.java:112) at org.openjump.core.ui.plugin.datastore.AddDataStoreLayerWizard.executeQuery(AddDataStoreLayerWizard.java:175) at org.openjump.core.ui.plugin.datastore.AddDataStoreLayerWizard.load(AddDataStoreLayerWizard.java:164) at org.openjump.core.ui.plugin.datastore.AddDataStoreLayerWizard.createLayer(AddDataStoreLayerWizard.java:138) at org.openjump.core.ui.plugin.datastore.AddDataStoreLayerWizard.createLayers(AddDataStoreLayerWizard.java:156) at org.openjump.core.ui.plugin.datastore.AddDataStoreLayerWizard.run(AddDataStoreLayerWizard.java:71) at org.openjump.core.ui.plugin.file.OpenWizardPlugIn.run(OpenWizardPlugIn.java:110) at com.vividsolutions.jump.workbench.ui.task.TaskMonitorManager$TaskWrapper.run(TaskMonitorManager.java:151) -Jukka- -Alkuperäinen viesti- Lähettäjä: edgar.sol...@web.de Lähetetty: tiistai 19. helmikuuta 2019 17.27 Vastaanottaja: Rahkonen Jukka (MML) ; jump devel ; Nicolas Ribot Aihe: Re: VS: [JPP-Devel] Slow parsing of date field from Spatialite database Jukka, i locally reverted Nico's last change to have the datastore tables listed and work on the date/time slowness. OJ r6129 should fix the slow date issue with the datastore functionality in OJ CORE. when Nico fixes the table listing issue you should be able to double check it. DB Query extension is a somewhat harder nut to crack as i would have to setup the external extension sources to patch them. do you think it is worth the effort or urgently needed? ..ede On 19.02.2019 13:44, Rahkonen Jukka (MML) wrote: > Hi, > > Here you can get a gpkg that works with DB Query > http://latuviitta.org/downloads/stones.gpkg. As I told, DB Query can't read > random_points.gpkg but the standard Run datastore query tool has no problem > with that. > Advice for reading gpkg can be found from the attached document. > > OJ build 5966 can build the layer list for the gpkg datastore, but build 5977 > can't so something has happened in between. > > > -Jukka- > > > -Alkuperäinen viesti- > Lähettäjä: edgar.sol...@web.de > Lähetetty: tiistai 19. helmikuuta 2019 13.54 > Vastaanottaja: Rahkonen Jukka (MML) > > Aihe: Re: [JPP-Devel] Slow parsing of date field from Spatialite > database > > hey Jukka, > > not familiar with the drivers. could you give a short step-by-step for both > (Run Datastore Query, DB Query) ? > > thanks.. ede > > On 17.02.2019 22:07, Rahkonen Jukka (MML) wrote: >> Hi, >> >> Test geopackage finally ready here >> http://latuviitta.org/downloads/random_points.gpkg. It contains >> random_points: 105000 points with a date column >> random_points_without_date : same points but date column dropped >> >> Data were created with the OpenJUMP Bean tools and GDAL. Observations: >> 1) GeoPackage does not work with OpenJDK 13. Connection to SQLite is OK but >> selecting spatial data fails. Use JRE 8 instead for these tests. >> 2) When I create a new spatialite connection into gpkg file OpenJUMP >> creates the connection but it does not find spatial tables and >> adding data to the map is not possible through the Add data... route >> 3) Run datastore query, however, does work. The test will be simply >> to run select * from random_points; select * from >> random_points_without_date; >> >> The first query creates a map in 30 seconds, the second one in one second. >> Tested with OJ Plus snapshot without Spatialite binaries. >> >> -Jukka- >> >> >> >> -Alkuperäinen viesti- >> Lähettäjä: Rahkonen Jukka (MML) >> Lähetetty: sunnuntai 17. helmikuuta 2019 14.26 >> Vastaanottaja: 'Edgar Soldin' >> Aihe: Re: [JPP-Devel] Slow parsing of date field from Spatialite >> database >> >> Hi Ede, >> >> Thanks for reminding, I will have a look at tis today. >>
[JPP-Devel] Metric distances for lat long coordinates
Hi, I was thinking about adding an option when measuring distance, to compute distance in meters (or km) if map units are latitude/longitude. For instance, here: a distance measure along a 1 degree segment: [image: Screenshot 2019-02-19 at 19.45.41.png] Option in Measurement would allow to activate this feature and choose between m and km (other units too ?) Do you think this feature would be useful ? Cheers Nicolas ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] SVN: [6130] core/trunk
Revision: 6130 http://sourceforge.net/p/jump-pilot/code/6130 Author: elnico Date: 2019-02-19 18:15:21 + (Tue, 19 Feb 2019) Log Message: --- corrected typo preventing spatial tables to be listed Modified Paths: -- core/trunk/ChangeLog core/trunk/src/com/vividsolutions/jump/datastore/spatialite/SpatialiteDSMetadata.java Modified: core/trunk/ChangeLog === --- core/trunk/ChangeLog2019-02-19 15:21:53 UTC (rev 6129) +++ core/trunk/ChangeLog2019-02-19 18:15:21 UTC (rev 6130) @@ -3,6 +3,9 @@ # 2. make sure that lines break at 80 chars for constricted display situations #< 80 chars --># +2019-02-19 Nicolas Ribot + * Corrected typo in SpatialiteDSMetadata datasetInfoQuery string preventing spatial tables to be listed + 2019-02-17 ede * PLUS upgrade KML extension - 0.2.5 (2019-02-17) also read if no exists Modified: core/trunk/src/com/vividsolutions/jump/datastore/spatialite/SpatialiteDSMetadata.java === --- core/trunk/src/com/vividsolutions/jump/datastore/spatialite/SpatialiteDSMetadata.java 2019-02-19 15:21:53 UTC (rev 6129) +++ core/trunk/src/com/vividsolutions/jump/datastore/spatialite/SpatialiteDSMetadata.java 2019-02-19 18:15:21 UTC (rev 6130) @@ -177,7 +177,7 @@ } else if (this.geometryColumnsLayout == GeometryColumnsLayout.OGC_GEOPACKAGE_LAYOUT) { datasetInfoQuery = "SELECT '' as table_schema, table_name, column_name, " + "case when z+m = 0 then 2 when z = 1 and m = 1 then 4 else 3 end as coord_dimension, " + - "srs_id, geometry_type_name FROM gpkg_geometry_columns where table_name = '%s'"; + "srs_id, geometry_type_name FROM gpkg_geometry_columns"; } else { datasetInfoQuery = "SELECT '' "; } ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Slow parsing of date field from Spatialite database
Jukka, i locally reverted Nico's last change to have the datastore tables listed and work on the date/time slowness. OJ r6129 should fix the slow date issue with the datastore functionality in OJ CORE. when Nico fixes the table listing issue you should be able to double check it. DB Query extension is a somewhat harder nut to crack as i would have to setup the external extension sources to patch them. do you think it is worth the effort or urgently needed? ..ede On 19.02.2019 13:44, Rahkonen Jukka (MML) wrote: > Hi, > > Here you can get a gpkg that works with DB Query > http://latuviitta.org/downloads/stones.gpkg. As I told, DB Query can't read > random_points.gpkg but the standard Run datastore query tool has no problem > with that. > Advice for reading gpkg can be found from the attached document. > > OJ build 5966 can build the layer list for the gpkg datastore, but build 5977 > can't so something has happened in between. > > > -Jukka- > > > -Alkuperäinen viesti- > Lähettäjä: edgar.sol...@web.de > Lähetetty: tiistai 19. helmikuuta 2019 13.54 > Vastaanottaja: Rahkonen Jukka (MML) > Aihe: Re: [JPP-Devel] Slow parsing of date field from Spatialite database > > hey Jukka, > > not familiar with the drivers. could you give a short step-by-step for both > (Run Datastore Query, DB Query) ? > > thanks.. ede > > On 17.02.2019 22:07, Rahkonen Jukka (MML) wrote: >> Hi, >> >> Test geopackage finally ready here >> http://latuviitta.org/downloads/random_points.gpkg. It contains >> random_points: 105000 points with a date column >> random_points_without_date : same points but date column dropped >> >> Data were created with the OpenJUMP Bean tools and GDAL. Observations: >> 1) GeoPackage does not work with OpenJDK 13. Connection to SQLite is OK but >> selecting spatial data fails. Use JRE 8 instead for these tests. >> 2) When I create a new spatialite connection into gpkg file OpenJUMP >> creates the connection but it does not find spatial tables and adding >> data to the map is not possible through the Add data... route >> 3) Run datastore query, however, does work. The test will be simply to >> run select * from random_points; select * from >> random_points_without_date; >> >> The first query creates a map in 30 seconds, the second one in one second. >> Tested with OJ Plus snapshot without Spatialite binaries. >> >> -Jukka- >> >> >> >> -Alkuperäinen viesti- >> Lähettäjä: Rahkonen Jukka (MML) >> Lähetetty: sunnuntai 17. helmikuuta 2019 14.26 >> Vastaanottaja: 'Edgar Soldin' >> Aihe: Re: [JPP-Devel] Slow parsing of date field from Spatialite >> database >> >> Hi Ede, >> >> Thanks for reminding, I will have a look at tis today. >> >> BTW I faced a total dead end today when trying to start OJ. It stopped >> on this line (here captured from a successful start) Loading Plugin >> org.openjump.core.ui.plugin.file.SaveLayersWithoutDataSourcePlugIn >> took 5.53s >> >> I could not even kill the JDK process from Windows 10 control panel. Same >> happened with OpenJDK 13 and Oracle JRE 1.8. After rebooting my computer >> this issue went away. >> >> -Jukka- >> >> -Alkuperäinen viesti- >> Lähettäjä: Edgar Soldin >> Lähetetty: sunnuntai 17. helmikuuta 2019 14.08 >> Vastaanottaja: Rahkonen Jukka (MML) >> >> Aihe: Fwd: [JPP-Devel] Slow parsing of date field from Spatialite >> database >> >> >> see below.. ede >> >> Forwarded Message >> Subject: Re: [JPP-Devel] Slow parsing of date field from Spatialite >> database >> Date: Tue, 22 Jan 2019 12:06:20 +0100 >> From: edgar.sol...@web.de >> Reply-To: OpenJump develop and use >> >> To: OpenJump develop and use , >> Rahkonen Jukka (MML) >> >> hey Jukka, >> >> just came across this mail. is this still an issue? can you provide me w/ a >> sample dataset? >> >> ..ede >> >> On 07.08.2017 09:46, edgar.sol...@web.de wrote: >>> probably exactly the same issue of realtime date parsing during loading >>> like w/ JML there. maybe i can speed up the parser, by meorizing the >>> successful pattern and trying that first on the next value. currently it >>> looks like it is bruteforcing the same pattern order over & over agn. >>> >>> ..ede >>> >>> On 8/6/2017 21:51, Rahkonen Jukka (MML) wrote: Hi, There seems to be some inefficiency in parsing date field from Spatialite database by using the Run database query tool. Reading 8 million points with one DATE field takes 15 minutes with my computer. If I do not select the DATE field it takes only 35 seconds to get all the data. DB Query plugin cannot parse DATE fields at all so I could not make a proper comparison. Query without DATE field took 3 times more time with DB Query. -Jukka Rahkonen- - - Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot
[JPP-Devel] SVN: [6129] core/trunk/src/com/vividsolutions/jump/datastore
Revision: 6129 http://sourceforge.net/p/jump-pilot/code/6129 Author: edso Date: 2019-02-19 15:21:53 + (Tue, 19 Feb 2019) Log Message: --- speedup loading datasets w/ date/time columns utilizing flex feature lazy conversion Modified Paths: -- core/trunk/src/com/vividsolutions/jump/datastore/jdbc/ValueConverterFactory.java core/trunk/src/com/vividsolutions/jump/datastore/spatialdatabases/SpatialDatabasesResultSetConverter.java Modified: core/trunk/src/com/vividsolutions/jump/datastore/jdbc/ValueConverterFactory.java === --- core/trunk/src/com/vividsolutions/jump/datastore/jdbc/ValueConverterFactory.java 2019-02-19 15:01:52 UTC (rev 6128) +++ core/trunk/src/com/vividsolutions/jump/datastore/jdbc/ValueConverterFactory.java 2019-02-19 15:21:53 UTC (rev 6129) @@ -117,23 +117,24 @@ public static class DateConverter implements ValueConverter { public AttributeType getType() { return AttributeType.DATE; } public Object getValue(ResultSet rs, int columnIndex) throws SQLException { -//return rs.getDate(columnIndex); -Object ret = null; -try { -ret = rs.getTimestamp(columnIndex); -if (rs.wasNull()) return null; -} catch (Exception e) { -// try to read date from string, as some SpatialDatabases like SQLite -// can store DATE type in string -FlexibleDateParser parser = new FlexibleDateParser(); -try { -ret = parser.parse(rs.getString(columnIndex), false); -} catch (Exception ee) { -System.err.println("cannot parse date value: \"" + rs.getString(columnIndex) -+ "\" Defaulting to null.\n" + ee.getMessage()); -} -} -return ret; + // always return string for dates and let FlexibleFeature convert later during runtime + return rs.getString(columnIndex); +//Object ret = null; +//try { +//ret = rs.getTimestamp(columnIndex); +//if (rs.wasNull()) return null; +//} catch (Exception e) { +//// try to read date from string, as some SpatialDatabases like SQLite +//// can store DATE type in string +//FlexibleDateParser parser = new FlexibleDateParser(); +//try { +//ret = parser.parse(rs.getString(columnIndex), false); +//} catch (Exception ee) { +//System.err.println("cannot parse date value: \"" + rs.getString(columnIndex) +//+ "\" Defaulting to null.\n" + ee.getMessage()); +//} +//} +//return ret; } } Modified: core/trunk/src/com/vividsolutions/jump/datastore/spatialdatabases/SpatialDatabasesResultSetConverter.java === --- core/trunk/src/com/vividsolutions/jump/datastore/spatialdatabases/SpatialDatabasesResultSetConverter.java 2019-02-19 15:01:52 UTC (rev 6128) +++ core/trunk/src/com/vividsolutions/jump/datastore/spatialdatabases/SpatialDatabasesResultSetConverter.java 2019-02-19 15:21:53 UTC (rev 6129) @@ -2,9 +2,9 @@ import com.vividsolutions.jump.datastore.jdbc.ValueConverter; import com.vividsolutions.jump.feature.AttributeType; -import com.vividsolutions.jump.feature.BasicFeature; import com.vividsolutions.jump.feature.Feature; import com.vividsolutions.jump.feature.FeatureSchema; +import com.vividsolutions.jump.feature.FlexibleFeature; import java.sql.Connection; import java.sql.ResultSet; @@ -47,7 +47,8 @@ public Feature getFeature() throws Exception { init(); -Feature f = new BasicFeature(featureSchema); +// use flex feature for lazy data type conversion +Feature f = new FlexibleFeature(featureSchema); for (int i = 0; i < mapper.length; i++) { f.setAttribute(i, mapper[i].getValue(rs, i + 1)); } ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] SVN: [6128] core/trunk/src/com/vividsolutions/jump/util
Revision: 6128 http://sourceforge.net/p/jump-pilot/code/6128 Author: edso Date: 2019-02-19 15:01:52 + (Tue, 19 Feb 2019) Log Message: --- parse dates containing ISO 8601 time zone "-08; -0800; -08:00" eg. "2019/02/17 22:44:35.325+02" Modified Paths: -- core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser.java core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser_dmy.txt core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser_mdy.txt Modified: core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser.java === --- core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser.java 2019-02-17 13:00:27 UTC (rev 6127) +++ core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser.java 2019-02-19 15:01:52 UTC (rev 6128) @@ -342,11 +342,6 @@ return formatters; } -public static void main(String[] args) throws Exception { -//System.out.println(new FlexibleDateParser().parse("03-Mars-1998", false)); -System.out.println(new SimpleDateFormat("-MM-dd'T'HH:mm:ss.SSSZ").parse("2008-11-11T00:00:00.000+0200")); -} - public void setVerbose(boolean b) { verbose = b; } @@ -356,5 +351,14 @@ instance = new FlexibleDateParser(); return instance; } - + +public static void main(String[] args) throws Exception { + FlexibleDateParser fdp = new FlexibleDateParser(); + fdp.setVerbose(true); + + //System.out.println(new FlexibleDateParser().parse("03-Mars-1998", false)); + //System.out.println(new SimpleDateFormat("-MM-dd'T'HH:mm:ss.SSSZ").parse("2008-11-11T00:00:00.000+0200")); + System.out.println(fdp.parse("2019/02/17 22:44:35.325+02", true)); + //System.out.println(new SimpleDateFormat("/MM/dd HH:mm:ss.SSSX").parse("2019/02/17 22:44:35.325+02")); +} } Modified: core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser_dmy.txt === --- core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser_dmy.txt 2019-02-17 13:00:27 UTC (rev 6127) +++ core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser_dmy.txt 2019-02-19 15:01:52 UTC (rev 6128) @@ -84,6 +84,20 @@ -MM-dd HH:mm:ss.SSS z -MM-dd HH:mm:ss z -MM-dd HH:mm +/MM/dd'T'HH:mm:ss +/MM/dd hh:mm:ss +/MM/dd HH:mm:ssz +/MM/dd HH:mm:sszzz +/MM/dd HH:mm:ss.SSSz +/MM/dd HH:mm:ss.SS z +/MM/dd HH:mm:ss.SSS z +/MM/dd HH:mm:ss z +/MM/dd HH:mm:ssX +/MM/dd HH:mm:ss.SSSX +/MM/dd HH:mm:ss.SS X +/MM/dd HH:mm:ss.SSS X +/MM/dd HH:mm:ss X +/MM/dd HH:mm -MM-d HH:mm:ss -DDD/HH:mm:ss.SSS '-'MM'-'dd Modified: core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser_mdy.txt === --- core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser_mdy.txt 2019-02-17 13:00:27 UTC (rev 6127) +++ core/trunk/src/com/vividsolutions/jump/util/FlexibleDateParser_mdy.txt 2019-02-19 15:01:52 UTC (rev 6128) @@ -84,6 +84,20 @@ -MM-dd HH:mm:ss.SSS z -MM-dd HH:mm:ss z -MM-dd HH:mm +/MM/dd'T'HH:mm:ss +/MM/dd hh:mm:ss +/MM/dd HH:mm:ssz +/MM/dd HH:mm:sszzz +/MM/dd HH:mm:ss.SSSz +/MM/dd HH:mm:ss.SS z +/MM/dd HH:mm:ss.SSS z +/MM/dd HH:mm:ss z +/MM/dd HH:mm:ssX +/MM/dd HH:mm:ss.SSSX +/MM/dd HH:mm:ss.SS X +/MM/dd HH:mm:ss.SSS X +/MM/dd HH:mm:ss X +/MM/dd HH:mm -MM-d HH:mm:ss -DDD/HH:mm:ss.SSS '-'MM'-'dd ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
Re: [JPP-Devel] Reading list of spatial tables from GeoPackage broken since October 2018
Hi, I will have a look at it as soon as possible. Nicolas On Tue, 19 Feb 2019 at 13:58, Rahkonen Jukka (MML) < jukka.rahko...@maanmittauslaitos.fi> wrote: > Hi, > > > > It seems that after this change OpenJUMP can’t get the list of spatial > tables from GeoPackage datastore. > > > > 2018-10-12 Nicolas Ribot * New mechanism for SpatialDatabasesDSMetadata to > get information about spatial tables: done in one query, to reduce the > number of queries sent to the backend (took several minutes on big DB) > > > > I have been testing with OpenJUMP Plus without having Spatialite binaries. > Version r5966 is OK but r5977 claims that no spatial tables were found. > > > > -Jukka Rahkonen- > ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
[JPP-Devel] Reading list of spatial tables from GeoPackage broken since October 2018
Hi, It seems that after this change OpenJUMP can't get the list of spatial tables from GeoPackage datastore. 2018-10-12 Nicolas Ribot * New mechanism for SpatialDatabasesDSMetadata to get information about spatial tables: done in one query, to reduce the number of queries sent to the backend (took several minutes on big DB) I have been testing with OpenJUMP Plus without having Spatialite binaries. Version r5966 is OK but r5977 claims that no spatial tables were found. -Jukka Rahkonen- ___ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel