[Geoserver-devel] Windows build server, almost there
Hi, the windows build keeps on running once per hour to locate spurious failures, as far as I could see during the last week we are down to two common failures: a) XML security tests leaving files open, which cannot be cleaned up after (, e.g. http://winbuild.geo-solutions.it/jenkins/job/GeoServer-Master/org.geoserver$gs-main/1422/testReport/junit/org.geoserver.security.xml/XMLRoleServiceTest/testInsert/ ) b) a rare failure that makes EPSG:4326 impossible to decode, with no information on how to debug it (no errors in the HSQL setup), and no way to reproduce it a) happens every 10-20 builds, b) seems to happen every 20-50 builds I was thinking to make the build server start sending hate mails in case of build failures once we fix a), given than b) seems to be un-crackable and relatively rare. What do you think? Cheers Andrea -- == GeoServer Professional Services from the experts! Visit http://goo.gl/NWWaa2 for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. --- -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
[Geoserver-devel] Jenkins build is back to normal : geoserver-2.7.x-nightly #64
See http://ares.opengeo.org/jenkins/job/geoserver-2.7.x-nightly/64/changes -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y ___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] Improving how importer deals with raster data
Hi all,
following up on this topic, I've implemented the support for gdal_translate
and gdaladdo in
the importer, generalizing a bit the tranfsormation chains, and providing
support for other
command line invocations, should we want to add more in the future.
You can look at the work here:
https://github.com/aaime/geoserver/commit/fefb4da90259a1e14e7a57d786ff4c57bbdacd9f
The transforms are to be used in a direct import, and are available only
from the REST api
(not the first example, there are others already).
Say you want to upload a tiff file and then retile/add overviews to it,
you'd use the following commands:
1) Create import:
curl -u admin:geoserver -XPOST -H Content-type: application/json -d
@import.json http://localhost:8080/geoserver/rest/imports;
Where import.json is:
{
import: {
targetWorkspace: {
workspace: {
name: sf
}
}
}
}
2) Upload the tiff file:
curl -u admin:geoserver -F name=test -F
filedata=@/home/aaime/devel/gisData/rain.tif
http://localhost:8080/geoserver/rest/imports/0/tasks;
3) Append the transformations:
curl -u admin:geoserver -XPOST -H Content-type: application/json -d
@gtx.json http://localhost:8080/geoserver/rest/imports/0/tasks/0/transforms
curl -u admin:geoserver -XPOST -H Content-type: application/json -d
@gad.json http://localhost:8080/geoserver/rest/imports/0/tasks/0/transforms
Where gtx.json is:
{
type: GdalTranslateTransform,
options: [ -co, TILED=YES, -co, BLOCKXSIZE=512, -co,
BLOCKYSIZE=512]
}
and gad.json is:
{
type: GdalAddoTransform,
options: [ -r, average],
levels : [2, 4, 8, 16]
}
4) Run the import:
curl -u admin:geoserver -XPOST
http://localhost:8080/geoserver/rest/imports/0;
If people have no objections, I'll use the above example in the docs, and
turn that github link above in a pull request
Not sure if you have feedback on the other two topics in my first mail in
this thread (have the server fetch files from
shares, indirect raster import), if I don't hear anything I'll pick my
choice next week and more on to implement those
as well
Cheers
Andrea
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
---
--
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y___
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] Improving how importer deals with raster data
On Fri, Apr 24, 2015 at 4:07 PM, Andrea Aime andrea.a...@geo-solutions.it wrote: Hi all, following up on this topic, I've implemented the support for gdal_translate and gdaladdo in the importer, generalizing a bit the tranfsormation chains, and providing support for other command line invocations, should we want to add more in the future. Oh, forgot one thing, currently the code assumes gdal_translate/gdaladdo are in the path. Fair assumption for Linux, may not be so for Windows. I was thinking of where to configure the location of the commands, importer currently has no configuration at all. Shall we go for the usual property file? $DATA_DIR/importer/gdal.properties? I was also thinking of a $DATA_DIR/importer.xml, based of a ImporterInfo bean, which could have a gdal section inside. Cheers Andrea -- == GeoServer Professional Services from the experts! Visit http://goo.gl/NWWaa2 for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. --- -- One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] Slow build, security setup an important portion of our build times
Hi Andrea I think the security configuration test data is ok, but the migrate method is called before the the test setup. The security dir is empty and the migration starts. The method org.geoserver.security.impl.GeoServerJ2eeRoleServiceTest.onSetUp(SystemTestData) is called after the migration and in my case the following method is never called: org.geoserver.data.test.SystemTestData.setUpSecurity() Look at the stack trace Thread [main] (Suspended (breakpoint at line 2119 in GeoServerSecurityManager)) GeoServerSecurityManager.migrateFrom21() line: 2119 GeoServerSecurityManager.onApplicationEvent(ApplicationEvent) line: 329 SimpleApplicationEventMulticaster.multicastEvent(ApplicationEvent) line: 97 GeoServerTestApplicationContext(AbstractApplicationContext).publishEvent(ApplicationEvent) line: 327 GeoServerJ2eeRoleServiceTest(GeoServerSystemTestSupport).setUp(SystemTestData) line: 208 GeoServerJ2eeRoleServiceTest(GeoServerSystemTestSupport).setUp(TestData) line: 1 GeoServerJ2eeRoleServiceTest(GeoServerBaseTestSupportT).doSetup() line: 151 NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method] NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57 DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43 Method.invoke(Object, Object...) line: 606 FrameworkMethod$1.runReflectiveCall() line: 47 FrameworkMethod$1(ReflectiveCallable).run() line: 12 FrameworkMethod.invokeExplosively(Object, Object...) line: 44 RunBefores.evaluate() line: 24 RunAfters.evaluate() line: 27 RunRules.evaluate() line: 20 BlockJUnit4ClassRunner(ParentRunnerT).runLeaf(Statement, Description, RunNotifier) line: 271 BlockJUnit4ClassRunner.runChild(FrameworkMethod, RunNotifier) line: 70 BlockJUnit4ClassRunner.runChild(Object, RunNotifier) line: 50 ParentRunner$3.run() line: 238 ParentRunner$1.schedule(Runnable) line: 63 BlockJUnit4ClassRunner(ParentRunnerT).runChildren(RunNotifier) line: 236 ParentRunnerT.access$000(ParentRunner, RunNotifier) line: 53 ParentRunner$2.evaluate() line: 229 RunBefores.evaluate() line: 26 RunAfters.evaluate() line: 27 BlockJUnit4ClassRunner(ParentRunnerT).run(RunNotifier) line: 309 JUnit4TestClassReference(JUnit4TestReference).run(TestExecution) line: 50 TestExecution.run(ITestReference[]) line: 38 RemoteTestRunner.runTests(String[], String, TestExecution) line: 459 RemoteTestRunner.runTests(TestExecution) line: 675 RemoteTestRunner.run() line: 382 RemoteTestRunner.main(String[]) line: 192 Any idea ?. I do not want to experiment with the test setup, its complicated enough. Cheers Christian On Wed, Apr 22, 2015 at 1:59 PM, Andrea Aime andrea.a...@geo-solutions.it wrote: On Wed, Apr 22, 2015 at 1:25 PM, Christian Mueller christian.muel...@os-solutions.at wrote: Hi Andrea I managed to get a yourkit Open Source license key. The key is valid for one installation and I plan to use this key at my home office. Hope to get some time for investigation during the weekend. Btw, can you tell me the test case(s) according to your screen shots. That's not a single test case, I did attach yourkit to the surefire test JVM while gs-wfs was running. Anyways, for reference the method that's called 3 million times, I did add a static counter into it and have it to a system out every time it was called, running one of the GetFeatureTests I checked it got called over 300 times (all during the test setup). Cheers Andrea -- == GeoServer Professional Services from the experts! Visit http://goo.gl/NWWaa2 for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or
Re: [Geoserver-devel] Windows build server, almost there
+1 Regards, Simone Giannecchini == GeoServer Professional Services from the experts! Visit http://goo.gl/NWWaa2 for more information. == Ing. Simone Giannecchini @simogeo Founder/Director GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 333 8128928 http://www.geo-solutions.it http://twitter.com/geosolutions_it --- AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. On Fri, Apr 24, 2015 at 8:26 AM, Andrea Aime andrea.a...@geo-solutions.it wrote: Hi, the windows build keeps on running once per hour to locate spurious failures, as far as I could see during the last week we are down to two common failures: a) XML security tests leaving files open, which cannot be cleaned up after (, e.g. http://winbuild.geo-solutions.it/jenkins/job/GeoServer-Master/org.geoserver$gs-main/1422/testReport/junit/org.geoserver.security.xml/XMLRoleServiceTest/testInsert/) b) a rare failure that makes EPSG:4326 impossible to decode, with no information on how to debug it (no errors in the HSQL setup), and no way to reproduce it a) happens every 10-20 builds, b) seems to happen every 20-50 builds I was thinking to make the build server start sending hate mails in case of build failures once we fix a), given than b) seems to be un-crackable and relatively rare. What do you think? Cheers Andrea -- == GeoServer Professional Services from the experts! Visit http://goo.gl/NWWaa2 for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content,
