Hi Andrea and Dré Vromans,
Yes, I am trying to import a shapefile into an already existing postgis
store. I am using Ubuntu 14.04, I have been able to successfully upload
files greater than 50MB previously.
A little background on the issue: What happened is I have a raster file
showing land degradation, I converted this to polygons and ended up with a
shapefile of 428MB which though slow I was able to successfully upload
using Geonode interface (sometime last month). However, I ran into an
issue, when downloading this file through Geonode (Geonode allows users to
download layers) only part of the shapefile was being downloaded. The
original shapefile has over 2M features the resulting downloaded shapefile
always has only 1M features. I then decided to 'dissolve' the features
using ArcMap so that I ended up with only 5 features instead. So now when
trying to upload this new dissolved shapefile which is still 428MB I run
into this problem. From yesterdays attempt to upload this shapefile using
geoservers' import data spatial files from directory or archive function
this is the stack trace:
2015-11-16 17:06:29,609 INFO [geoserver.importer] - begining import
2015-11-16 19:13:44,312 WARN [geotools.jdbc] - There is code leaving
feature readers/iterators open, this is leaking statements and
connections!
2015-11-16 23:57:50,798 INFO [geoserver.wms] -
Request: getServiceInfo
And from trying to replicate this process this morning I end up with the
following stack trace
2015-11-17 12:40:09,548 INFO [geoserver.importer] - begining import
2015-11-17 13:01:39,969 ERROR [wicket.RequestCycle] - Error attaching this
container for rendering: [MarkupContainer [Component id = listContainer]]
org.apache.wicket.WicketRuntimeException: Error attaching this container
for rendering: [MarkupContainer [Component id = listContainer]]
at
org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1765)
at org.apache.wicket.Component.onBeforeRender(Component.java:3946)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at
org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1753)
at org.apache.wicket.Component.onBeforeRender(Component.java:3946)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at org.apache.wicket.Component.prepareForRender(Component.java:2292)
at org.apache.wicket.Component.prepareForRender(Component.java:2329)
at
org.apache.wicket.ajax.AjaxRequestTarget.respondComponent(AjaxRequestTarget.java:830)
at
org.apache.wicket.ajax.AjaxRequestTarget.respondComponents(AjaxRequestTarget.java:682)
at
org.apache.wicket.ajax.AjaxRequestTarget.respond(AjaxRequestTarget.java:592)
at
org.apache.wicket.request.AbstractRequestCycleProcessor.respond(AbstractRequestCycleProcessor.java:105)
at
org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1258)
at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1436)
at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
at
org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:484)
at
org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:138)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at
org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:159)
at
org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.geoserver.filters.ThreadLocalsCleanupFilter.doFilter(ThreadLocalsCleanupFilter.java:27)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:74)
at
org.geoserver.wms.animate.AnimatorFilter.doFilter(AnimatorFilter.java:70)
at
org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:70)
at
org.geoserver.filters.SpringDelegatingFilter.doFilter(SpringDelegatingFilter.java:45)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.geoserver.platform.AdvancedDispatchFilter.doFilter(AdvancedDispatchFilter.java:49)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at
org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at
org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at
org.geoserver.security.filter.GeoServerAnonymousAuthenticationFilter.doFilter(GeoServerAnonymousAuthenticationFilter.java:53)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at
org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at
org.geoserver.security.filter.GeoServerUserNamePasswordAuthenticationFilter.doFilter(GeoServerUserNamePasswordAuthenticationFilter.java:115)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at
org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at
org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at
org.geonode.security.GeoNodeCookieProcessingFilter.doFilter(GeoNodeCookieProcessingFilter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at
org.geoserver.security.filter.GeoServerSecurityContextPersistenceFilter$1.doFilter(GeoServerSecurityContextPersistenceFilter.java:52)
at
org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at
org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
at
org.geoserver.security.GeoServerSecurityFilterChainProxy.doFilter(GeoServerSecurityFilterChainProxy.java:134)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:75)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:42)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.geoserver.filters.SessionDebugFilter.doFilter(SessionDebugFilter.java:47)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.geoserver.importer.web.ImportTaskProvider.getItems(ImportTaskProvider.java:40)
at
org.geoserver.web.wicket.GeoServerDataProvider.getFilteredItems(GeoServerDataProvider.java:209)
at
org.geoserver.web.wicket.GeoServerDataProvider.size(GeoServerDataProvider.java:227)
at
org.apache.wicket.markup.repeater.data.DataViewBase.internalGetItemCount(DataViewBase.java:141)
at
org.apache.wicket.markup.repeater.AbstractPageableView.getItemCount(AbstractPageableView.java:258)
at
org.apache.wicket.markup.repeater.AbstractPageableView.getRowCount(AbstractPageableView.java:240)
at
org.apache.wicket.markup.repeater.AbstractPageableView.getViewSize(AbstractPageableView.java:352)
at
org.apache.wicket.markup.repeater.AbstractPageableView.getItemModels(AbstractPageableView.java:105)
at
org.apache.wicket.markup.repeater.RefreshingView.onPopulate(RefreshingView.java:94)
at
org.apache.wicket.markup.repeater.AbstractRepeater.onBeforeRender(AbstractRepeater.java:131)
at
org.apache.wicket.markup.repeater.AbstractPageableView.onBeforeRender(AbstractPageableView.java:121)
at org.apache.wicket.Component.internalBeforeRender(Component.java:1071)
at org.apache.wicket.Component.beforeRender(Component.java:1105)
at
org.apache.wicket.MarkupContainer.onBeforeRenderChildren(MarkupContainer.java:1753)
... 109 more
On Tue, Nov 17, 2015 at 2:09 PM, Dré Vromans <bahum...@gmail.com> wrote:
> Hi Viola,
>
> Could you provide more information (for example, the specific Error you
> receive, what you initially used to upload the file, Linux or Windows,
> Database Shapefile or Directly in a folder)?
> Also, are you sure the settings allow you to upload files of such a size?
> If you use a Linux Server for example, you will use Tomcat which has a
> default maximum allowed file size of (I think) 50MB.
>
> Cheers,
> Dré Vromans
>
> Op di 17 nov. 2015 om 10:04 schreef Andrea Aime <
> andrea.a...@geo-solutions.it>:
>
>> On Tue, Nov 17, 2015 at 9:22 AM, Viola Anne <violaa...@gmail.com> wrote:
>>
>>> I am unable to upload large shapefiles onto geoserver, it takes
>>> extremely long and I eventually have to cancel the process, more than an
>>> hour. Initially I was using geonode interface to upload and when this did
>>> not work, I saved the file on disk and and tried to use the geoserver
>>> import layer but this did not work either. Any help will be appreciated.
>>>
>>
>> Hi Viola,
>> are you trying to import into another store, like postgis?
>> Also, what does "geoserver import did not work" mean, do you have stack
>> traces or error logs?
>>
>> Cheers
>> Andrea
>>
>> --
>> ==
>> GeoServer Professional Services from the experts! Visit
>> http://goo.gl/it488V for more information.
>> ==
>>
>> Ing. Andrea Aime
>> @geowolf
>> Technical Lead
>>
>> GeoSolutions S.A.S.
>> Via Poggio alle Viti 1187
>> 55054 Massarosa (LU)
>> Italy
>> phone: +39 0584 962313
>> fax: +39 0584 1660272
>> mob: +39 339 8844549
>>
>> http://www.geo-solutions.it
>> http://twitter.com/geosolutions_it
>>
>> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>>
>> Le informazioni contenute in questo messaggio di posta elettronica e/o
>> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
>> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
>> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
>> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
>> darcene notizia via e-mail e di procedere alla distruzione del messaggio
>> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
>> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
>> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
>> principi dettati dal D.Lgs. 196/2003.
>>
>>
>>
>> The information in this message and/or attachments, is intended solely
>> for the attention and use of the named addressee(s) and may be confidential
>> or proprietary in nature or covered by the provisions of privacy act
>> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
>> Code).Any use not in accord with its purpose, any disclosure, reproduction,
>> copying, distribution, or either dissemination, either whole or partial, is
>> strictly forbidden except previous formal approval of the named
>> addressee(s). If you are not the intended recipient, please contact
>> immediately the sender by telephone, fax or e-mail and delete the
>> information in this message that has been received in error. The sender
>> does not give any warranty or accept liability as the content, accuracy or
>> completeness of sent messages and accepts no responsibility for changes
>> made after they were sent or for other risks which arise as a result of
>> e-mail transmission, viruses, etc.
>>
>> -------------------------------------------------------
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Geoserver-users mailing list
>> Geoserver-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>
>
--
Kind Regards,
Viola
------------------------------------------------------------------------------
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
