Re: [Geoserver-users] Geoserver Migration+ PostGIS(JNDI)
Andrea, Thank you for your response. Let me put it in a different way. PostGIS Database Server has 45+ databases and tables(imported shapefiles) are stored in default public schema. Each database has 8-10 tables. When I create PostGIS connection to a database using GeoServer interface, I limit max connections to that database as 8. Based on your reply, I think it is where the connection pooling is enabled. Am I correct? Looks like JNDI doesn't help with my data organization. I will have to restructure my data to put them in different schemas in the same database. Correct? Is there any script(s)/mechanism that help me to re-publish my data to make use of JNDI? I check the RAM usage using Webmin module where approx 2GB of RAM is given to Java/Tomcat/GeoServer followed by the list of processes(sample provided below). There is a significant amount of styling for each of the layers. PID Owner Size Command 32426 pgsql 87580 kB postgres: postgres MyLayer1 127.0.0.1(33444) idle Size is what telling me the RAM usage(Correct me if I am wrong). The connections are made and you can see they become idle very soon after they are made.May be they are sitting in the buffer pool of DB? I do not check the HTTP Caching option while publishing the data layers. Please let me know. Thanks, Ravi. From: Andrea Aime andrea.a...@geo-solutions.it To: Ravi Pavuluri ravith...@ymail.com Cc: geoserver-users@lists.sourceforge.net Sent: Thursday, April 21, 2011 2:45 AM Subject: Re: [Geoserver-users] Geoserver Migration+ PostGIS(JNDI) On Thu, Apr 21, 2011 at 12:38 AM, Ravi Pavuluri ravith...@ymail.com wrote: Hi All, We are using 450+ vector layers(45+ databases) in PostGIS database served through GeoServer(2.0.2) most of which is WMS. Given the way it is, RAM is being used up ~60-100MB per layer(expected traffic is not very high as of now).We may hit the RAM limit if traffic becomes higher, since there is no queuing mechanism for total PostGIS connections . No such problem with RAM was encountered with shapefiles, though they are not recommended in production. We have not tried PostGIS (JNDI) but I read it that helps in connection pooling. I don't follow... the PostGIS connection pool has a maximum configurable number of connections. If you say the max is 20 every request trying to get the 21th connection will be queued. That said, with 45 databases I guess you have a very large number if you sum the max number of connections. Is it GeoServer or the postgis processes that are consuming most of the memory? I'm also puzzled as to how you computed that 60-100MB per layer memory consumption. The memory required by WMS is largely dependent on the number of requests going, their size, and the styling used, it's not something statically depending on the number of layers. 1) What is the best way to have all the databases from the same PostGIS schema to JNDI connections? Any estimate of how much(time) of an effort that would be? JNDI helps if you have one database with 45 different schemas, if you have 45 phisically separate databases I don't believe you can setup a single connection pool to talk with them. 2) Also, what would be the best way to migrate all of them to a stable version of GeoServer 2.1(whenever available)? Does combining JNDI connections and GeoServer 2.1 migration save time if we do them together? Any additional documentation on JNDI connection is appreciated. We have a tutorial here: http://docs.geoserver.org/2.0.0/user/tutorials/tomcat-jndi/tomcat-jndi.html Cheers Andrea -- --- Ing. Andrea Aime GeoSolutions S.A.S. Tech lead Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 962313 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://www.youtube.com/user/GeoSolutionsIT http://www.linkedin.com/in/andreaaime http://twitter.com/geowolf - Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Geoserver Migration+ PostGIS(JNDI)
On Thu, Apr 21, 2011 at 6:02 PM, Ravi Pavuluri ravith...@ymail.com wrote: Andrea, Thank you for your response. Let me put it in a different way. PostGIS Database Server has 45+ databases and tables(imported shapefiles) are stored in default public schema. Each database has 8-10 tables. When I create PostGIS connection to a database using GeoServer interface, I limit max connections to that database as 8. Based on your reply, I think it is where the connection pooling is enabled. Am I correct? Correct. With this setup you have at most 45 * 8 open connections: 360. Looks like JNDI doesn't help with my data organization. I will have to restructure my data to put them in different schemas in the same database. Correct? Is there any script(s)/mechanism that help me to re-publish my data to make use of JNDI? Not aware of any, you might want to search on Google. I check the RAM usage using Webmin module where approx 2GB of RAM is given to Java/Tomcat/GeoServer followed by the list of processes(sample provided below). There is a significant amount of styling for each of the layers. PIDOwner Size Command 32426 pgsql 87580 kB postgres: postgres MyLayer1 127.0.0.1(33444) idle Size is what telling me the RAM usage(Correct me if I am wrong). The connections are made and you can see they become idle very soon after they are made.May be they are sitting in the buffer pool of DB? I do not check the HTTP Caching option while publishing the data layers. They are sitting in the pool, that's what the pool is for: keeping connections open to avoid the (significant) overhead involved in closing and opening them. Cheers Andrea -- --- Ing. Andrea Aime GeoSolutions S.A.S. Tech lead Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 962313 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://www.youtube.com/user/GeoSolutionsIT http://www.linkedin.com/in/andreaaime http://twitter.com/geowolf --- -- Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Geoserver Migration+ PostGIS(JNDI)
Thanks again Andrea. I will let the group know if I come across any script or come up with something on my own. Thanks, Ravi. From: Andrea Aime andrea.a...@geo-solutions.it To: Ravi Pavuluri ravith...@ymail.com Cc: geoserver-users@lists.sourceforge.net geoserver-users@lists.sourceforge.net Sent: Thursday, April 21, 2011 12:09 PM Subject: Re: [Geoserver-users] Geoserver Migration+ PostGIS(JNDI) On Thu, Apr 21, 2011 at 6:02 PM, Ravi Pavuluri ravith...@ymail.com wrote: Andrea, Thank you for your response. Let me put it in a different way. PostGIS Database Server has 45+ databases and tables(imported shapefiles) are stored in default public schema. Each database has 8-10 tables. When I create PostGIS connection to a database using GeoServer interface, I limit max connections to that database as 8. Based on your reply, I think it is where the connection pooling is enabled. Am I correct? Correct. With this setup you have at most 45 * 8 open connections: 360. Looks like JNDI doesn't help with my data organization. I will have to restructure my data to put them in different schemas in the same database. Correct? Is there any script(s)/mechanism that help me to re-publish my data to make use of JNDI? Not aware of any, you might want to search on Google. I check the RAM usage using Webmin module where approx 2GB of RAM is given to Java/Tomcat/GeoServer followed by the list of processes(sample provided below). There is a significant amount of styling for each of the layers. PID Owner Size Command 32426 pgsql 87580 kB postgres: postgres MyLayer1 127.0.0.1(33444) idle Size is what telling me the RAM usage(Correct me if I am wrong). The connections are made and you can see they become idle very soon after they are made.May be they are sitting in the buffer pool of DB? I do not check the HTTP Caching option while publishing the data layers. They are sitting in the pool, that's what the pool is for: keeping connections open to avoid the (significant) overhead involved in closing and opening them. Cheers Andrea -- --- Ing. Andrea Aime GeoSolutions S.A.S. Tech lead Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 962313 http://www.geo-solutions.it http://geo-solutions.blogspot.com/ http://www.youtube.com/user/GeoSolutionsIT http://www.linkedin.com/in/andreaaime http://twitter.com/geowolf - Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
