Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
Thanks for the hints. Between those and what we've already known, we should be fine... Bob. Johan Compagner wrote: Tomcat can do that: http://tomcat.apache.org/tomcat-5.5-doc/config/context.html see anti resource and anti jar locking. I think the end result will be that tomcat extracts everything. (dont know which one it really is) But as igor said, a high load of users shouldn't affect the number of open files to the wicket jar if you are in deployment mode. If you are in development mode then 1.2.4still have the problem that the modification watcher does a check every second or so. And that will increase open file handles a lot. johan On 2/2/07, Igor Vaynberg [EMAIL PROTECTED] wrote: the os should be able to handle at least a few thousand open file handles. furthermore you can configure it to have more if you need to. only one handle should be open per resource (no way to avoid that) in a jar, so a spike of user activity shouldnt affect this at all as resources cache globally. but yes, unpacking should eliminate these problems. i wish there was an option in tomcat to unpack the jars on deployment, it can already do this for the war. -igor On 2/1/07, beboris [EMAIL PROTECTED] wrote: It's hard for me to say, what the right way to fix this situation is. The number of open files (that look to the OS as wicket .jar files) will go up significantly and then go down (down - probably due to the garbage collection). It is during those spikes caused by many users hitting wicket pages around the same time that we got too many open files error. It almost looks like the only way to really eliminate those unnecessary open files to numerous Wicket resources (and to guarantee too many open files will never hit us) is to unpack those resources from the wicket .jar-s and put them into WEB-INF/classes directory instead. Once we did it, we didn't see all those dozens (and even hundreds) open files ever again... Bob. Eelco Hillenius wrote: Too bad, Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: why would you cache? and which one? the url connection to an entry in a jar file (thats the JarUrlConnection) or (i guess) the FileUrlConnection (to the jar file itself) both don't make much sense to cache the first one we don't need to cache we only need to use it once by really loading the resource and i guess when it is finalized it is cleaned up. We already don't use it anymore for the last modified. Because there we use only the second one So the fileUrlConnection to the jarFile itself thats is inside the JarUrlConnection object. on that one we call last modified everytime, But that will not cause the file to open. (because it doesn't have to read the file itself) And we can't construct JarUrlConnections (for reading the jar entries) with the same file url connection because there is no way to initialize the jar url connection directly with the file url connection so they all would use the same. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Would it be possible and useful to cache the URL connection? Does it have a time out and/ or does it use an exclusive lock? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: that is what you would think... But why generates a modification check one file handle for every check in the file? because UrlConnection.connect() has again a JarUrlConnection internally that makes a new connection to that jar file and UrlConnection does have a connect() but not a disconnect() so you can't clear it. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Yeah, but that would be always one fd for a jar, no matter how many files in it that have to be read, right? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8751579 Sent from the Wicket - User mailing list archive at Nabble.com. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8751579 Sent from the Wicket - User mailing list archive at Nabble.com. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642___ Wicket-user mailing list Wicket-user@lists.sourceforge.net
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com . - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8751579 Sent from the Wicket - User mailing list archive at Nabble.com . - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com . - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8751579 Sent from the Wicket - User mailing list archive at Nabble.com . - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
Yeah, but that would be always one fd for a jar, no matter how many files in it that have to be read, right? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com . - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8751579 Sent from the Wicket - User mailing list archive at
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
that is what you would think... But why generates a modification check one file handle for every check in the file? because UrlConnection.connect() has again a JarUrlConnection internally that makes a new connection to that jar file and UrlConnection does have a connect() but not a disconnect() so you can't clear it. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Yeah, but that would be always one fd for a jar, no matter how many files in it that have to be read, right? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
Would it be possible and useful to cache the URL connection? Does it have a time out and/ or does it use an exclusive lock? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: that is what you would think... But why generates a modification check one file handle for every check in the file? because UrlConnection.connect() has again a JarUrlConnection internally that makes a new connection to that jar file and UrlConnection does have a connect() but not a disconnect() so you can't clear it. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Yeah, but that would be always one fd for a jar, no matter how many files in it that have to be read, right? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com . - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
why would you cache? and which one? the url connection to an entry in a jar file (thats the JarUrlConnection) or (i guess) the FileUrlConnection (to the jar file itself) both don't make much sense to cache the first one we don't need to cache we only need to use it once by really loading the resource and i guess when it is finalized it is cleaned up. We already don't use it anymore for the last modified. Because there we use only the second one So the fileUrlConnection to the jarFile itself thats is inside the JarUrlConnection object. on that one we call last modified everytime, But that will not cause the file to open. (because it doesn't have to read the file itself) And we can't construct JarUrlConnections (for reading the jar entries) with the same file url connection because there is no way to initialize the jar url connection directly with the file url connection so they all would use the same. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Would it be possible and useful to cache the URL connection? Does it have a time out and/ or does it use an exclusive lock? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: that is what you would think... But why generates a modification check one file handle for every check in the file? because UrlConnection.connect() has again a JarUrlConnection internally that makes a new connection to that jar file and UrlConnection does have a connect() but not a disconnect() so you can't clear it. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Yeah, but that would be always one fd for a jar, no matter how many files in it that have to be read, right? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
Too bad, Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: why would you cache? and which one? the url connection to an entry in a jar file (thats the JarUrlConnection) or (i guess) the FileUrlConnection (to the jar file itself) both don't make much sense to cache the first one we don't need to cache we only need to use it once by really loading the resource and i guess when it is finalized it is cleaned up. We already don't use it anymore for the last modified. Because there we use only the second one So the fileUrlConnection to the jarFile itself thats is inside the JarUrlConnection object. on that one we call last modified everytime, But that will not cause the file to open. (because it doesn't have to read the file itself) And we can't construct JarUrlConnections (for reading the jar entries) with the same file url connection because there is no way to initialize the jar url connection directly with the file url connection so they all would use the same. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Would it be possible and useful to cache the URL connection? Does it have a time out and/ or does it use an exclusive lock? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: that is what you would think... But why generates a modification check one file handle for every check in the file? because UrlConnection.connect() has again a JarUrlConnection internally that makes a new connection to that jar file and UrlConnection does have a connect() but not a disconnect() so you can't clear it. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Yeah, but that would be always one fd for a jar, no matter how many files in it that have to be read, right? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
It's hard for me to say, what the right way to fix this situation is. The number of open files (that look to the OS as wicket .jar files) will go up significantly and then go down (down - probably due to the garbage collection). It is during those spikes caused by many users hitting wicket pages around the same time that we got too many open files error. It almost looks like the only way to really eliminate those unnecessary open files to numerous Wicket resources (and to guarantee too many open files will never hit us) is to unpack those resources from the wicket .jar-s and put them into WEB-INF/classes directory instead. Once we did it, we didn't see all those dozens (and even hundreds) open files ever again... Bob. Eelco Hillenius wrote: Too bad, Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: why would you cache? and which one? the url connection to an entry in a jar file (thats the JarUrlConnection) or (i guess) the FileUrlConnection (to the jar file itself) both don't make much sense to cache the first one we don't need to cache we only need to use it once by really loading the resource and i guess when it is finalized it is cleaned up. We already don't use it anymore for the last modified. Because there we use only the second one So the fileUrlConnection to the jarFile itself thats is inside the JarUrlConnection object. on that one we call last modified everytime, But that will not cause the file to open. (because it doesn't have to read the file itself) And we can't construct JarUrlConnections (for reading the jar entries) with the same file url connection because there is no way to initialize the jar url connection directly with the file url connection so they all would use the same. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Would it be possible and useful to cache the URL connection? Does it have a time out and/ or does it use an exclusive lock? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: that is what you would think... But why generates a modification check one file handle for every check in the file? because UrlConnection.connect() has again a JarUrlConnection internally that makes a new connection to that jar file and UrlConnection does have a connect() but not a disconnect() so you can't clear it. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Yeah, but that would be always one fd for a jar, no matter how many files in it that have to be read, right? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage
Re: [Wicket-user] Fighting Too many open files problem related to wicket resource files
the os should be able to handle at least a few thousand open file handles. furthermore you can configure it to have more if you need to. only one handle should be open per resource (no way to avoid that) in a jar, so a spike of user activity shouldnt affect this at all as resources cache globally. but yes, unpacking should eliminate these problems. i wish there was an option in tomcat to unpack the jars on deployment, it can already do this for the war. -igor On 2/1/07, beboris [EMAIL PROTECTED] wrote: It's hard for me to say, what the right way to fix this situation is. The number of open files (that look to the OS as wicket .jar files) will go up significantly and then go down (down - probably due to the garbage collection). It is during those spikes caused by many users hitting wicket pages around the same time that we got too many open files error. It almost looks like the only way to really eliminate those unnecessary open files to numerous Wicket resources (and to guarantee too many open files will never hit us) is to unpack those resources from the wicket .jar-s and put them into WEB-INF/classes directory instead. Once we did it, we didn't see all those dozens (and even hundreds) open files ever again... Bob. Eelco Hillenius wrote: Too bad, Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: why would you cache? and which one? the url connection to an entry in a jar file (thats the JarUrlConnection) or (i guess) the FileUrlConnection (to the jar file itself) both don't make much sense to cache the first one we don't need to cache we only need to use it once by really loading the resource and i guess when it is finalized it is cleaned up. We already don't use it anymore for the last modified. Because there we use only the second one So the fileUrlConnection to the jarFile itself thats is inside the JarUrlConnection object. on that one we call last modified everytime, But that will not cause the file to open. (because it doesn't have to read the file itself) And we can't construct JarUrlConnections (for reading the jar entries) with the same file url connection because there is no way to initialize the jar url connection directly with the file url connection so they all would use the same. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Would it be possible and useful to cache the URL connection? Does it have a time out and/ or does it use an exclusive lock? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: that is what you would think... But why generates a modification check one file handle for every check in the file? because UrlConnection.connect() has again a JarUrlConnection internally that makes a new connection to that jar file and UrlConnection does have a connect() but not a disconnect() so you can't clear it. johan On 2/2/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Yeah, but that would be always one fd for a jar, no matter how many files in it that have to be read, right? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: yes the modification checker. But we do need to really load the resource out of the jar file once. So that file handle will be used. johan On 2/1/07, Eelco Hillenius [EMAIL PROTECTED] wrote: Wasn't it optimized now so that it only hits jars once and only hits per resource where normal files are involved? Eelco On 2/1/07, Johan Compagner [EMAIL PROTECTED] wrote: one per resource will i think not really change. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We will, when we are on production. Now that we are are still in alpha we prefer development (hey, we wrote our first line of wicket code 5-6 weeks ago!) Also, even in deployment mode 'lsof' still shows us a lot of open files (one per resource) if we don't unpack resources from the .jar . It may be smaller number than in development mode, but still... I imagine your workaround would close those unnecessarily open files. If you can't backport it, please, tell me where it is in SVN. We'll hack it in oursleves for now... Bob Johan Compagner wrote: can't you run in deployment mode instead of development mode? then that shouldn't happen. I can see if we can backport it. johan On 2/1/07, beboris [EMAIL PROTECTED] wrote: We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh
[Wicket-user] Fighting Too many open files problem related to wicket resource files
We have found a pretty weird situation with too many open files error on our alpha-testing site. Further analysis showed that on each page refresh the following resources get repeatedly obtained from the wicket's .jar and add to the number of open files: 'wicket/ajax/wicket-ajax.js' 'wicket/extensions/ajax/markup/html/modal/res/modal.css' 'wicket/extensions/ajax/markup/html/modal/res/modal.js' 'wicket/extensions/markup/html/tree/res/tree.css' 'wicket/extensions/markup/html/tree/res/tree.js' Eventually (due to garbage collection?) the number of open files goes down again. But we wanted to know why those files stayed open in the first place and were not closed upon retrieving a resource. It looks like switching to the deployment mode from development one would significantly reduce the peak numbers of the open files/streams to wicket .jar-s, and extracting resources from the .jar would reduce it even better. We were told that the original reason for files staying open is a Java bug (the fact that URLConnection doesnt have a .close), which causes those nasty results when combined with development mode trying to monitor [resource] files for changes and reloading them. We were also told that there is a workaround for that problem in SVN somewhere, but it's probably not backported to 1.2.5 . I have 2 questions in that regard: (1) Where can we find those workarounds in the code? (2) Is it too much work to backport them to 1.2.5 so when it's released it doesn't contain the problem? Thanks, Bob. -- View this message in context: http://www.nabble.com/Fighting-%22Too-many-open-files%22-problem-related-to-wicket-resource-files-tf3153256.html#a8743682 Sent from the Wicket - User mailing list archive at Nabble.com. - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Wicket-user mailing list Wicket-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wicket-user
