i just dont see what we can do about this... we are calling
connection.getLastModified(), we are not creating a file, etc...

-igor

On Wed, Oct 21, 2009 at 10:18 AM, adambender <[email protected]> wrote:
>
> This issue looks like it is directly related to
> http://issues.apache.org/jira/browse/WICKET-438 and the access of the
> lastModified header. Every time a URLResourceStream is constructed the
> lastModified time is requested at line 85 and the number of file handles
> goes up by one. The solution to the jira issue indicated that upgrading the
> version of linux fixed the problem but it doesnt seem to be the case with OS
> X or Red Hat Enterprise Linux...
>
>
> adambender wrote:
>>
>> I was able to distill the problem down to what I think is the core issue
>> here.
>>
>> When an AJAX Wicket page is rendered it contains a reference to two .js
>> files as Igor had mentioned, the Web URLs of these files look like this:
>>
>> <App Context
>> Root>/resources/org.apache.wicket.markup.html.WicketEventReference/wicket-event.js
>> <App Context
>> Root>/resources/org.apache.wicket.markup.html.WicketEventReference/wicket-ajax.js
>>
>> These requests are of course fielded by the Wicket filter which ends up
>> trying to load the resources using the UrlResourceStream. In order to
>> actually load these files Wicket gets the JAR URL for each file, converts
>> them to a URIs and then passes them to the File constructor. The problem
>> is that the URI that is created is opaque -
>> http://www.docjar.com/docs/api/java/net/URI.html  see here for a good
>> explanation  in the case of wicket-event.js the URI string is like the
>> following :
>>
>> jar:file:/<absolute-path-tojar>/wicket-1.4.1.jar!/org/apache/wicket/markup/html/wicket-event.js
>>
>> It is similar for the case of wicket-ajax.js
>>
>> A quick glance at the File constructor shows that an opaque URL cannot be
>> used to create a File as it is considered non-hierarchical. This of course
>> leaves you with lots of requests to load this file, none of which can be
>> completed and results in a lot of open URL connections which can never be
>> closed.
>>
>> I guess my question is, should this loading mechanism be working because
>> it doesnt seem like it could the way it is currently written? Is it
>> possible to configure how Wicket finds these files embedded in the jar?
>> Have I missed something?
>>
>>
>> Adam Bender-2 wrote:
>>>
>>> Thanks for the explanation I think that helps shed some light... The
>>> tests are actually JMeter tests driving load by emulating a web
>>> browser - the application the are testing is running in Deployment
>>> mode set up as though it were a production server. After a little more
>>> digging I have been able to determine that is only a problem on pages
>>> which use Ajax - and it looks like there is a related debug message
>>> coming from an exception handler regarding the wicket-event.js file
>>> not being accessible (URI is not hierarchical). This would actually
>>> explain  why the problem only manifests when the JMeter tests are set
>>> to request embedded assets as wicket ajax pages embed requests for
>>> additional pieces of javascript - which in the case of wicket-event.js
>>> are causing exceptions to be thrown and the JVM bug you mentioned is
>>> probably preventing these resources from being cleaned up properly.
>>>
>>> Does that sound right?
>>>
>>> Adam
>>>
>>>
>>> On Oct 20, 2009, at 8:41 PM, Igor Vaynberg wrote:
>>>
>>>> when you are requesting an embedded resource wicket needs to stream
>>>> the file out of a jar, the way the jvm handles that is by creating a
>>>> jar url connection object that streams the file. unfortunately, there
>>>> is a bug in the jdk where the url connection does not have a close
>>>> method and so wicket or any other java app cannot release the
>>>> connection. this is addressed, afair, in jdk 7.
>>>>
>>>> i have many apps deployed in production and have not managed yet to
>>>> run out of the handles with the limit set about 4K. not sure why this
>>>> is different in your case. you mentioned "tests", are those unit tests
>>>> and is wicket there running in deployment mode?
>>>>
>>>> the resource watcher, which should be disabled, will cause the handles
>>>> to run out because it continuously monitors markup files for changes
>>>> (hot reloading in dev mode) and everytime it checks a markup file in a
>>>> jar it creates the url connection and leaks it. by default it is
>>>> disabled in deployment mode but if you manually set the poll frequency
>>>> in your settings it will be reenabled.
>>>>
>>>> -igor
>>>>
>>>>
>>>> On Tue, Oct 20, 2009 at 4:07 PM, Adam Bender <[email protected]>
>>>> wrote:
>>>>> We have run with a limit as high as 10,000 files and our tests can
>>>>> bring it
>>>>> to the limit in 20 minutes, but that still doesn't explain why so
>>>>> many
>>>>> copies of the jar are needed - and only when we are also requesting
>>>>> embedded
>>>>> assets...
>>>>>
>>>>> On Oct 20, 2009, at 5:04 PM, Major Péter wrote:
>>>>>
>>>>>> resolution -> solution...
>>>>>> Just set the ulimit in the initscript and run it with start-stop-
>>>>>> daemon,
>>>>>> it will make the problem disappear (but still there would be many
>>>>>> open
>>>>>> files...)
>>>>>>
>>>>>> Peter
>>>>>>
>>>>>> 2009-10-21 00:38 keltezéssel, Major Péter írta:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> we also had this issue, because (I think) we are running two
>>>>>>> different
>>>>>>> wicket application in only one glassfish domain. Our resolution
>>>>>>> was,
>>>>>>> that we are running now the glassfish domains with custom init
>>>>>>> scripts
>>>>>>> with ulimit settings. Maybe this will help for you.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Peter
>>>>>>>
>>>>>>> 2009-10-21 00:14 keltezéssel, Martin Grigorov írta:
>>>>>>>>
>>>>>>>> Hi Adam,
>>>>>>>>
>>>>>>>> You may try to debug what is the problem with
>>>>>>>> https://wiki.sdn.sap.com/wiki/display/Java/JPicus
>>>>>>>>
>>>>>>>> El mar, 20-10-2009 a las 15:39 -0600, Adam Bender escribió:
>>>>>>>>>
>>>>>>>>> Greetings all,
>>>>>>>>>
>>>>>>>>> Recently I have been performance testing a Wicket application
>>>>>>>>> (1.4.1)
>>>>>>>>> and I
>>>>>>>>> am running into the dreaded Too Many Files Open issue. I have
>>>>>>>>> searched
>>>>>>>>> the
>>>>>>>>> mailing lists and most of the trouble around this issue seems
>>>>>>>>> to come
>>>>>>>>> from
>>>>>>>>> being in DEVELOPMENT mode so I made sure we were in DEPLOYMENT
>>>>>>>>> mode.
>>>>>>>>> When I
>>>>>>>>> run 'lsof -p xxxx | grep wicket-1.4.1.jar' I see over 1000
>>>>>>>>> entries and
>>>>>>>>> it's
>>>>>>>>> growing monotonically. This seems really unusual - why would
>>>>>>>>> wicket
>>>>>>>>> need
>>>>>>>>> 1000 copies of this jar open? An additional bit of weirdness
>>>>>>>>> came up
>>>>>>>>> when
>>>>>>>>> our load tests stopped loading the embedded assets (css, js and
>>>>>>>>> images)
>>>>>>>>> in
>>>>>>>>> each page - this seemed to cap the number of lsof entries at
>>>>>>>>> 5... This
>>>>>>>>> is
>>>>>>>>> even weirder because our app serves these static items from httpd
>>>>>>>>> without
>>>>>>>>> tomcat ever knowing about them. How could our loading of
>>>>>>>>> embedded items
>>>>>>>>> really affect the number of file handles wicket needs?
>>>>>>>>>
>>>>>>>>> For completeness we are running this app on Red Hat Enterprise
>>>>>>>>> Linx 5
>>>>>>>>> with
>>>>>>>>> Tomcat 6.0.20 and Java 1.6
>>>>>>>>>
>>>>>>>>> Adam
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [email protected]
>>>>>> For additional commands, e-mail: [email protected]
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [email protected]
>>>>> For additional commands, e-mail: [email protected]
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [email protected]
>>>> For additional commands, e-mail: [email protected]
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [email protected]
>>> For additional commands, e-mail: [email protected]
>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: 
> http://www.nabble.com/Too-Many-Files-Wicket-1.4.1-tp25983047p25996646.html
> Sent from the Wicket - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to