Re: jk2 and daemon ( was Re: commons-daemon release ?)
Pier Fumagalli wrote: On 11/10/02 3:14, Jon Scott Stevens [EMAIL PROTECTED] wrote: on 2002/10/10 6:50 PM, Pier Fumagalli [EMAIL PROTECTED] wrote: I can tell you that our main Java instance for VNUNET.COM takes approximately 4 to 5 minutes to start... OUCH. Our main web-app has roughly 400 JSP pages to compile (you never know), 20 servlets loaded on startup, 4 lucene indexes to open, 500 connections to the database, 350/400 megabytes of cached objects to de-serialize and put down into memory, and some initial synchronization checks with the DB to find out what are the articles that need to be displayed first (articles ranking) out of an history of some hundred thousand of them (all on line)... It is a big bubba, the JVM memory size is somewhat in the range of 640 Mb... :-) (and it's just 1 web-application out of 6) Only problem? Tomcat doesn't scale that high :-( Couldn't you split the load on different tomcat engines and use for JK for example to spray the load ? -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk2 and daemon ( was Re: commons-daemon release ?)
Costin Manolache wrote: Pier Fumagalli wrote: Costin Manolache [EMAIL PROTECTED] wrote: - starting a VM using exec / monitor the child process. It is not implemented yet in jk2 - but pretty important ( it's one of the features from jserv that wasn't yet ported). It seems daemon has a bit of code - as I mentioned from reading it I don't think it works, and it would be better to use the code from jserv for this - whenever we do implement this. That was the feature which created more problems in JServ... I remember me and Ed hammering on it for months in 97/98. My hint, forget about it, also because if you tie it to the web server process, when you take down the Web Server, also your servlet engine is going to go down, and that's not a very desirable feature given how much time it takes to initialize 7/8 web applications I know about this - and I wasn't thinking to implement it in the same way ( i.e. have the web server directly start/stop tomcat ). The 'feature' is that all tomcat processes ( and you may run more than one in a load balanced mode ) can be started automatically and monitored. If one dies, it'll be automatically restarted. What do you plan to use to do the real-time monitoring ? Threads embedded in Apache 2.0 or will Apache 2.0 execing daemon/service monitors which in turn launch and monitor the JVM ? To make things interesting, this information ( and other like that ) needs to be communicated to apache servers ( to stop sending requests to not-ready servers ), and potentially to an eventual JMX proxy. This is the kind of 'control channel' that was proposed several times and will have to be implemented for other purposes. Ajp13++ ? Well with that proposal, we'll be very similar to WebSphere for example, which have such features, and allow a sysadmin to launch instance (JVM) from a single control panel. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk2 and daemon ( was Re: commons-daemon release ?)
Henri Gomez wrote: What do you plan to use to do the real-time monitoring ? Threads embedded in Apache 2.0 or will Apache 2.0 execing daemon/service monitors which in turn launch and monitor the JVM ? The solution must work for apache1.3 and other single-threaded and multiprocess servers. So scoreboard is probably the best solution. Various apache thread/processes will update the scoreboard with the information we need, and will monitor the scoreboard for requsts for config changes. ( the code for the last part is already there - that's how reloading jk2.properties when changed works ). We need a process that reads/write to the scoreboard - it can be a tomcat ( using JNI+JK2 ). As a note, aparently the solution is also used in other applications - like gnome/RedHat config ( I think kde too ). ( I was just browsing freedesktop.org ) -- Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
jk2 and daemon ( was Re: commons-daemon release ?)
As I mentioned, there's a lot of duplication - and likely we'll see more. I don't see this as a major problem - duplication is sometimes good. It would be however nice to have similar behavior when possible and make sure we pick each other's fixes. The areas of duplication: - starting and embeded machine using JNI. The code seems similar, I did reviewed it in daemon and didn't find anything to grab - but more eyes to look at the code would help. - starting a VM using exec / monitor the child process. It is not implemented yet in jk2 - but pretty important ( it's one of the features from jserv that wasn't yet ported). It seems daemon has a bit of code - as I mentioned from reading it I don't think it works, and it would be better to use the code from jserv for this - whenever we do implement this. - configuration for started processes. Daemon is using CLI, jk uses a file - nothing to do here ( but it would be good if daemon would use properties too ). - Win32 services. This is not yet ported to jk2 - and I'm not sure what to do about jk_nt_service. It works very well, but the code is messy. It would be worth adding a jk2 component in the style of the win32 event log. - chuid/kill. The code in daemon seems very good - that's what I'm using for jk2_user ( I'll check it in after I test more ). As Mladen mentioned, it may be usefull to have an asynchronous channel between tomcat and the web server. It is also very usefull to add the 'monitor/exec' features from jserv. And if we integrate the features from nt_service, jk2 will have all the code that's needed for launching. So it may be worth adding a small 'main()' to jk2. It would read a config - including components that would start/monitor tomcats, async communication, manage the shmem ( so that tomcat doesn't have to use JNI ), etc. Most of the code is already available - in either daemon or jserv. The main questions are 'when' and 'who'. For the first - I suspect not the near future, unless more people are interested and volunteer for the 'who' part :-) -- Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk2 and daemon ( was Re: commons-daemon release ?)
Costin Manolache [EMAIL PROTECTED] wrote: - starting a VM using exec / monitor the child process. It is not implemented yet in jk2 - but pretty important ( it's one of the features from jserv that wasn't yet ported). It seems daemon has a bit of code - as I mentioned from reading it I don't think it works, and it would be better to use the code from jserv for this - whenever we do implement this. That was the feature which created more problems in JServ... I remember me and Ed hammering on it for months in 97/98. My hint, forget about it, also because if you tie it to the web server process, when you take down the Web Server, also your servlet engine is going to go down, and that's not a very desirable feature given how much time it takes to initialize 7/8 web applications Pier -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk2 and daemon ( was Re: commons-daemon release ?)
Pier Fumagalli wrote: Costin Manolache [EMAIL PROTECTED] wrote: - starting a VM using exec / monitor the child process. It is not implemented yet in jk2 - but pretty important ( it's one of the features from jserv that wasn't yet ported). It seems daemon has a bit of code - as I mentioned from reading it I don't think it works, and it would be better to use the code from jserv for this - whenever we do implement this. That was the feature which created more problems in JServ... I remember me and Ed hammering on it for months in 97/98. My hint, forget about it, also because if you tie it to the web server process, when you take down the Web Server, also your servlet engine is going to go down, and that's not a very desirable feature given how much time it takes to initialize 7/8 web applications I know about this - and I wasn't thinking to implement it in the same way ( i.e. have the web server directly start/stop tomcat ). The 'feature' is that all tomcat processes ( and you may run more than one in a load balanced mode ) can be started automatically and monitored. If one dies, it'll be automatically restarted. To make things interesting, this information ( and other like that ) needs to be communicated to apache servers ( to stop sending requests to not-ready servers ), and potentially to an eventual JMX proxy. This is the kind of 'control channel' that was proposed several times and will have to be implemented for other purposes. Having apache directly start/stop tomcat is not the best idea - it can just check if a 'jk2_main' process is started ( the 'communication controler' or monitor ) and launch it if not - but without stoping it if apache stops or having any further relation except normal communication. So there are separate issues - the most important beeing the startup of tocmat(s) automatically ( like jserv - but not strictly tied to apache process lifecycle ). -- Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk2 and daemon ( was Re: commons-daemon release ?)
On 10/10/02 20:21, Costin Manolache [EMAIL PROTECTED] wrote: The 'feature' is that all tomcat processes ( and you may run more than one in a load balanced mode ) can be started automatically and monitored. If one dies, it'll be automatically restarted. tip http://cr.yp.to/daemontools.html It's there, it works, don't reinvent the wheel /tip To make things interesting, this information ( and other like that ) needs to be communicated to apache servers ( to stop sending requests to not-ready servers ), and potentially to an eventual JMX proxy. This is the kind of 'control channel' that was proposed several times and will have to be implemented for other purposes. I read Covalent Managed Servers Console all over the place on this one! :-) Incredible what marketing does to people! :-) I also know that maybe a couple of clients of yours here in London might like that feature, as they asked me if it was possible to implement... :-) As far as I'm concerned, I'm happy with my old way of CVSing out web-applications and deploying them on my servers keeping them in sync, and if something dies, Mr. Bergstein (cr.yp.to) already wrote everything I need! :-) So there are separate issues - the most important beeing the startup of tocmat(s) automatically ( like jserv - but not strictly tied to apache process lifecycle ). I can tell you that our main Java instance for VNUNET.COM takes approximately 4 to 5 minutes to start... If I don't reply to HTTP when that thing is down, I'm going to loose my job, so it's not something I feel that in a real-life production environment comes handy... That said, if that's your itch, scratch it... I'm not going to use it! :-) Pier -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk2 and daemon ( was Re: commons-daemon release ?)
on 2002/10/10 6:50 PM, Pier Fumagalli [EMAIL PROTECTED] wrote: I can tell you that our main Java instance for VNUNET.COM takes approximately 4 to 5 minutes to start... OUCH. -jon -- StudioZ.tv /\ Bar/Nightclub/Entertainment 314 11th Street @ Folsom /\ San Francisco http://studioz.tv/ -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk2 and daemon ( was Re: commons-daemon release ?)
On 11/10/02 3:14, Jon Scott Stevens [EMAIL PROTECTED] wrote: on 2002/10/10 6:50 PM, Pier Fumagalli [EMAIL PROTECTED] wrote: I can tell you that our main Java instance for VNUNET.COM takes approximately 4 to 5 minutes to start... OUCH. Our main web-app has roughly 400 JSP pages to compile (you never know), 20 servlets loaded on startup, 4 lucene indexes to open, 500 connections to the database, 350/400 megabytes of cached objects to de-serialize and put down into memory, and some initial synchronization checks with the DB to find out what are the articles that need to be displayed first (articles ranking) out of an history of some hundred thousand of them (all on line)... It is a big bubba, the JVM memory size is somewhat in the range of 640 Mb... :-) (and it's just 1 web-application out of 6) Only problem? Tomcat doesn't scale that high :-( Pier -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk2 and daemon ( was Re: commons-daemon release ?)
Quoting Pier Fumagalli [EMAIL PROTECTED]: Our main web-app has roughly 400 JSP pages to compile (you never know), 20 servlets loaded on startup, 4 lucene indexes to open, 500 connections to the database, 350/400 megabytes of cached objects to de-serialize and put down into memory, and some initial synchronization checks with the DB to find out what are the articles that need to be displayed first (articles ranking) out of an history of some hundred thousand of them (all on line)... It is a big bubba, the JVM memory size is somewhat in the range of 640 Mb... :-) (and it's just 1 web-application out of 6) Only problem? Tomcat doesn't scale that high :-( Since you like segfaults better then NPE's and you seem to need performance, maybe you should try CSP's: http://astro.temple.edu/~john43/ ;-) Bojan -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
