Re: [Devel] Re: [Vserver] VServer vs OpenVZ
Eugen Leitl wrote: Before I try OpenVZ I would like to hear comments of people who've ran both VServer and OpenVZ, preferrably on the same hardware, on how both compare. Factors of interest are stability, Debian support, hardware utilization, documentation and community support, security. As the owner of one of the larger VPS providers around we do use Virtuozzo (commercial support, less work packaging, etc) although I am a big fan of the VServer project. For a hosting service I'd have to recommend Virtuozzo over either, as is a large purpose of OpenVZ I believe :) If you feel you can do the work to keep track of kernel changes, participate actively in debugging, etc then VServer would be fine. I suppose OpenVZ can do the same :) if you like debugging :) As far as the technical debate between the projects, I have more interest in inputting on that. 1) "Fair scheduling" - as far as I can tell the VZ "fair scheduler" does nothing the VServer QoS/Limit system does. If anything, the VZ fair scheduler is not yet O(1) which is a big negative. VServer is built on standard kernel and therefore uses the O(1) scheduler (an absolute must when you have so many processes running on a single kernel). this is not true! Fairscheduler in current implementation on 2.6 kernel is O(1) and doesn't depend on number of processes anyhow! And we are working on improving it much more and implement some additional features in it. 2) Networking - The VZ venet0 is not perfect (no IPv6, still limited iptables, etc), but it still allows a lot more to be performed than VServer in the networking arena. Mmm, if you are an SWsoft customer you can always request some netfilter module to be virtualized. It's a matter of couple of days actually... And there are no any problems with it. We just virtualized the basic set of iptable matches and targets and those which are rarely used didn't. There were only 4(!) IPv6 requests during 2 years. It's the same as netfilters - no much demand, no feature. 3) Disk/memory sharing - OpenVZ has nothing. Virtuozzo uses an overlay fs "vzfs". The templates are good for an enterprise environment, but really prove useless in a hosting environment. vzfs is overlay and therefore suffers from double caching (it caches both files in /vz/private (backing) and /vz/root (mount)). not sure what you mean... memory caching?! it is not true again then... VServer uses CoW links which are modified hard links and eliminates the double caching. The Vserver vunify program (to re-link identical files due to user upgrading to same RPM's across VPS's, etc) takes a few minutes to run. The Virtuozzo vzcache program to do the same can take 2+ days to run on a host with 60+ VPS's. 4) In most other areas OpenVZ and VServer are similar. OpenVZ has many UBC's, but since most oversell some such as vmguarpages really have no affect. Vserver limits the major memory limits (with RSS being a key one that OpenVZ cannot do). On the flip side OpenVZ can limit lowmem (kmemsize) while VServer cannot. RSS is good yeah, but there are lot's of DoS possible if you limit RSS only. No lowmem, no TCP bufs, etc... I personally know many ways of DoSing of other resources, but if you don't care security this is probably ok. 5) Quota inside VPS - The new way of linking the quota user/group files to /proc in OpenVZ is very good (genius?) idea I think. Currently I'm not even 100% sure quota-inside-VPS works under VServer, it's been a few months since I've experimented with it or talked to Herbert Glad to hear that. So which is better? Neither, it depends what the user requires and what they are willing and wanting to do. My 2 cents.. thanks for a lot of information! Kirill ___ Devel mailing list [EMAIL PROTECTED] https://openvz.org/mailman/listinfo/devel ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
[Vserver] Why din't I recieved answers from vserver m-l?
Hello *, I would like to thanks evrybody's help by reply to their answers but I didn't recieved it? here is my mailman options: Mail delivery Enable Set Digest Mode On Get MIME or Plain Text Digests? Plain Text Receive your own posts to the list? Yes Receive acknowledgement mail when you send mail to the list? Yes Get password reminder email for this list? Yes Conceal yourself from subscriber list? Yes What language do you prefer? English (USA) Which topic categories would you like to subscribe to? No topics defined Do you want to receive messages that do not match any topic filter? Yes Avoid duplicate copies of messages? Yes Fwiw, I well recieved Disted and acknowledgement but neither a copy of my post nor answers? What's my mistakes? Thanks again for help, Joel ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: a small patch to fix this pb[Was: [Vserver] testme.sh-0.14 failure on parisc-linux ]
Joel Soete wrote: Hello all, [...] it should also work fine if you compile the tools from scratch (i.e. get the 0.30.209 tools and do ./configure, make, make install) HTH, Herbert [...] mmm, I trust that the actual bug is: --- ./lib/vserver-syscall-def.h.Orig2005-12-10 20:33:19.0 +0100 +++ ./lib/vserver-syscall-def.h 2005-12-10 20:33:40.0 +0100 @@ -28,7 +28,7 @@ #elif defined(__mips__) && (_MIPS_SIM == _MIPS_SIM_NABI32) #define __NR_vserver 236 #elif defined(__hppa__) -#define __NR_vserver 273 +#define __NR_vserver 263 #elif defined(__powerpc__) #define __NR_vserver 257 #elif defined(__s390__) <> Daniel Hokka Zakrisson wrote: >This file is not present at all in vanilla util-vserver, and looks a lot >like lib/syscall-fallback.h from util-vserver-0.30.209, except for the >incorrect syscall number. You had perfectly wright ;-) How uglily this debian pkg was so populated?? I will so coming back from upstream stuff awaiting a better dpkg from Micah. Thanks again, Joel ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
[Vserver] memory problems?
The following is logged quite often to the syslog in a vserver and I'm worried since I don't see this in any of our regular servers: postfix/sendmail[32274]: warning: fork: Cannot allocate memory This is on Debian sarge on a 2.6 kernel with vserver v 2.0. The postfix server in question is not busy at all and the vserver has plenty of memory to spare. Any reason to worry here? -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] memory problems?
On Sun, Dec 11, 2005 at 08:19:11PM +0200, Roché Compaan wrote: > The following is logged quite often to the syslog in a vserver and I'm > worried since I don't see this in any of our regular servers: > > postfix/sendmail[32274]: warning: fork: Cannot allocate memory interesting, what does your /proc/virtual//limits file (with the xid of that guest) show? > This is on Debian sarge on a 2.6 kernel with vserver v 2.0. The postfix > server in question is not busy at all and the vserver has plenty of > memory to spare. > > Any reason to worry here? well, I would try with a recent patch and see if it is still an issues ... best, Herbert > -- > Roché Compaan > Upfront Systems http://www.upfrontsystems.co.za > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] memory problems?
hi, i'm having similar errors (I do have limits and scheduler set, using rlimits (as, rss, nproc) and scheduler) whenever i do stress testing, (overloading mta or web server for example). during a stress test, some applications die because of no memory available or can't fork, some stop with segmentation fault (not able to do vserver enter),. when attacking httpd might have httpd . the only thing helps is apps configuration overview, i.e. for httpd I've changed maxServers to a smaller number for others something esle. I didn't think about it as something bad, always tought it's because of limits, am I wrong? what I don't like when such things happen, some apps die and you have to start them manualy again. I'm using this patch on fc4, should I try something else? http://www.13thfloor.at/vserver/s_rel26/v2.0/patch-2.6.12.4-vs2.0.diff --alex On 12/11/05, Herbert Poetzl <[EMAIL PROTECTED]> wrote: > On Sun, Dec 11, 2005 at 08:19:11PM +0200, Roché Compaan wrote: > > The following is logged quite often to the syslog in a vserver and I'm > > worried since I don't see this in any of our regular servers: > > > > postfix/sendmail[32274]: warning: fork: Cannot allocate memory > > interesting, what does your /proc/virtual//limits > file (with the xid of that guest) show? > > > This is on Debian sarge on a 2.6 kernel with vserver v 2.0. The postfix > > server in question is not busy at all and the vserver has plenty of > > memory to spare. > > > > Any reason to worry here? > > well, I would try with a recent patch and see if it > is still an issues ... > > best, > Herbert > > > -- > > Roché Compaan > > Upfront Systems http://www.upfrontsystems.co.za > > > > ___ > > Vserver mailing list > > Vserver@list.linux-vserver.org > > http://list.linux-vserver.org/mailman/listinfo/vserver > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver > ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
[Vserver] vserver as an image file mounted using loop
hi, is it ok to keep vserver as an image file and then mount it using loop device? let say I have vs1.img, vs2.img ... vsN.img, all i want to do just mount them whenever I need, and do vserver vsN start image contains something like /conf /root and, /vservers/vs1 (symlink to /mountpoint/vs1/root) /etc/vservers/vs1 (symlink to /mountpoint/vs1/conf) thanks --alex ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Devel] Re: [Vserver] VServer vs OpenVZ
1) "Fair scheduling" - as far as I can tell the VZ "fair scheduler" does nothing the VServer QoS/Limit system does. If anything, the VZ fair scheduler is not yet O(1) which is a big negative. VServer is built on standard kernel and therefore uses the O(1) scheduler (an absolute must when you have so many processes running on a single kernel). this is not true! Fairscheduler in current implementation on 2.6 kernel is O(1) and doesn't depend on number of processes anyhow! And we are working on improving it much more and implement some additional features in it. Great, why not provide packages against latest Virtuozzo (with modules, vzfs, etc) for better real world testing? What numbers are you seeing in regards to load, based on my estimates with 3000 procs and an avg of 3 running on a server with a load average of 3 should drop to much lower. What kind of numbers do you see in your tests? 3) Disk/memory sharing - OpenVZ has nothing. Virtuozzo uses an overlay fs "vzfs". The templates are good for an enterprise environment, but really prove useless in a hosting environment. vzfs is overlay and therefore suffers from double caching (it caches both files in /vz/private (backing) and /vz/root (mount)). not sure what you mean... memory caching?! it is not true again then... The kernel caches based on inode number. If you modified the caching part of the module then I may be incorrect in my thinking. Take example: # ls -ai /vz/private/1/root/bin/ls 41361462 /vz/private/1/root/bin/ls # ls -ai /vz/template/redhat-as3-minimal/coreutils-4.5.3-26/bin/ls 1998864 /vz/template/redhat-as3-minimal/coreutils-4.5.3-26/bin/ls # ls -ai /vz/root/1/bin/ls 41361462 /vz/root/1/bin/ls The kernel will cache both inodes 41361462 and 1998864. Knowing that, when I look at my host servers with 8GB of RAM and see 4GB being used for cache/buffers I get angry. vzfs appears to be a standard unionfs with support for CoW to those who do not see the source. You ignored responding to how VServer does it which results in using a patched kernel to have special CoW links without a union mount. The links are based on a hard link architecture resulting in 1 inode. Also commenting on was ignored vunify and vzcache speeds. Pertaining paragraph: VServer uses CoW links which are modified hard links and eliminates the double caching. The Vserver vunify program (to re-link identical files due to user upgrading to same RPM's across VPS's, etc) takes a few minutes to run. The Virtuozzo vzcache program to do the same can take 2+ days to run on a host with 60+ VPS's. > 4) In most other areas OpenVZ and VServer are similar. OpenVZ has many UBC's, but since most oversell some such as vmguarpages really have no affect. Vserver limits the major memory limits (with RSS being a key one that OpenVZ cannot do). On the flip side OpenVZ can limit lowmem (kmemsize) while VServer cannot. RSS is good yeah, but there are lot's of DoS possible if you limit RSS only. No lowmem, no TCP bufs, etc... I personally know many ways of DoSing of other resources, but if you don't care security this is probably ok. It does RSS and VM limiting with no guarantees. It also does locked pages, sockets, etc. The argument of who has more structures to limit is actually rather pointless now as VServer could take the OpenVZ limits to see what they can limit and decide which they want to implement. That is only a matter of time. I'm sure Herbert has seen output of /proc/user_beancounters before OpenVZ was even released and didn't see a reason for some of the limits. What I was pointing out was differences currently. A very minor advantage of VServer if they virtualize the meminfo structure to reflect memory/swap total/usage based on the RSS/VM limits. Thank you, Matt Ayres ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Devel] Re: [Vserver] VServer vs OpenVZ
The kernel caches based on inode number. If you modified the caching part of the module then I may be incorrect in my thinking. Take example: # ls -ai /vz/private/1/root/bin/ls 41361462 /vz/private/1/root/bin/ls # ls -ai /vz/template/redhat-as3-minimal/coreutils-4.5.3-26/bin/ls 1998864 /vz/template/redhat-as3-minimal/coreutils-4.5.3-26/bin/ls # ls -ai /vz/root/1/bin/ls 41361462 /vz/root/1/bin/ls This may actually be an error. This new info was on a linked file (using templates). I tried on a VPS that has broken the link and found this: # ls -ali /vz/private/101/root/bin/ls 245325878 -rwxr-xr-x1 root root67700 Sep 29 11:42 /vz/private/101/root/bin/ls # ls -ali /vz/root/101/bin/ls 245325878 -rwxr-xr-x1 root root67700 Sep 29 11:42 /vz/root/101/bin/ls Or - the third thought is that we are better off not using the CoW/vzfs part of Virtuozzo at all and OpenVZ is a more scalable alternative to Virtuozzo. ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] memory problems?
On Sun, 2005-12-11 at 19:26 +0100, Herbert Poetzl wrote: > On Sun, Dec 11, 2005 at 08:19:11PM +0200, Roché Compaan wrote: > > The following is logged quite often to the syslog in a vserver and I'm > > worried since I don't see this in any of our regular servers: > > > > postfix/sendmail[32274]: warning: fork: Cannot allocate memory > > interesting, what does your /proc/virtual//limits > file (with the xid of that guest) show? [EMAIL PROTECTED]:~# cat /proc/virtual/49196/limit PROC: 120 266 -1 0 VM: 407806 50 50 505 VML: 0 0 -1 0 RSS:194897 25 25 97 ANON: 181688 182048 -1 0 FILES: 9361970 -1 0 OFD: 331 678 -1 0 LOCKS: 10 25 -1 0 SOCK: 128 355 -1 0 MSGQ:0 0 -1 0 SHM: 0 0 -1 0 > > > This is on Debian sarge on a 2.6 kernel with vserver v 2.0. The postfix > > server in question is not busy at all and the vserver has plenty of > > memory to spare. > > > > Any reason to worry here? > > well, I would try with a recent patch and see if it > is still an issues ... The latest patch in Debian (unstable) seems to be patch-2.6.14.3-vs2.0.1-rc5.diff. Is this recent enough? -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] memory problems?
On Sun, 2005-12-11 at 11:34 -0800, Alexander Kabanov wrote: > hi, > > i'm having similar errors (I do have limits and scheduler set, using > rlimits (as, rss, nproc) and scheduler) whenever i do stress testing, > (overloading mta or web server for example). > > during a stress test, some applications die because of no memory > available or can't fork, some stop with segmentation fault (not able > to do vserver enter),. when attacking httpd might have httpd > . I am experiencing the same problems. The segfaults and not being able to enter a vserver from the host *really* worries me. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
[Vserver] Basic resource limits for a vserver
Hi List! Anyone care to share their thoughts on how to limit the reources for a virtual server in the "best" way. My requirements basically are the following: - No single vserver should be allowed to bring the other vservers and the host down. (Given that the other vservers are "behaving" properly.) - Each vserver should be allowed to use at much as available of the system resources as long as the other vservers and the host do not suffer noticeably from this. - The host should hopefully never be brought down, and should always reply easily to ssh administration logins and commands. - Disk resources are not important in this setup. (Have lotsa disk space now, and vservers on separate partitions.) Any suggestions of limits settings for the vservers to achieve this? Best regards, Tor Rune Skoglund <[EMAIL PROTECTED]> ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] memory problems?
On Sun, Dec 11, 2005 at 10:18:48PM +0200, Roché Compaan wrote: > On Sun, 2005-12-11 at 19:26 +0100, Herbert Poetzl wrote: > > On Sun, Dec 11, 2005 at 08:19:11PM +0200, Roché Compaan wrote: > > > The following is logged quite often to the syslog in a vserver and I'm > > > worried since I don't see this in any of our regular servers: > > > > > > postfix/sendmail[32274]: warning: fork: Cannot allocate memory > > > > interesting, what does your /proc/virtual//limits > > file (with the xid of that guest) show? > > [EMAIL PROTECTED]:~# cat /proc/virtual/49196/limit > PROC: 120 266 -1 0 > VM: 407806 50 50 505 > VML: 0 0 -1 0 > RSS:194897 25 25 97 looks like you guest did hit both, the VM and the RSS limits quite a number of times ... (505 and 97 times) > ANON: 181688 182048 -1 0 > FILES: 9361970 -1 0 > OFD: 331 678 -1 0 > LOCKS: 10 25 -1 0 > SOCK: 128 355 -1 0 > MSGQ:0 0 -1 0 > SHM: 0 0 -1 0 > > > > This is on Debian sarge on a 2.6 kernel with vserver v 2.0. The postfix > > > server in question is not busy at all and the vserver has plenty of > > > memory to spare. > > > > > > Any reason to worry here? > > > > well, I would try with a recent patch and see if it > > is still an issues ... > > The latest patch in Debian (unstable) seems to be > patch-2.6.14.3-vs2.0.1-rc5.diff. Is this recent enough? yep, that is quite fine, tx micah! > > -- > Roché Compaan > Upfront Systems http://www.upfrontsystems.co.za ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] memory problems?
On Sun, Dec 11, 2005 at 10:28:04PM +0200, Roché Compaan wrote: > On Sun, 2005-12-11 at 11:34 -0800, Alexander Kabanov wrote: > > hi, > > > > i'm having similar errors (I do have limits and scheduler set, using > > rlimits (as, rss, nproc) and scheduler) whenever i do stress testing, > > (overloading mta or web server for example). > > > > during a stress test, some applications die because of no memory > > available or can't fork, some stop with segmentation fault (not able > > to do vserver enter),. when attacking httpd might have httpd > > . > > I am experiencing the same problems. The segfaults and not being able to > enter a vserver from the host *really* worries me. well, if the limits are reached, the guest can not create new processes and/or instantiate more memory ... of course this might lead to program termination and the fact that a guest cannot be entered (as the limits are hard limits). raising them will make it all work again ... in the future we will add soft limits and guarantees too so that the 'hard' limit can be seen as last resort (and to prevent DoS) HTC, Herbert > -- > Roché Compaan > Upfront Systems http://www.upfrontsystems.co.za > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] vserver as an image file mounted using loop
On Sun, Dec 11, 2005 at 11:42:37AM -0800, Alexander Kabanov wrote: > hi, > > is it ok to keep vserver as an image file and then mount it using loop > device? yep, perfectly fine, although you add one kernel indirection and lose the ability to unify the guest with others ... > let say I have vs1.img, vs2.img ... vsN.img, all i want to do just > mount them whenever I need, and do vserver vsN start > > image contains something like > /conf > /root > > and, > /vservers/vs1 (symlink to /mountpoint/vs1/root) > /etc/vservers/vs1 (symlink to /mountpoint/vs1/conf) > > thanks best, Herbert > > --alex > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver
Re: [Vserver] Basic resource limits for a vserver
On Sun, Dec 11, 2005 at 10:02:19PM +0100, Tor Rune Skoglund wrote: > Hi List! > > Anyone care to share their thoughts on how to limit the > reources for a virtual server in the "best" way. > > My requirements basically are the following: > > - No single vserver should be allowed to bring the > other vservers and the host down. (Given that the > other vservers are "behaving" properly.) > - Each vserver should be allowed to use at much > as available of the system resources as long as > the other vservers and the host do not suffer > noticeably from this. > - The host should hopefully never be brought down, and > should always reply easily to ssh administration logins > and commands. > - Disk resources are not important in this setup. (Have > lotsa disk space now, and vservers on separate partitions.) > > Any suggestions of limits settings for the vservers to > achieve this? I'd give about 50-80% of your resources to each guest, which should assure that you can still do something (with the remaining 20%) if a guest starts to act nasty ... best, Herbert > Best regards, > Tor Rune Skoglund > <[EMAIL PROTECTED]> > > > > ___ > Vserver mailing list > Vserver@list.linux-vserver.org > http://list.linux-vserver.org/mailman/listinfo/vserver ___ Vserver mailing list Vserver@list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver