Re: [Users] /etc/init.d/service stop
So isn't it a global way to avoid killing the processes of VEs from the host? Nope. A killall would include processes that run in containers. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] fs capacity difference
Why is the fs 37% filled on the hardware node and 75% filled on the container Your quota files are out of date. A very common cause of this, would be copying files directly into /vz/private/3251 from the HN. Copying directly into a VE's directory will bypass the quota calculation. For copying files from the HN into a VE, I use SFTP or similar. Seems silly to SFTP to what's basically localhost, but it does avoid hosing your quotas. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] fs capacity difference
Richard Ray wrote: That is good to know but I did not do that The container is a vzdump restore Oh, okay. Dunno about that one then. How do I get the quota back in sync Shut down the VE. Rename or delete its quota file in /var/vzquota Start it up. The quota will be recalculated as it starts. Restarting to fix quotas is kind of unpleasant, but it's the method I know. Perhaps someone can suggest a way of recalculating the quota file without restarting? -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] High Load
how to know, wich proccess belong to wich container??? Use vzpid Example: To see that PHP process 18798 in your top output: vzpid 18798 A lot of system load may not be visible by 'top' though, particularly disk I/O. For this I use 'atop' which can do disk I/O, example: atop -dl 60 It's similar to top, but the -d means to show disk I/O stats, and the 60 means it will refresh every minute. After the first minute (when it's showing averages since boot) you can see which processes are using the most disk I/O, then perhaps use 'ionice' on them. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] Cannot stop or restart container
Hey there. Thanks for the tip about vzps, Thorsten. We are still in that situation I described a few days ago, with a container which won't stop (operation timed out). vzps shows this. # vzps -E 84 ax VEID PID TTY STAT TIME COMMAND 84 32226 ?Z 0:00 [init defunct] Does this offer any clue to what went wrong, how to prevent it, and how possibly to undo it without rebooting the HN? I understand what zombies/defunct processes are, and that they tend not to go away; but if their hanging around can cause a VE to cease functioning, that's not so great. Any ideas on how we can fix this? -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
[Users] Cannot start container, error -12
Hey guys. What does this mean? CT: 85: failed to start with err=-12 I've run it through Google, and get nothing specific: some folks upgraded their kernel, some folks rebooted and it went away, etc. Any advice on this? -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
[Users] Cannot stop or restart container
Hey guys. I reported a bug some months ago, not being able to stop and restart a container, getting a timeout instead. When I reported the bug months ago, I was unable to replicate it at the time. But here it is once again: I cannot stop or restart the VE. This clip shows the results which I see: # vzctl restart Customer Restarting container Stopping container ... Unable to stop container: operation timed out # vzlist Customer CTID NPROC STATUS IP_ADDR HOSTNAME 84 3 running - Customer The 3 processes, I don't know what they are but they may be related to this, and may be a clue: After the initial vzctl restart, it got down to 1 process before it timed out. I ran vzctl exec Customer ps ax and now I notice that the NPROC increments each time I do this. However from the HN, ps ax | grep ps only shows the one 'ps' which I am running, not the ones in the VE. Any thoughts on how I can finally stop this thing and restart it? Note that this is a production HN with production VEs, so rebooting it is not an attractive option. HN OS is Fedora 9 Kernel is 2.6.24.ovz009.1 vzctl-3.0.23-1.x86_64 vzctl-lib-3.0.23-1.x86_64 -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] Cannot stop or restart container
It gets locked up trying to stop nfsmount. I believe the problem is the virtual nic is down before the nfsmount tries to stop and nfsmount waits forever to close the connection to the server.. Interesting. No connections made here, though; no NFS or similar. Would that apply to open listening sockets as well, e.g. httpd still running? -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
[Users] Cannot start container, error -12
What does this mean? CT: 85: failed to start with err=-12 I see that -12 is ENOMEM. Does this really indicate that my HN is full and can't run anymore VEs? Also, can someone explain to me the proper usage of vzmemcheck and how to interpret its output? -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] question
mattias wrote: Only a short question Are openvz customed to run on rhel / centos and not debian Iven if there are packages for debian Are you asking whether you can run OpenVZ on RHEL and CentOS? The answer is yes. Are you asking whether you can run RHEL and CentOS inside OpenVZ containers? The answer is also yes to CentOS, and I don't know ahout RHEL. I meen Now i run a mail server with openvz Not hard to setup on rhel But on debian On debian it was inpossible How do you mean? You set up OpenVZ, then ran a Debian container, and tried to set up mail in Debian? I couldn't help you there, as I don't know Debian. But I do run a dozen mail servers all under OpenVZ, and can say that OpenVZ does not cause any new problems for hosting email. If you can host a mail server on CentOS, then you can host it on CentOS inside OpenVZ. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] VE affecting all server node.
Good answer, Solar! Some of your ideas may apply to our own situation, as well; we have a MySQL user causing some impact with constant queries. Questions: Can I mount a tmpfs under my simfs? I can't mount the whole VPS as tmpfs nor noatime, but a tmpfs under simfs sounds like a great solution here. If I can then set MySQL's tempfile directory to it, you're right that it could reduce our disk traffic significantly. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] Re: Hosts not responding, kinda urgent
It sounds like the switches they're attached to don't have the right arp information. Kinda what I thought, but why? Make sure proxy_arp is enabled, the sysctl is something like: net.ipv4.conf.eth1.proxy_arp = 1 Yep: net.ipv4.conf.lo.proxy_arp = 0 net.ipv4.conf.all.proxy_arp = 0 net.ipv4.conf.default.proxy_arp = 0 net.ipv4.conf.eth0.proxy_arp = 1 net.ipv4.conf.eth1.proxy_arp = 1 net.ipv4.conf.bond0.proxy_arp = 1 net.ipv4.conf.venet0.proxy_arp = 0 net.ipv4.conf.veth6/0.proxy_arp = 1 net.ipv4.conf.veth9/0.proxy_arp = 1 net.ipv4.conf.veth10/0.proxy_arp = 1 net.ipv4.conf.veth16/0.proxy_arp = 1 net.ipv4.conf.veth21/0.proxy_arp = 1 net.ipv4.conf.veth22/0.proxy_arp = 1 net.ipv4.conf.veth23/0.proxy_arp = 1 net.ipv4.conf.veth24/0.proxy_arp = 1 net.ipv4.conf.veth26/0.proxy_arp = 1 net.ipv4.conf.veth27/0.proxy_arp = 1 net.ipv4.conf.veth76/0.proxy_arp = 1 net.ipv4.conf.veth79/0.proxy_arp = 1 net.ipv4.conf.veth20/0.proxy_arp = 1 -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] Re: Hosts not responding, kinda urgent
I just left an arpsend job running in cron every 5 minutes. I tried that too, called it garpd But it didn't really help and eventually I stopped doing it. I've also had something like this happen when the firewall (wrongly) had an alias that was the same as the IP of a virtual node. Yeah, first thing I checked was the iptables. Even turning iptables off didn't help in this case though. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
[Users] Hosts not responding, kinda urgent
Hey guys. I have 4 VEs, out of 13 on this server, which suddenly stopped being networked. No pings, no HTTP, no nothing -- except from the HN itself. The VEs cannot ping out, no DNS, nothing. I know that no changes were made to the HN, as I'm the only one who works on it. And it's kinda urgent as these are live. :) I have tried comparing the 4 that don't work against the 9 which do work, and am coming up totally blank here. * The VEs are running. All VEs use veths. * Double-checked the route -n and ifconfig output inside the VE. * iptables -- Turned off, I'm positive. iptables -L shows nothing but ACCEPTs. * The route -n output on the HN is perfectly ordinary; proper veths and host-IP entries, no null-routes or other such shenanigans. * I compared sysctl -a | grep vethXX | grep ipv4 outputs, and identical except for the vethXX itself. * Restarted one of the affected VEs. No effect at all. So I'm at a loss here. If the routing and veths are OK, and the sysctl output is identical, why would 4 of them suddenly have ceased functioning, and how can I fix them? Following is an example of sysctl -a | grep ipv4.conf | grep veth16 net.ipv4.conf.veth16/0.forwarding = 1 net.ipv4.conf.veth16/0.mc_forwarding = 0 net.ipv4.conf.veth16/0.accept_redirects = 1 net.ipv4.conf.veth16/0.secure_redirects = 1 net.ipv4.conf.veth16/0.shared_media = 1 net.ipv4.conf.veth16/0.rp_filter = 1 net.ipv4.conf.veth16/0.send_redirects = 1 net.ipv4.conf.veth16/0.accept_source_route = 0 net.ipv4.conf.veth16/0.proxy_arp = 1 net.ipv4.conf.veth16/0.medium_id = 0 net.ipv4.conf.veth16/0.bootp_relay = 0 net.ipv4.conf.veth16/0.log_martians = 0 net.ipv4.conf.veth16/0.tag = 0 net.ipv4.conf.veth16/0.arp_filter = 0 net.ipv4.conf.veth16/0.arp_announce = 0 net.ipv4.conf.veth16/0.arp_ignore = 0 net.ipv4.conf.veth16/0.arp_accept = 0 net.ipv4.conf.veth16/0.disable_xfrm = 0 net.ipv4.conf.veth16/0.disable_policy = 0 net.ipv4.conf.veth16/0.force_igmp_version = 0 net.ipv4.conf.veth16/0.promote_secondaries = 0 The route -n line for its veth: Destination Gateway Genmask Flags Metric RefUse Iface 216.93.173.151 0.0.0.0 255.255.255.255 UH0 00 veth16.0 Any thoughts on further diagnostics? If venets are better, and may solve this, perhaps specific instructions on converting one to a venet and trying again? -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
[Users] Re: Hosts not responding, kinda urgent
Well, guys, here's a weird one. I fixed it, but you won't believe how. I used vzctl enter to enter the VE, then route -n to see its gateway. Then ping the gateway, and it worked; skipped 1-2 pings, but the pings came back and the VEs were once again visible on the Internet. Weird, huh? Thing is, I did them one at a time (after the first one, which I was surprised worked) and it's definitely causative here. I sit there pinging from my PC and from our monitoring server, and nothing nothing nothing; then the instant I send 1 ping to the gateway from within the VPS, it comes up. Four VEs in a row did this. Any ideas as to what could have caused this outage in the first place, and why sending a ping would have fixed it? -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] Re: Hosts not responding, kinda urgent
Any ideas as to what could have caused this outage in the first place, and why sending a ping would have fixed it? I can supply some further info, which may be useful. * The bonding driver is in use, slaving eth0 and eth1 into bond0. * There are 2 switches. eth0 goes into one and eth1 into the other. The two switches have a cross-connect. The Internet uplink is on one of the two switches. Perhaps there's some ARP-related issue happening between the bonding and the switches? * During the time of the outage, arping to the IP address continued working. Then again, this was probably proxy ARP from the HN. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] Host shutdown hangs on vz service stop
That does, indeed, sound very familiar. I just vzcfgvalidate-d my config files and it threw up no problems or errors. :-/ Aww, darn. My thread is here: http://bugzilla.openvz.org/show_bug.cgi?id=1345 It's embarrassing, though a relief, that by the time I had opened the ticket the problem never happened again. We checked vzcfgvalidate, checked vzctl-libs versions, and changed to other kernels but it never happened again. So, we never positively identified a cause and solution. The only changes I had made in previous months were to upgrade the kernel (we use 2.6.24, now at ovz009.1) and to fix the config flaws. So if you're not seeing config issues, maybe a kernel upgrade? They had also suggested updating vzctl and vzctl-libs, and verifying that both packages' versions matched. Though this wasn't our issue, it was one thing they suggested during hunting this bug. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ Linux+ PHP PostgreSQL MySQL DHTML/JavaScript/AJAX No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] Shutdown problems
To clarify further on versions: The HN is 2.6.24 ovz009.1 on Fedora 9. We must use 2.6.24 despite its development status because 2.6.18 lacks support for AMCC/3ware RAID controllers. Aside from this shutdown issue, we have used it for 14 months now under high loads without issue. Aside from this shutdown issue, we consider it stable and production-grade. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ No one cares if you can back up — only if you can recover. ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
[Users] vzctl start yields err=-12
I am having a problem creating and starting a new VE. vzctl start gives me: mounted, container start failed, unmounting. dmesg shows only this: CT: 30: stopped CT: 30: failed to start with err=-12 The verbose log (level 10) is no more useful to me: Starting container ... Running: /usr/sbin/vzquota show 30 Running: /usr/sbin/vzquota on 30 -r 0 -b 104857700 -B 104857700 -i 2100 -I 2100 -e 0 -n 0 -s 0 Mounting root: /vz/root/30 /vz/private/30 Container is mounted Set iptables mask 0x17bf Set features mask / Container start failed Running: /usr/sbin/vzquota stat 30 -f Running: vzquota setlimit 30 -b 104857600 -B 104857600 -i 2000 -I 2000 -e 0 -n 0 Running: /usr/sbin/vzquota stat 30 -f Running: /usr/sbin/vzquota off 30 Container is unmounted It can't possibly be a RAM shortage. This hardware has 24 GB physical, and only 9 is allocated amongst the other VEs. There are presently 8 VEs, and this would make 9 if it would start. Any thoughts on how I can debug this? -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] vzctl start yields err=-12
John Knight wrote: To fix this issue, recompile your kernel with this option commented out: #CONFIG_FAIR_USER_SCHED #CONFIG_FAIR_GROUP_SCHED Thanks a lot for the same-day response, John. I think I can use this tonight on one of our test systems. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] Re: Logging of logins via vzctl enter
Please file the appropriate bug to http://bugzilla.openvz.org/. Please set severity to enhancement and don't use the word ASAP :) Nicely done, Kir! Spoken like a true open-source man. :) -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] veth address persistence
Am I missing some step? Maybe another EXTERNAL_SCRIPT to initialize the ip/routes? Yeppers. http://wiki.openvz.org/Veth#Making_a_veth-device_persistent -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: [Users] VE loses connectivity
in a period of time some of my VE's (both on subnet1 and subnet2) loses connections. I've had a long-standing problem, as well, with HNs and VPSs randomly losing connectivity for seconds at a time. It's been enough to set off Nagios, to delay SMTP, etc. My workaround (I won't call it a solution) was to write a simple gratuitous ARP daemon or garpd This works on the same principle as you discovered: if I retransmit ARP, it solves the problem for a little while. If you want the garpd code, assuming the folks here can't provide a real fix, I'm glad to post it to the list and/or wiki. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
[Users] waiting for lo to become free
Hey guys. I never did get a response, and thought to ask again and to post a status update. My original question: http://www.mail-archive.com/users@openvz.org/msg02202.html I was perplexed, since both bugs which had been known to cause this symptom have been marked as fixed for some time. I was running 2.6.24ovz006.4 and 006.5 on the machines which suffered from this, which were newer than the fixed tickets. Anyway, after this bit me yesterday and I was forced to reboot, the 008.1 kernel came up. It has now been 24 hours, and I was able to successfully restart all VPSs on this one server. Perhaps 008.1 has fixed this. I'll let you know. -- HostGIS, Open Source solutions for the global GIS community Greg Allensworth - SysAdmin, Programmer, GIS Person, Security Network+ Server+ A+ Security+ ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
Re: SV: [Users] Virtuozzo
mattias wrote: I reed virtuozzo only support virtulation with the same os on the physical computer ??? Yep, it's kernel-level virtualization. One kernel runs multiple containers each of which has their own ps and network ports and IP addresses. It is the same OS in that they're all using the very selfsame kernel, but you can run different OS distros, sharing the same kernel. It's very spiffy -- IF you don't have a need to run muliple kernels, multiple OSs, etc. -- Gregor Mosheh / Greg AllensworthBS, A+, Network+, Security+, Server+ System Administrator, Lead Programmer HostGIS development hosting services, http://www.HostGIS.com/ Remember that no one cares if you can back up, only if you can restore. - AMANDA ___ Users mailing list Users@openvz.org https://openvz.org/mailman/listinfo/users
