Re: [CentOS] Server locking up everyday around 3:30 AM - (INFO: task wget:13608 blocked for more than 120 seconds) need sleep, help.
Almost everyday around 3:30 AM the server completely locks up and has to be power cycled before it will come back online. (this means someone hat to wake up and reboot the server, oh how I love being an internet janitor! :) Smells like a hardware issue to me too, but I went through all of the dell diagnostics, updated the firmware, everything checks out as being okay, RAID, disks, RAM, etc... Spent an hour on the phone with a Dell tech. No hardware issues, at least that we were able to find. There are no cron jobs that run at 3:30, no backups, the server has a load of 0, nothing is scheduled around that time... do you have smartd set to run short/long hard disk checks during the night? it is done via /etc/smartd.conf, not via cron. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Server locking up everyday around 3:30 AM
On 03/11/2011 04:06 PM, PJ wrote: snip Interesting entries in /var/log/cron: -snip- (this runs 24/7 every 5 minutes as normal...) Mar 11 02:20:01 web1 crond[12919]: (webuser) CMD (wget -q www.domain.com/cron.php /dev/null 21) snip (fast forward to 3 AM, the same cron job starts getting delayed by 3:27 the server was non responsive. Never seen this before) Mar 11 03:01:01 web1 crond[13613]: (root) CMD (run-parts /etc/cron.hourly) Mar 11 03:07:20 web1 crond[13727]: (webuser) error: Job execution of per-minute job scheduled for 03:05 delayed into subsequent minute 03:07. Skipping job run. Mar 11 03:07:20 web1 crond[13727]: CRON (webuser) ERROR: cannot set security context snip I don't think it is a coincidence I'm seeing CRON (webuser) ERROR: cannot set security context around the same time the server stops responding. I'm not familiar with this message, anyone here seen it? cron daily fires off at 4:02, after all this stuff... nothing in cron.hourly.. Getting warmer I think, but still cant figure it out! OK, did the webuser job run at 03:00:01 or does it start at 03:05:01? The system likely thinks that cron job is still running from the last time it was initiated. (Be it at 03:00:01 or 02:55:01). Is there anything that the php file called by the cron (via wget) is supposed to do at 03:00 that is different than other times? You might try something like this: */5 0,1,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23 * * * current_cron_command 15,30,45 3 * * * current_cron_comamnd (those commands should run the cron normally except starting at 03:00, where it should kick off at 3:15 instead of 03:05) If it also fails to start at 03:15 then that would suggest that something is happening to the cron job the last time it is run to make it hang (or make the system think it is hung). signature.asc Description: OpenPGP digital signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] wich filesystem to store 250E6 small files in same or hashed dire
Hi I need to store about 250.000.000 files. Files are less than 4k. On a ext4 (fedora 14) the system crawl at 10.000.000 in the same directory. I tried to create hash directories, two level of 4096 dir = 16.000.000 but I had to stop the script to create these dir after hours and rm -rf would have taken days ! mkfs was my friend I tried two levels, first of 4096 dir, second of 64 dir. The creation of the hash dir took only few minutes, but copying 1 files make my HD scream for 120s ! I take only 10s when working in the same directory. The filenames are all 27 chars and the first chars can be used to hash the files. My question is : Which filesystem and how to store these files ? Regards -- Alain Spineux | aspineux gmail com Monitor your iT Backups | http://www.magikmon.com Free Backup front-end | http://www.magikmon.com/mksbackup Your email 100% available | http://www.emailgency.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Web server becoming unresponsive : high load
As John says: from where did you get it? Or is this from CentOS-testing? (I didn't follow testing or extras for versions.) I recommend ius for those packages that replace base packages. Kai ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Race condition with mdadm at boot [still mystifying]
On the particular Supermicro motherboard I'm using, there is a very long delay (10 or 15 sec) between power-on and initiation of visible BIOS activity, so all disk drives have ample time to spin up and stabilize. Yeah, I have used Supermicro in the past and they had the same long pause when you turn them on. Good boards, except I had one die recently. I was wondering how many drives total, and how many watts the PSU is? Also, is the controller's firmware up to date? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] wich filesystem to store 250E6 small files in same or hashed dire
I haven't tried it but could you possibly use a database to hold all those files instead? At less than 4K per row, performance from an indexed database might be faster. On 3/12/11, Alain Spineux aspin...@gmail.com wrote: Hi I need to store about 250.000.000 files. Files are less than 4k. On a ext4 (fedora 14) the system crawl at 10.000.000 in the same directory. I tried to create hash directories, two level of 4096 dir = 16.000.000 but I had to stop the script to create these dir after hours and rm -rf would have taken days ! mkfs was my friend I tried two levels, first of 4096 dir, second of 64 dir. The creation of the hash dir took only few minutes, but copying 1 files make my HD scream for 120s ! I take only 10s when working in the same directory. The filenames are all 27 chars and the first chars can be used to hash the files. My question is : Which filesystem and how to store these files ? Regards -- Alain Spineux | aspineux gmail com Monitor your iT Backups | http://www.magikmon.com Free Backup front-end | http://www.magikmon.com/mksbackup Your email 100% available | http://www.emailgency.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] wich filesystem to store 250E6 small files in same or hashed dire
Hi I need to store about 250.000.000 files. Files are less than 4k. On a ext4 (fedora 14) the system crawl at 10.000.000 in the same directory. I tried to create hash directories, two level of 4096 dir = 16.000.000 but I had to stop the script to create these dir after hours and rm -rf would have taken days ! mkfs was my friend I tried two levels, first of 4096 dir, second of 64 dir. The creation of the hash dir took only few minutes, but copying 1 files make my HD scream for 120s ! I take only 10s when working in the same directory. The filenames are all 27 chars and the first chars can be used to hash the files. My question is : Which filesystem and how to store these files ? Did you try XFS? Deletes may be slow but apart from that it did a nice jobs when I last used it. But we had only around 50.000.000 files at the time. However, also ext3 worked quite well after *removing* dir_index. Also, did you run a x86_64 kernel? We were having all kind of troubles with big boxes and i686-PAE kernel, because direntry and inode caches were very small. Simon ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] CentOS MD RAID 1 on Openfiler iSCSI
On 3/2/11, Rudi Ahlers r...@softdux.com wrote: I'm very interested to find out what happened with this project and what you ended up doing? There were delays after changes after delays for that project that was meant to run on the VM setup. Spent more time hacking temporary solutions to their problems with NAS and such. So I haven't actually got this done. But if you can wait, I could probably answer you in about 4 weeks... when the hardware and fibre connectivity for the client's server room comes in. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] What iSCSI is used in Centos 5 and RHEL6?
I was looking up on iSCSI in preparation and became aware that there are different iSCSI software/drivers/whatever-is-the-correct-term available. e.g. IET http://iscsitarget.sourceforge.net/ SCST http://scst.sourceforge.net/ STGT http://stgt.berlios.de/ LIO http://linux-iscsi.org/ Based on what I can see, it seems to be STGT because the site provides a link to Redhat advisory which seems to imply that it's what's being used in RHEL. Does anybody know for sure? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] What iSCSI is used in Centos 5 and RHEL6?
On Sun, Mar 13, 2011 at 02:41:07AM +0800, Emmanuel Noobadmin wrote: I was looking up on iSCSI in preparation and became aware that there are different iSCSI software/drivers/whatever-is-the-correct-term available. e.g. IET http://iscsitarget.sourceforge.net/ SCST http://scst.sourceforge.net/ STGT http://stgt.berlios.de/ LIO http://linux-iscsi.org/ Based on what I can see, it seems to be STGT because the site provides a link to Redhat advisory which seems to imply that it's what's being used in RHEL. Does anybody know for sure? STGT is used in both RHEL5: $ rpm -qi scsi-target-utils Name: scsi-target-utilsRelocations: (not relocatable) Version : 0.0 Vendor: Red Hat, Inc. Release : 5.20080917snap.el5Build Date: Fri 19 Sep 2008 10:33:12 AM PDT Install Date: Wed 28 Jan 2009 08:14:48 AM PST Build Host: hs20-bc1-5.build.redhat.com Group : System Environment/DaemonsSource RPM: scsi-target-utils-0.0-5.20080917snap.el5.src.rpm Size: 274831 License: GPLv2 Signature : DSA/SHA1, Mon 15 Dec 2008 03:01:02 PM PST, Key ID 5326810137017186 Packager: Red Hat, Inc. http://bugzilla.redhat.com/bugzilla URL : http://stgt.berlios.de Summary : The SCSI target daemon and utility programs Description : The SCSI target package contains the daemon and tools to setup a SCSI targets. Currently, software iSCSI targets are supported. and RHEL6: # rpm -qi scsi-target-utils Name: scsi-target-utilsRelocations: (not relocatable) Version : 1.0.4 Vendor: Red Hat, Inc. Release : 3.el6_0.1 Build Date: Tue 18 Jan 2011 02:30:36 PM PST Install Date: Sat 12 Mar 2011 10:42:14 AM PST Build Host: x86-004.build.bos.redhat.com Group : System Environment/DaemonsSource RPM: scsi-target-utils-1.0.4-3.el6_0.1.src.rpm Size: 377828 License: GPLv2 Signature : RSA/8, Mon 24 Jan 2011 10:55:21 PM PST, Key ID 199e2f91fd431d51 Packager: Red Hat, Inc. http://bugzilla.redhat.com/bugzilla URL : http://stgt.sourceforge.net/ Summary : The SCSI target daemon and utility programs Description : The SCSI target package contains the daemon and tools to setup a SCSI targets. Currently, software iSCSI targets are supported. I think LIO has been blessed as the next official version by the kernel folks[1]. Ray [1] https://lwn.net/Articles/424004/ ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] What iSCSI is used in Centos 5 and RHEL6?
Thanks for the confirmation and the note about LIO On 3/13/11, Ray Van Dolson ra...@bludgeon.org wrote: On Sun, Mar 13, 2011 at 02:41:07AM +0800, Emmanuel Noobadmin wrote: I was looking up on iSCSI in preparation and became aware that there are different iSCSI software/drivers/whatever-is-the-correct-term available. e.g. IET http://iscsitarget.sourceforge.net/ SCST http://scst.sourceforge.net/ STGT http://stgt.berlios.de/ LIO http://linux-iscsi.org/ Based on what I can see, it seems to be STGT because the site provides a link to Redhat advisory which seems to imply that it's what's being used in RHEL. Does anybody know for sure? STGT is used in both RHEL5: $ rpm -qi scsi-target-utils Name: scsi-target-utilsRelocations: (not relocatable) Version : 0.0 Vendor: Red Hat, Inc. Release : 5.20080917snap.el5Build Date: Fri 19 Sep 2008 10:33:12 AM PDT Install Date: Wed 28 Jan 2009 08:14:48 AM PST Build Host: hs20-bc1-5.build.redhat.com Group : System Environment/DaemonsSource RPM: scsi-target-utils-0.0-5.20080917snap.el5.src.rpm Size: 274831 License: GPLv2 Signature : DSA/SHA1, Mon 15 Dec 2008 03:01:02 PM PST, Key ID 5326810137017186 Packager: Red Hat, Inc. http://bugzilla.redhat.com/bugzilla URL : http://stgt.berlios.de Summary : The SCSI target daemon and utility programs Description : The SCSI target package contains the daemon and tools to setup a SCSI targets. Currently, software iSCSI targets are supported. and RHEL6: # rpm -qi scsi-target-utils Name: scsi-target-utilsRelocations: (not relocatable) Version : 1.0.4 Vendor: Red Hat, Inc. Release : 3.el6_0.1 Build Date: Tue 18 Jan 2011 02:30:36 PM PST Install Date: Sat 12 Mar 2011 10:42:14 AM PST Build Host: x86-004.build.bos.redhat.com Group : System Environment/DaemonsSource RPM: scsi-target-utils-1.0.4-3.el6_0.1.src.rpm Size: 377828 License: GPLv2 Signature : RSA/8, Mon 24 Jan 2011 10:55:21 PM PST, Key ID 199e2f91fd431d51 Packager: Red Hat, Inc. http://bugzilla.redhat.com/bugzilla URL : http://stgt.sourceforge.net/ Summary : The SCSI target daemon and utility programs Description : The SCSI target package contains the daemon and tools to setup a SCSI targets. Currently, software iSCSI targets are supported. I think LIO has been blessed as the next official version by the kernel folks[1]. Ray [1] https://lwn.net/Articles/424004/ ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Race condition with mdadm at boot
On 03/12/2011 09:00 AM, compdoc wrote: On the particular Supermicro motherboard I'm using, there is a very long delay (10 or 15 sec) between power-on and initiation of visible BIOS activity, so all disk drives have ample time to spin up and stabilize. Yeah, I have used Supermicro in the past and they had the same long pause when you turn them on. Good boards, except I had one die recently. I was wondering how many drives total, and how many watts the PSU is? Also, is the controller's firmware up to date? I agree, Supermicro does design good boards, with a lot of server-grade features that consumer boards lack. Not cheap, but worth the money. I assume the long power-on pause is due to the BIOS silently checking RAM, with the side effect of giving the disks ample time to spin up. I have a total of 15 drives. Three of them are fast 2-1/2 drives used for the host OS filesystems, running on the ICH-10 SATA controller. No problem at all with those. The other 12 drives are 2TB WD Caviar Black series. 6 are connected to the on-board 8-port LSI 1068E chip, and 6 divided equally between 2 Tempo 'Sonnet' 4-port PCIe cards (Marvell chips). All drives have the same 8 partitions each. All drives have the same 8-partition configuration. The LSI controller supports 8 RAID-6 arrays using 5 active drives plus a hot spare. The Tempo controllers support the same arrangement using the other 6 drives. The firmware on everything is up to date. Other than the annoying failure to bind 1 or 2 random partitions on the last drive in each group of 6, the performance of this configuration is quite good. I still have the 'scsi_mod.scan=sync' kernel parameter in place, so there may be some other cause of the failure to bind some of the partitions. FWIW, in more than 10 system boots, there has never been any consistent pattern to these failures. Chuck ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] wich filesystem to store 250E6 small files in same or hashed dire
Thanks to everybody for answering. I thing 250E6 is a lot and keep decent read and write access speed is unreal using mutli-purpose filesystems like ext? and other ?FS. I would need a dedicated filesystem for that. This problem was only a possible solution to another problem. I will solve the original problem using another way. Regards On Sat, Mar 12, 2011 at 3:12 PM, Simon Matter simon.mat...@invoca.ch wrote: Hi I need to store about 250.000.000 files. Files are less than 4k. On a ext4 (fedora 14) the system crawl at 10.000.000 in the same directory. I tried to create hash directories, two level of 4096 dir = 16.000.000 but I had to stop the script to create these dir after hours and rm -rf would have taken days ! mkfs was my friend I tried two levels, first of 4096 dir, second of 64 dir. The creation of the hash dir took only few minutes, but copying 1 files make my HD scream for 120s ! I take only 10s when working in the same directory. The filenames are all 27 chars and the first chars can be used to hash the files. My question is : Which filesystem and how to store these files ? Did you try XFS? Deletes may be slow but apart from that it did a nice jobs when I last used it. But we had only around 50.000.000 files at the time. However, also ext3 worked quite well after *removing* dir_index. Also, did you run a x86_64 kernel? We were having all kind of troubles with big boxes and i686-PAE kernel, because direntry and inode caches were very small. Simon ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- Alain Spineux | aspineux gmail com Monitor your iT Backups | http://www.magikmon.com Free Backup front-end | http://www.magikmon.com/mksbackup Your email 100% available | http://www.emailgency.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] try with succes rpm bind 9.7.3 with koji el4 and el5
rehello centos list after some modifications rpm fedora core 14 of bind-9.7.3 and patching. diff -u unpatched/configure.in patched/configure.in --- unpatched/configure.in 2010-07-05 14:02:20.0 +0200 +++ patched/configure.in 2010-07-05 14:03:48.0 +0200 @@ -282,7 +282,8 @@ AC_C_INLINE AC_C_VOLATILE AC_CHECK_FUNC(sysctlbyname, AC_DEFINE(HAVE_SYSCTLBYNAME)) -AC_C_FLEXIBLE_ARRAY_MEMBER +# RvR: this breaks things on RHEL5 +#AC_C_FLEXIBLE_ARRAY_MEMBER I managed to compile on EL4 and EL5 koji result http://koji.fedoraproject.org/koji/taskinfo?taskID=2907187 http://koji.fedoraproject.org/koji/taskinfo?taskID=2907203 -- gpg --keyserver pgp.mit.edu --recv-key 092164A7 http://pgp.mit.edu:11371/pks/lookup?op=getsearch=0x092164A7 signature.asc Description: Ceci est une partie de message numériquement signée ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] wich filesystem to store 250E6 small files in same or hashed dire
On Sat, Mar 12, 2011 at 4:04 PM, Alain Spineux aspin...@gmail.com wrote: Thanks to everybody for answering. I thing 250E6 is a lot and keep decent read and write access speed is unreal using mutli-purpose filesystems like ext? and other ?FS. I would need a dedicated filesystem for that. This problem was only a possible solution to another problem. I will solve the original problem using another way. Why, exactly, are you doing this? The normal approach for such dense repositories is to create a hierarchy of subdirectories. File aaa12345 goes in $DIR/a/a/a/12345 File abc6789 goes in $DIR/a/b/c/6789 And whatever is accessing or creating the files is taught the algorithm used. This requires some programming up front, but helps prevent precisely the outrageous directory size you describe. Handling, sorting, and reporting on that many files in one directory is an old and painful problem. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Alternatives to apache for php based servers: Nginx
On Sat, Mar 12, 2011 at 6:19 AM, Geoff Galitz ge...@galitz.org wrote: Anyone here using Nginx as alternative to apache in conjunction to php scripts? Care to share thoughts and caveats? Yes, we use nginx + PHP and also apache + nginx + PHP extensively and it works well. You can also look at lighttpd + PHP as another alternative that works well. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos You could also have a look at Hiawatha. The developer claims it's been built with PHP in mind. http://www.hiawatha-webserver.org/ ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Which file system to use for a USB backup
--On Wednesday, March 09, 2011 2:56 PM -0800 Todd Cary t...@aristesoftware.com wrote: I have some photographs on my Centos 4 server that I want to copy to a USB drive. However, I want to be able to access the files from Windows or Mac OS's. Where should I look for instructions on how to mount and format the USB drive and is FAT32 the only option? I don't know about Mac, but you could set up NTFS with Fuse on CentOS to allow you to format and mount it as an NTFS filesystem. You could also format as ext3 and install a filesystem driver on Windows to understand ext3. http://www.fs-driver.org/ Since the Mac is BSD-based, it might even understand ext3. There's this project: http://ext2fsx.sourceforge.net/ I haven't used these, as I haven't needed to export files to another OS this way. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Mumble gamers' VOIP server (murmur)
I was unable to build the whole Mumble system on CentOS but it's available for Fedora Development. I just wanted the server part on my headless server, and a static build is available from the Mumble project on Sourceforge. So I grabbed the Mumble SRPM from Fedora Development, the static build from Sourceforge, and stripped the spec file down to the minimum needed to just install the static-build server. You'll find the result here: http://sewingwitch.com/ken/Stuff/murmur-1.2.3-11.kp.src.rpm If you're paranoid (a good admin should be), just unpack to use the spec file and grab the sources from Fedora Development and Sourceforge and overwrite those included in the SRPM before building. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Web server becoming unresponsive : high load
Hi, Yes Centos-Testing. I've been using for a while (1 year) with no problems. The last ius package that I've used (5.2.15?) kept aborting without any good reason. On Sat, Mar 12, 2011 at 8:31 AM, Kai Schaetzl mailli...@conactive.comwrote: As John says: from where did you get it? Or is this from CentOS-testing? (I didn't follow testing or extras for versions.) I recommend ius for those packages that replace base packages. Kai ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
