Hi Konrad,
Konrad Rzeszutek schrieb:
> On Tue, Apr 28, 2009 at 04:00:08AM +0200, Maddin wrote:
>
>> Hi,
>>
>> i'm using centos with dm-multipath and iscsi as a database system. All
>> packages are normal centos packages from their mirrors.
>>
>> This night I was updating centos to the current release (5.3, previous
>> version was 5.2). So the first thing I have done was stopping the multipathd
>> and iscsi. After this I was doing the normal update stuff (yum
>> check-updates, yum clean all, yum update glibc\*, yum update). During the
>> last yum update the new kernel was installed (2.6.18-128.1.6.el5, previous
>> was 2.6.18-92.1.22.el5).
>>
>> During the installation process of the kernel, the yum process hangs.
>> Looking with ps for the hanging processes, I see that the kernel package was
>> building the initrd. After killing this process and repeating the yum update
>> process (this time only the kernel-package) about 5 times, the same strange
>> failure occurs (process hangs on building initrd). The sixth time I was
>> stracing the whole update process and see that the mkinitrd process does a
>> wait-syscall for sth.. After killing this process again I went down to the
>> server room to reboot the system and try another update process. I reboot
>> the systems, stops multipathd and iscsi and starting the update process. On
>> this update I could see that during the mkinitrd process, a kernel message
>> "device-mapper: failed path x:x:x:x" occurs. Hmm this is strange I thought,
>> because I've stopped multipathd and iscsi, so why means the kernel that a
>> path is failed? Once again killing the mkinitrd process and then starting
>>
>
> The device mapper (a kernel component) still has your multipath entries even
> thought you have
> killed multipathd and iscsi. It doesn't delete them, unless you
> specifically instructs the daemon (multipathd) to do so.
>
> From the standpoint of the kernel, the underlaying block devices got
> yanked out and the "brain" (multipathd) terminated. And any I/O that
> touches that multipath device (such as 'pvscan', 'lvscan' which mkinitrd
> uses to figure out the boot device) would cause the kernel to notice
> that the I/O's aren't going anywhere and mark them as failed.
>
>
>
Ok that sounds logical to me.
>> multipathd and iscsi I've tried another update process. And, yeha, it
>> works???
>>
>
> .. and depending on your multipath.conf file, the I/O can be queued forever
> (queue_if_no_path), or up to a certain time-limit ('no_path_retry'). Or
> error out immediately (if you don't have any of those options set). From
> your experiment it seems that those flags are set and the mkinitrd
> process is in the D state, waiting for the kernel to return a value
> after a read/write system call.
>
> If you had deleted the device mapper entries before terminating
> multipathd and iscsi this shouldn't have happend. Or if you had set the
> multipath configuration to error out immediately you wouldn't hit this.
>
>
I've set queue_if_no_path to all targets. I'll try to delete all path's
and then updating again.
So thanks in advance for your help.
>> I could reproduce this behavior on 2 machines, both centos with standard
>> packages.
>>
>
> Correclty so.
>
>> So can anybody comprehend this "failure" or any ideas?
>>
>
>
>> Cheers
>> Maddin
>>
>>
>>
>>
>>
>>
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To post to this group, send email to open-iscsi@googlegroups.com
To unsubscribe from this group, send email to
open-iscsi+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/open-iscsi
-~----------~----~----~----~------~----~------~--~---
