Re: [Toybox] [PATCH] losetup: fix the race.

[Rob Landley]

On 8/6/19 5:08 PM, Ryan Prichard wrote:
> On Tue, Aug 6, 2019 at 1:56 PM Rob Landley  > wrote:
> 
> On 8/5/19 7:40 PM, Ryan Prichard via Toybox wrote:
> > I think this patch fixes a race I noticed, but in practice I was 
> hitting a
> > different race. After LOOP_CTL_GET_FREE had created a new loop device, 
> losetup
> > tried to open /dev/block/loopXXX before the device file existed,
> 
> Why is the ioctl returning before the block device exists?
> 
> 
> I think the /dev/block/loopXXX file is created by a userspace daemon (probably
> ueventd in Android). The ioctl creates the kernel device, then sends an
> asynchronous uevent message to ueventd about the new device. ueventd and 
> losetup
> then race each other to create the file and to open it.

So wait, you're _not_ using devtmpfs?
> Summarizing the util-linux workaround:
>  - Make 16 attempts to open the device file.
>  - If the open fails with EACCES or ENOENT, sleep for 25ms, then try again.

Add a workaround for not using devtmpfs. Right...

Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] [PATCH] losetup: fix the race.

[Rob Landley]

On 8/7/19 10:59 AM, enh wrote:
> and if the next stage in your personal "surely this can't be true?"
> journey is the same as mine, the answer i got from our init maintainer
> for the "why?" is that the kernel doesn't want to have an opinion on
> permissions etc. (they pointed out that Android's setup actually
> reduces the size of the window compared to the desktop.)

I remember the lwn.net argument, but I thought they _did_ merge default
permissions in? (I remember the argument but not the resolution...)

Looks like permissions went in: https://lwn.net/Articles/353849/

Ah, I see what happened to ownership:

https://lwn.net/Articles/546464/

It was submitted to the kernel by a bad messenger:

https://www.theregister.co.uk/2014/04/05/torvalds_sievers_dust_up/

(Who of course went on to become one of the core devs of systemd. Which is why
I'm using devuan.)

What the kernel devs probably _really_ wanted was some way the system init
script could echo "123:456" > /sys/class/misc/loop-control/owner" (before
mounting devtmpfs) and then the kernel would remember the ownership it should
create the devices with and do so without a race at runtime, BUT it would have
been provided by userspace rather than hardwired into the kernel.

> iirc, the adb maintainer has similar problems where when you first
> connect an Android device to a Linux host, it shows up but with the
> wrong permissions which then fix themselves if you wait+retry. which
> for my money is even worse. (but maybe not. there are pros and cons
> either way, and neither is ideal. and i'm guessing that Ryan's mention
> of EACCES means that he's seen that problem with losetup too.)

Alas back when half the linux devs were in the pay of IBM they intentionally
sabotaged a lot of the non-hotplug ways of doing things. That's why /dev/hda got
deprecated and there's no /dev/usb0 devices but instead they throw everything
into /dev/sd* and give it a big stir so it's all asynchronously mounted in
heisenberg order that can differ from boot to boot.

The theory was to force the Linux workstation users to solve the device naming
problems the S390 guys were having. In practice nothing was ever solved (when
Ingo Molnar switched jobs they let usb devices reliably always come after sata
devices due to stable driver load order and the ability to disable asynchronous
probing, which solved 80% of it), but it did push Linux on the Desktop back
another 5 years until the "windows vista _really_ sucks, isn't there _anything_
else?" window had closed.

*shrug* I was kind of angry about it at the time. Now I'm expecting Linux to age
out and a new OS to replace it someday...

> anyway, ping on the patch (which is actually for a completely
> different EBUSY race condition)?

I'm still catching up from unexpectedly buying my first ssd. (Well, first in my
laptop. I don't trust any technology that self-destructs from overuse, but at
least this one has a 3 year manufacturer's warantee so that's when I can expect
to schedule its replacement I guess. I _regularly_ throw my laptop into
swap-thrashing and if I can't _tell_ I'm doing it, but that usage pattern will
burn out the ssd in 3 months...)

> i'll see if i can reproduce this EACCES/ENOENT race condition when i
> have some spare time, now i understand the difference...

Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

[Toybox] Toybox in Devuan / Debian. Was: [PATCH] losetup: fix the race.

[David Seikel]

On Thu, 8 Aug 2019 08:10:12 -0500 Rob Landley said :
> On 8/7/19 10:59 AM, enh wrote:
> > and if the next stage in your personal "surely this can't be true?"
> > journey is the same as mine, the answer i got from our init
> > maintainer for the "why?" is that the kernel doesn't want to have
> > an opinion on permissions etc. (they pointed out that Android's
> > setup actually reduces the size of the window compared to the
> > desktop.)  
> 
> I remember the lwn.net argument, but I thought they _did_ merge
> default permissions in? (I remember the argument but not the
> resolution...)
> 
> Looks like permissions went in: https://lwn.net/Articles/353849/
> 
> Ah, I see what happened to ownership:
> 
> https://lwn.net/Articles/546464/
> 
> It was submitted to the kernel by a bad messenger:
> 
> https://www.theregister.co.uk/2014/04/05/torvalds_sievers_dust_up/
> 
> (Who of course went on to become one of the core devs of systemd.
> Which is why I'm using devuan.)

Now that you mention it Devuan has a busybox package, via Debian, but
no toybox package.  If we want to do something about that the question
becomes - do we push it into Debian, or push it into Devuan?  I might
be able to help with the later.

-- 
A big old stinking pile of genius that no one wants
coz there are too many silver coated monkeys in the world.
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] [PATCH] losetup: fix the race.

[Ryan Prichard via Toybox]

On Thu, Aug 8, 2019 at 5:51 AM Rob Landley  wrote:

> On 8/6/19 5:08 PM, Ryan Prichard wrote:
> > On Tue, Aug 6, 2019 at 1:56 PM Rob Landley  > > wrote:
> >
> > On 8/5/19 7:40 PM, Ryan Prichard via Toybox wrote:
> > > I think this patch fixes a race I noticed, but in practice I was
> hitting a
> > > different race. After LOOP_CTL_GET_FREE had created a new loop
> device, losetup
> > > tried to open /dev/block/loopXXX before the device file existed,
> >
> > Why is the ioctl returning before the block device exists?
> >
> >
> > I think the /dev/block/loopXXX file is created by a userspace daemon
> (probably
> > ueventd in Android). The ioctl creates the kernel device, then sends an
> > asynchronous uevent message to ueventd about the new device. ueventd and
> losetup
> > then race each other to create the file and to open it.
>
> So wait, you're _not_ using devtmpfs?
>

The Android devices I've seen all use an ordinary tmpfs on /dev, not
devtmpfs.

(on a walleye device)
$ adb shell mount | grep ' /dev '
tmpfs on /dev type tmpfs
(rw,seclabel,nosuid,relatime,size=1853688k,nr_inodes=463422,mode=755)

-Ryan
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] [PATCH] losetup: fix the race.

[Josh Gao via Toybox]

On Wed, Aug 7, 2019 at 8:59 AM enh via Toybox  wrote:
> iirc, the adb maintainer has similar problems where when you first
> connect an Android device to a Linux host, it shows up but with the
> wrong permissions which then fix themselves if you wait+retry. which
> for my money is even worse. (but maybe not. there are pros and cons
> either way, and neither is ideal. and i'm guessing that Ryan's mention
> of EACCES means that he's seen that problem with losetup too.)

For anyone interested, the race here is that libusb has two implementations for
noticing when USB devices are hotplugged, one that listens to udev over dbus,
and one that listens to the kernel directly over an AF_NETLINK socket. We use
netlink because we can't link against libudev since it isn't API or ABI stable,
and that means we get a notification that a device has arrived before udev has
gotten a chance to chown/chmod its device node.
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] [PATCH] losetup: fix the race.

[enh via Toybox]

On Thu, Aug 8, 2019 at 1:45 PM Ryan Prichard  wrote:
>
> On Thu, Aug 8, 2019 at 5:51 AM Rob Landley  wrote:
>>
>> On 8/6/19 5:08 PM, Ryan Prichard wrote:
>> > On Tue, Aug 6, 2019 at 1:56 PM Rob Landley > > > wrote:
>> >
>> > On 8/5/19 7:40 PM, Ryan Prichard via Toybox wrote:
>> > > I think this patch fixes a race I noticed, but in practice I was 
>> > hitting a
>> > > different race. After LOOP_CTL_GET_FREE had created a new loop 
>> > device, losetup
>> > > tried to open /dev/block/loopXXX before the device file existed,
>> >
>> > Why is the ioctl returning before the block device exists?
>> >
>> >
>> > I think the /dev/block/loopXXX file is created by a userspace daemon 
>> > (probably
>> > ueventd in Android). The ioctl creates the kernel device, then sends an
>> > asynchronous uevent message to ueventd about the new device. ueventd and 
>> > losetup
>> > then race each other to create the file and to open it.
>>
>> So wait, you're _not_ using devtmpfs?
>
>
> The Android devices I've seen all use an ordinary tmpfs on /dev, not devtmpfs.
>
> (on a walleye device)
> $ adb shell mount | grep ' /dev '
> tmpfs on /dev type tmpfs 
> (rw,seclabel,nosuid,relatime,size=1853688k,nr_inodes=463422,mode=755)

i think Rob thought that it would cure your race, but as he seems to
have discovered later (and i remember Tom explaining to me when i
asked him the same question months ago), it just moves the problem
around rather than actually fixing it.

> -Ryan
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] [PATCH] losetup: fix the race.

[Rob Landley]

On 8/8/19 4:07 PM, enh wrote:
>> The Android devices I've seen all use an ordinary tmpfs on /dev, not 
>> devtmpfs.
>>
>> (on a walleye device)
>> $ adb shell mount | grep ' /dev '
>> tmpfs on /dev type tmpfs 
>> (rw,seclabel,nosuid,relatime,size=1853688k,nr_inodes=463422,mode=755)
> 
> i think Rob thought that it would cure your race, but as he seems to
> have discovered later (and i remember Tom explaining to me when i
> asked him the same question months ago), it just moves the problem
> around rather than actually fixing it.

Actually fixing it would be having the startup code do:

  echo "major:minor=uid:gid,major:*=uid:gid..." > 
/sys/module/devtmpfs/owner_mapping

either before mounting devtmpfs or have devtmpfs automatically chown the ones
that aren't "dirty" yet when the mapping changes (since it already tracks that
state for udev), so the kernel would know who devices should belong to (and
could just give everything it hasn't got a mapping for to root), WITHOUT having
policy in kernel space.

Alas, Kay Sievers proposed something stupid instead and got the entire concept
shot down, and nobody ever followed up, and now "this flaw exists therefore we
must retain it" inertia has set in. :(

Anyway, yes race exists today and requires workaround. Got it. Still not caught
up on email from the disk crash though. :)

Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net

Re: [Toybox] Toybox in Devuan / Debian. Was: [PATCH] losetup: fix the race.

[Rob Landley]

On 8/8/19 9:00 AM, David Seikel wrote:
> Now that you mention it Devuan has a busybox package, via Debian, but
> no toybox package.  If we want to do something about that the question
> becomes - do we push it into Debian, or push it into Devuan?  I might
> be able to help with the later.

Devuan is a fairly light reskin of debian, if it goes into one it'll presumably
go into the other before too long.

A .deb packaging of toybox has been on my todo list for a while, if you want to
take a stab at that I'm happy to support you however I can. :)

Rob

___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net