Re: [1/4] DST: Distributed storage documentation.
On Dec 10, 2007 12:47 PM, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > diff --git a/Documentation/dst/sysfs.txt b/Documentation/dst/sysfs.txt > new file mode 100644 > index 000..79d79dc > --- /dev/null > +++ b/Documentation/dst/sysfs.txt > @@ -0,0 +1,30 @@ > +This file describes sysfs files created for each storage. > + > +1. Per-storage files. > +Each storage has its own dir /sysfs/devices/$storage_name, It's always /sys/devices/. > +which contains following files: > + > +alg - contains name of the algorithm used to created given storage > +name - name of the storage > +nodes - map of the storage (list of nodes and their sizes and starts) > +remove_all_nodes - writable file which allows to remove all nodes from given > + storage > +n-$start-$cookie - per node directory, where > + $start - start of the given node in sectors, > + $cookie - unique node's id used by DST > + > +2. Per-node files. > +Node's files are located in /sysfs/devices/$storage_name/n-$start-$cookie > +directory, described above. To which class or bus do the devices you create belong? Care to show a "tree" or "ls -la" of the device? Kay -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [1/4] DST: Distributed storage documentation.
On Mon, 2007-12-10 at 15:58 +0300, Evgeniy Polyakov wrote: > On Mon, Dec 10, 2007 at 01:51:43PM +0100, Kay Sievers ([EMAIL PROTECTED]) > wrote: > > On Dec 10, 2007 12:47 PM, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > > > diff --git a/Documentation/dst/sysfs.txt b/Documentation/dst/sysfs.txt > > > new file mode 100644 > > > index 000..79d79dc > > > --- /dev/null > > > +++ b/Documentation/dst/sysfs.txt > > > @@ -0,0 +1,30 @@ > > > +This file describes sysfs files created for each storage. > > > + > > > +1. Per-storage files. > > > +Each storage has its own dir /sysfs/devices/$storage_name, > > > > It's always /sys/devices/. > > I meant that for each new device, it will be placed into > /sys/devices/its_name, but it can also be accessed via > /sys/bus/dst/devices/ Still, it looks like a path. :) Please don't reference any device directly with a /sys/devices/ path. You have to use the subsystem links to the devices in /sys/bus/dst/devices/. Devices are free to move around in /sys/devices, even during runtime. Yours don't do, but anyway, please remove all mentioning of direct access to /sys/devices/. Btw, where is the top-level /sys/devices/storage/ coming from? I don't see that in the code. We don't accept any new "virtual parents" here. Your devices will automatically appear in /sys/devices/virtual/dst/, and not below your own parent. But that path does not matter anyway, because you should only access them from the /sys/bus/dst/devices/ directory. And in general please don't claim generic names like "storage" in any namespace for a very specific subsystem like this. > > > +which contains following files: > > > + > > > +alg - contains name of the algorithm used to created given storage > > > +name - name of the storage > > > +nodes - map of the storage (list of nodes and their sizes and starts) > > > +remove_all_nodes - writable file which allows to remove all nodes from > > > given > > > + storage > > > +n-$start-$cookie - per node directory, where > > > + $start - start of the given node in sectors, > > > + $cookie - unique node's id used by DST > > > + > > > +2. Per-node files. > > > +Node's files are located in /sysfs/devices/$storage_name/n-$start-$cookie > > > +directory, described above. > > > > To which class or bus do the devices you create belong? Care to show a > > "tree" or "ls -la" of the device? > > It is 'dst' bus. > > uganda:~/codes# ls -la /sys/devices/staorge/ > total 0 > drwxr-xr-x 4 root root0 2007-12-10 11:46 . > drwxr-xr-x 9 root root0 2007-12-10 11:46 .. > -r--r--r-- 1 root root 4096 2007-12-10 11:46 alg > lrwxrwxrwx 1 root root0 2007-12-10 11:46 bus -> ../../bus/dst > drwxr-xr-x 3 root root0 2007-12-10 11:46 n-0-81003e24117 > -r--r--r-- 1 root root 4096 2007-12-10 11:46 name > -r--r--r-- 1 root root 4096 2007-12-10 11:46 nodes > drwxr-xr-x 2 root root0 2007-12-10 11:46 power > -rw-r--r-- 1 root root 4096 2007-12-10 11:46 remove_all_nodes > lrwxrwxrwx 1 root root0 2007-12-10 11:46 subsystem -> ../../bus/dst > -rw-r--r-- 1 root root 4096 2007-12-10 11:46 uevent Ok, how does: ls -l /sys/devices/storage/n-0-81003e24117 look? > uganda:~/codes# ls -l /sys/bus/dst/ > total 0 > drwxr-xr-x 2 root root0 2007-12-10 09:52 devices > drwxr-xr-x 2 root root0 2007-12-10 09:52 drivers > -rw-r--r-- 1 root root 4096 2007-12-10 11:46 drivers_autoprobe > --w--- 1 root root 4096 2007-12-10 11:46 drivers_probe How does: ls -l /sys/bus/dst/devices look? Further questions: Why do you do your own refcounting instead of using kref? Why don't you use groups for the attributes? Why don't you use default attributes for the device, where you get all error handling done by the core. Kay -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [1/4] DST: Distributed storage documentation.
On Mon, 2007-12-10 at 17:50 +0300, Evgeniy Polyakov wrote: > On Mon, Dec 10, 2007 at 03:31:48PM +0100, Kay Sievers ([EMAIL PROTECTED]) > wrote: > > > I meant that for each new device, it will be placed into > > > /sys/devices/its_name, but it can also be accessed via > > > /sys/bus/dst/devices/ > > > > Still, it looks like a path. :) > > > > Please don't reference any device directly with a /sys/devices/ path. > > You have to use the subsystem links to the devices > > in /sys/bus/dst/devices/. Devices are free to move around > > in /sys/devices, even during runtime. Yours don't do, but anyway, please > > remove all mentioning of direct access to /sys/devices/. > > Ok, I will update documentation to reference /sys/bus/dst/devices > instead of /sys/devices Great, thanks! > > Btw, where is the top-level /sys/devices/storage/ coming from? I don't > > see that in the code. We don't accept any new "virtual parents" here. > > > > Your devices will automatically appear in /sys/devices/virtual/dst/, and > > not below your own parent. But that path does not matter anyway, because > > you should only access them from the /sys/bus/dst/devices/ directory. > > > > And in general please don't claim generic names like "storage" in any > > namespace for a very specific subsystem like this. > > It is not a parent - it is an example for device called 'storage', if it > will be called 'testing', then path will be /sys/devices/testing or more > correct /sys/bus/dst/devices/testing :) Ah, I see. > > > It is 'dst' bus. > > > > > > uganda:~/codes# ls -la /sys/devices/staorge/ > > > total 0 > > > drwxr-xr-x 4 root root0 2007-12-10 11:46 . > > > drwxr-xr-x 9 root root0 2007-12-10 11:46 .. > > > -r--r--r-- 1 root root 4096 2007-12-10 11:46 alg > > > lrwxrwxrwx 1 root root0 2007-12-10 11:46 bus -> ../../bus/dst > > > drwxr-xr-x 3 root root0 2007-12-10 11:46 n-0-81003e24117 > > > -r--r--r-- 1 root root 4096 2007-12-10 11:46 name > > > -r--r--r-- 1 root root 4096 2007-12-10 11:46 nodes > > > drwxr-xr-x 2 root root0 2007-12-10 11:46 power > > > -rw-r--r-- 1 root root 4096 2007-12-10 11:46 remove_all_nodes > > > lrwxrwxrwx 1 root root0 2007-12-10 11:46 subsystem -> ../../bus/dst > > > -rw-r--r-- 1 root root 4096 2007-12-10 11:46 uevent > > > > Ok, how does: > > ls -l /sys/devices/storage/n-0-81003e24117 > > look? > > uganda:~/codes# ls -l /sys/devices/storage/n-0-81003ebc220/ > total 0 > drwxr-xr-x 2 root root0 2007-12-10 13:23 power > -r--r--r-- 1 root root 4096 2007-12-10 13:30 size > -r--r--r-- 1 root root 4096 2007-12-10 13:30 start > -r--r--r-- 1 root root 4096 2007-12-10 13:30 type > -rw-r--r-- 1 root root 4096 2007-12-10 13:30 uevent This is a "struct device" instance without a subsystem (bus/class), right? It will not send an uevent to userspace. Is that intended? Why don't you add them all to the dst bus? > > > uganda:~/codes# ls -l /sys/bus/dst/ > > > total 0 > > > drwxr-xr-x 2 root root0 2007-12-10 09:52 devices > > > drwxr-xr-x 2 root root0 2007-12-10 09:52 drivers > > > -rw-r--r-- 1 root root 4096 2007-12-10 11:46 drivers_autoprobe > > > --w--- 1 root root 4096 2007-12-10 11:46 drivers_probe > > > > How does: > > ls -l /sys/bus/dst/devices > > look? > > uganda:~/codes# ls -la /sys/bus/dst/devices/ > total 0 > drwxr-xr-x 2 root root 0 2007-12-10 13:30 . > drwxr-xr-x 4 root root 0 2007-12-10 13:22 .. > lrwxrwxrwx 1 root root 0 2007-12-10 13:30 storage -> ../../../devices/storage > > Here 'storage' is just a name for device called 'storage', it can be > anything else. Fine. > > Further questions: > > Why do you do your own refcounting instead of using kref? > > That's because I always used atomic operations as a reference counters > and did not tried krefs :) > They are the same actually (module tricky arches where smp_mb_* are > required), so I can replace them in the next release. On Mon, 2007-12-10 at 18:12 +0300, Evgeniy Polyakov wrote: > Actually not - I have to set reference counter to something other than 1 > or +/- 1, and thus will have to call kref_get() in a loop, which is a > very ugly step. Is there kref_set() or somethinglike that? At least not > in 2.6.22 what I'm using for now. Yeah, a loop would look pretty ugly. How about just adding kref_set(), if you need it. > > Why don't you use groups for the attributes? >
Re: [1/4] DST: Distributed storage documentation.
On Mon, 2007-12-10 at 22:33 +0300, Evgeniy Polyakov wrote: > On Mon, Dec 10, 2007 at 08:02:28PM +0100, Kay Sievers ([EMAIL PROTECTED]) > wrote: > > > uganda:~/codes# ls -l /sys/devices/storage/n-0-81003ebc220/ > > > total 0 > > > drwxr-xr-x 2 root root0 2007-12-10 13:23 power > > > -r--r--r-- 1 root root 4096 2007-12-10 13:30 size > > > -r--r--r-- 1 root root 4096 2007-12-10 13:30 start > > > -r--r--r-- 1 root root 4096 2007-12-10 13:30 type > > > -rw-r--r-- 1 root root 4096 2007-12-10 13:30 uevent > > > > This is a "struct device" instance without a subsystem (bus/class), > > right? It will not send an uevent to userspace. Is that intended? Why > > don't you add them all to the dst bus? > > I created dst bus for storage devices only, nodes are very different > objects, and actually they do not need any events from above, but I need > to put some attributes somewhere, so it is 'empty' device. Ok. > > > Actually not - I have to set reference counter to something other than 1 > > > or +/- 1, and thus will have to call kref_get() in a loop, which is a > > > very ugly step. Is there kref_set() or somethinglike that? At least not > > > in 2.6.22 what I'm using for now. > > > > Yeah, a loop would look pretty ugly. How about just adding kref_set(), > > if you need it. > > Well, then it distributed storage will not be able to build as > standalone module, and kref_set() itself will not be accepted as a single > patch, since there are no in-kernel users :) > It is easily doable though. Most rules have exceptions. :) Send a patch, so we can see how it looks like. > > > > Why don't you use groups for the attributes? > > > > > > For 3-4 attributes it is faster to register them in a loop than typing > > > another structure :) > > > > Yeah, but if you would need to recover from an error when the creation > > of a file fails, a group would do the proper "rollback". > > I do not care about such errors - if there is such an error for a file, > which exports information about type of the node (i.e. string "L" or "R") > or some other very meaningful info, then system has enough to care about > instead of this, so dst does not do anything special - it ignores such > errors :) > > On exit path it will be checked and removed correctly. > If there will be additional sysfs files, I think group is a good way to > implement them. > > > > > Why don't you use default attributes for the device, where you get all > > > > error handling done by the core. > > > > > > What is 'default attributes' and for what devices? > > > All my sysfs files are so much trivial, so they do not need anything > > > special and I do not see what is error handling you mentioned. > > > > If all devices of a subsystem (bus/class) are of the same type, you can > > set a default array of attributes in the "struct bus/class" to be > > created at every device. If you have multiple types of devices in the > > same subsytem (bus/class) you can to assign a the "device_type", which > > has the default attribute group. > > That way the core will create the files before the event is sent out to > > userspace, and the files can be access from the event itself. Not sure > > if that is needed for dst. > > Ok, I see. > > DST right now has 3 types of files - storage files, it is common for > every storage device; node files, which are the same for every node; and > per-algorithm private devices - they can be different (actually only > mirroring algorithm exports something to userspace). > > I think it is possible to use default attributes for storage devices, > but node device does not have a bus/class, so they will be untouched. Sounds fine. Thanks, Kay -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [1/4] DST: Distributed storage documentation.
On Mon, 2007-12-10 at 22:51 +0300, Evgeniy Polyakov wrote: > On Mon, Dec 10, 2007 at 08:44:55PM +0100, Kay Sievers ([EMAIL PROTECTED]) > wrote: > > > > > Actually not - I have to set reference counter to something other > > > > > than 1 > > > > > or +/- 1, and thus will have to call kref_get() in a loop, which is a > > > > > very ugly step. Is there kref_set() or somethinglike that? At least > > > > > not > > > > > in 2.6.22 what I'm using for now. > > > > > > > > Yeah, a loop would look pretty ugly. How about just adding kref_set(), > > > > if you need it. > > > > > > Well, then it distributed storage will not be able to build as > > > standalone module, and kref_set() itself will not be accepted as a single > > > patch, since there are no in-kernel users :) > > > It is easily doable though. > > > > Most rules have exceptions. :) Send a patch, so we can see how it looks > > like. > > It looks really non-trivial :) Yeah, it does. :) We miss an EXPORT_SYMBOL(), right? Kay -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [2.6.23 PATCH 13/18] dm: netlink
On 7/13/07, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: On Thu, Jul 12, 2007 at 04:31:15PM -0700, Andrew Morton ([EMAIL PROTECTED]) wrote: > > > Need a dependency on NET there? > > > > It's really sad to make DM dependent on the network layer. > > > > Yes, it would be somewhat sad. However one can presumably continue to use > DM, just without "DM netlink events". On my machine device mapper (lvm, raid) does not work without sockets, (maybe not dm, but hotplug, which creates nodes, or configuration) so it already depends on it. Hotplug depends on networking. You missed the day when everyone started to depend on networking :) Right, uevents (udev) depend on networking. One could still use the /sbin/hotplug fork-bomb, but boxes that don't want networking are often that small, that the amount of events the kernel creates today, leads immediately to OOM, because of too many event processes forked at the same time. Kay - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Network: convert network devices to use struct device instead of class_device
On Thu, 2007-02-08 at 22:59 -0500, Dmitry Torokhov wrote: > On Thursday 08 February 2007 19:56, Greg KH wrote: > > On Thu, Feb 08, 2007 at 12:29:12PM -0500, Dmitry Torokhov wrote: > > > On 2/8/07, Stephen Hemminger <[EMAIL PROTECTED]> wrote: > > > >On Thu, 08 Feb 2007 07:43:18 -0500 > > > >> > Network: convert network devices to use struct device instead of > > > >class_device > > > >> > > > > >> > This lets the network core have the ability to handle > > > >suspend/resume > > > >> > issues, if it wants to. > > > > > > > >It fixes a non-problem. I would like to see the network core > > > >suspend/resume > > > >proposal as well. Last time I examined doing network core suspend help, > > > >the problem was that the physical device suspend was called before the > > > >class device. It is not clear how this change would help. > > > > > > If physical devices are registered before class devices then when > > > suspending class devices are naturally suspended first. It is still > > > not clear to me why we need to convert everythign to struct device, I > > > believe I've shown (with patches) that it is possible to integrate > > > struct class_device into PM framework and avoid reshuffling half of > > > the kernel code. > > > > I don't want to have two separate device trees in the kernel (well, one > > big device tree and a bunch of little class_device trees.) The code > > duplication in the class_device code is just too much, and I get > > questions all the time as to what the differences are. > > > > While duplication of code is a real concern my worry is constant fattening > of struct device. For example most physical devices do not interface > directly with userspace but every single one of them now has dev_t. > Former class_devices do not need suspend/resume early framework either. > And so on, and so forth. The dev_t is a good example for the mess we try to fix here. Not having a dev_t for "devices" lead to the creation of a lot of otherwise completely useless "class devices" which are just a total pain to interpret, and follow the events they create, from userspace. Things like the scsi_device devices, usb_device devices, ... just exist, because only this type of devices was allowed to pass information for device nodes to userspace. Thanks, Kay - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [1/4] DST: Distributed storage documentation.
On Dec 17, 2007 4:03 PM, Evgeniy Polyakov <[EMAIL PROTECTED]> wrote: > +++ b/Documentation/dst/sysfs.txt > @@ -0,0 +1,33 @@ > +This file describes sysfs files created for each storage. > + > +1. Per-storage files. > +Each storage has its own dir /sysfs/devices/$storage_name, > +2. Per-node files. > +Node's files are located in /sysfs/devices/$storage_name/n-$start-$cookie As already pointed out last time, you can't reference /sys/devices/ directly, please use the path from the bus/class directory which points there. Thanks, Kay -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 2.6.24.1 - kernel does not boot; IRQ trouble?
On Feb 18, 2008 9:06 PM, Stephen Hemminger <[EMAIL PROTECTED]> wrote: > On Mon, 18 Feb 2008 19:42:25 + (GMT) > Chris Rankin <[EMAIL PROTECTED]> wrote: > > > --- Stephen Hemminger <[EMAIL PROTECTED]> wrote: > > > > > sysfs: duplicate filename 'bridge' can not be created > > > > > WARNING: at fs/sysfs/dir.c:424 sysfs_add_one() > > > > > Pid: 1, comm: swapper Not tainted 2.6.24.1 #1 > > > > > [] show_trace_log_lvl+0x1a/0x2f > > > > > [] show_trace+0x12/0x14 > > > > > [] dump_stack+0x6c/0x72 > > > > > [] sysfs_add_one+0x57/0xbc > > > > > [] sysfs_create_link+0xc2/0x10d > > > > > [] pci_bus_add_devices+0xbd/0x103 > > > > > [] pci_legacy_init+0x56/0xe3 > > > > > [] kernel_init+0x157/0x2c3 > > > > > [] kernel_thread_helper+0x7/0x10 > > > > > === > > > > > pci :00:01.0: Error creating sysfs bridge symlink, continuing... > > > > > sysfs: duplicate filename 'bridge' can not be created > > > > > WARNING: at fs/sysfs/dir.c:424 sysfs_add_one() > > > > > Pid: 1, comm: swapper Not tainted 2.6.24.1 #1 > > > > > [] show_trace_log_lvl+0x1a/0x2f > > > > > [] show_trace+0x12/0x14 > > > > > [] dump_stack+0x6c/0x72 > > > > > [] sysfs_add_one+0x57/0xbc > > > > > [] sysfs_create_link+0xc2/0x10d > > > > > [] pci_bus_add_devices+0xbd/0x103 > > > > > [] pci_bus_add_devices+0xa5/0x103 > > > > > [] pci_legacy_init+0x56/0xe3 > > > > > [] kernel_init+0x157/0x2c3 > > > > > [] kernel_thread_helper+0x7/0x10 > > > > > === > > > > > > > > I have a vague feeling that this was fixed, perhaps in 2.6.24.x? > > > > > > Never heard of this, what is the initialization script that causes this? > > > Also do you have the SYSFS_DEPRECATED option configured? that caused > > > issues > > > with regular network drivers. > > > > Yes, SYSFS_DEPRECATED is enabled. And the init scripts are from Fedora 8. > > There was a bug (fixed in 2.6.24) that had to do with sysfs_create_link > and SYSFS_DEPRECATED probably there is a similar problem with directories. Chris, could you enable CONFIG_DEBUG_KOBJECT=y, it might show what objects try to claim the same name. Thanks, Kay -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Linux 2.6.24.1 - kernel does not boot; IRQ trouble?
On Tue, Feb 19, 2008 at 9:47 AM, Kay Sievers <[EMAIL PROTECTED]> wrote: > On Feb 18, 2008 9:06 PM, Stephen Hemminger <[EMAIL PROTECTED]> wrote: > > On Mon, 18 Feb 2008 19:42:25 + (GMT) > > Chris Rankin <[EMAIL PROTECTED]> wrote: > > > > > --- Stephen Hemminger <[EMAIL PROTECTED]> wrote: > > > > > > sysfs: duplicate filename 'bridge' can not be created > > > > > > WARNING: at fs/sysfs/dir.c:424 sysfs_add_one() > > > > > > Pid: 1, comm: swapper Not tainted 2.6.24.1 #1 > > > > > > [] show_trace_log_lvl+0x1a/0x2f > > > > > > [] show_trace+0x12/0x14 > > > > > > [] dump_stack+0x6c/0x72 > > > > > > [] sysfs_add_one+0x57/0xbc > > > > > > [] sysfs_create_link+0xc2/0x10d > > > > > > [] pci_bus_add_devices+0xbd/0x103 > > > > > > [] pci_legacy_init+0x56/0xe3 > > > > > > [] kernel_init+0x157/0x2c3 > > > > > > [] kernel_thread_helper+0x7/0x10 > > > > > > === > > > > > > pci :00:01.0: Error creating sysfs bridge symlink, > continuing... > > > > > > sysfs: duplicate filename 'bridge' can not be created > > > > > > WARNING: at fs/sysfs/dir.c:424 sysfs_add_one() Greg, it seems that: arch/x86/pci/legacy.c :: pci_legacy_init() tries to create already created "bridge" symlinks in 2.6.24. So we discover the same devices twice? Can this be a reason for the hang? I guess in 2.6.25, the warning is gone with: commit fd7d1ced29e5beb88c9068801da7a362606d8273 Author: Greg Kroah-Hartman <[EMAIL PROTECTED]> Date: Tue May 22 22:47:54 2007 -0400 PCI: make pci_bus a struct device This moves the pci_bus class device to be a real struct device and at the same time, place it in the device tree in the correct location. Note, the old "bridge" symlink is now gone. Thanks, Kay -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
