Re: Kernel-space flirting with user-space
Secondly, I wonder how I can bind a user-space program/daemon with the kernel-space of the module. What about netlink and mmap? Hi Devendra, I glanced at netlink and I really like the idea. I think it would allow me to build the whole thing in kernel-space and a user-space process could simply proxy the packets over TCP/IP, SSL and friends! ;) I also looked at mmap and while it may be used for communication, it seems to be used for many other things. I fear this could be confusing or that documentation will not address my need specifically though. An educated guess tells me it might be more efficient than netlink however. I'll give it a shot later and compare. Thanks, Simon ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel-space flirting with user-space
Hi Henrique,
I browsed the two links and I think I've read parts of the LDD3, but I
could find an answer to my questions. But considering it is sold as a real
book, I might prefer to buy it. I find it easier to read in a book than on
screen, for large texts at least.
And the second link with the howto is seriously impressive! In fact, if
I had found it earlier, I don't think I would have needed to write my OP!
:)
Thank you very much,
Simon
On Thu, Jan 10, 2013 at 10:05 PM, Henrique Rodrigues
henriquesil...@gmail.com wrote:
I think the LDD book is a good starting point: http://lwn.net/Kernel/LDD3/
You can find some examples of kernel/user communication here:
http://people.ee.ethz.ch/~arkeller/linux/kernel_user_space_howto.html
--
Henrique Rodrigues
http://www.dcc.ufmg.br/~hsr
On Thu, Jan 10, 2013 at 1:09 PM, Simon turne...@gmail.com wrote:
Hi there,
I'm rather new to kernel development, but I am somewhat experienced
with Linux, kernel compilation, etc. I'm just passed writing a helloworld
module, dummy filesystem (ramfs wrapper) and proc entry. I would like to
know how communication between kernel user-space is done (the standard)
for a high number of tiny and large messages going in both directions
(large messages would need to be transfered incrementally, in chunks, can
be sequential or random access). I plan on developping a network
filesystem. I would appreciate if someone could clarify where I'm wrong
and confirm where I'm right, and ideally suggest some vocabulary, source
files to lookup, online docs or even books. The minimum you can give will
be most appreciated! =)
Firstly, I was wondering if it would be possible to implement the
filesystem entirely as a kernel module. I would need TCP/UDP sockets. I
think this is really not the recommended approach, but an advantage I see
is it could be used to mount the root filesystem before calling init (that
would be before user-space exists, right?). On the other hand, separating
the work to have a program/daemon in user-space do the communication and
processing would allow me to write that part in C++ (which I personally
prefer).
Secondly, I wonder how I can bind a user-space program/daemon with
the kernel-space of the module.
Using procfs: It seems technically possible to achieve my goal with
it, but I feel this would not be the standard approach. I would rather
keep it for live config live status reporting.
Using sysfs: I've read too much misleading information that I'm not
sure anymore what it's for or even if it's still used or deprecated! Can
someone clarify? It looks technically identical to procfs, but it's
mission is a different one, correct?
Using pipes: This may be a good way, but again, I'm not sure if it's
the standard way.
Using IPC: As far as I understand, IPC would be the way if we'd be
talking of two threads within the same process, or at least within
kernel-space. Though I am likely to be wrong.
Using a char device: It seems technically possible as well, but it may
be difficult to deal with if I ever have more than one user-space process.
Using a block device: My file system works with files and their
metadata, but not with blocks, so this is not suitable. Though it might be
a nice experiment to build a network block device containing an ext2
filesystem (which I think would be similar to iSCSI).
Using net interface: Not applicable. And an experiment to try to make
it applicable doesn't even seem remotely fun. ;)
I've seen the functions copy_to/from_user() or something like that, but
I haven't seen the user-space counter-part of these. I think this may
actually be the best approach by far, but I really lack info/docs to go
that way. The truly best way might be to use shared memory to avoid
copying data internally, but I haven't looked into that much yet.
Finally, is it possible/correct/standard to do something similar to the
following for the local kernel user-space communication? (I haven't
reviewed the syntax, so take it as some kind of pseudo-code!)
struct foo {...} outgoingFoo;
// foo contains no pointers, so it is of a fixed size always.
fwrite(outgoingFoo, 1, sizeof(outgoingFoo), output);
[...then on the other end...]
if(sizeof(fread( buffer, 1, buffer_size, input);
struct foo * incomingFoo = (struct foo *) buffer;
Regards,
Simon
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel-space flirting with user-space
On Fri, Jan 11, 2013 at 10:32 AM, Kristof Provost kris...@sigsegv.be wrote: On 2013-01-10 16:09:49 (-0500), Simon turne...@gmail.com wrote: Firstly, I was wondering if it would be possible to implement the filesystem entirely as a kernel module. I would need TCP/UDP sockets. I think this is really not the recommended approach, but an advantage I see is it could be used to mount the root filesystem before calling init (that would be before user-space exists, right?). On the other hand, separating the work to have a program/daemon in user-space do the communication and processing would allow me to write that part in C++ (which I personally prefer). It's certainly possible to implement a network file system entirely in kernel space. That's where NFS is implemented. If you *really* care about performance it's pretty much the only way to go. If you don't you should keep as much as possible in user space. In either case I'd recommend you start your development on top of fuse though. That means you can implement all new code in user space, in whatever language you prefer. It'll be much easier to debug in any case. Think of it as a fast prototype, to validate your protocol. Hi Kristof, thanks for your reply. On my personnal projects, when starting from scratch, I usually set the goal to refuse optimization the code or the goal to write inefficient code. This is mostly a psychological barrier to avoid premature optimization, which I am prone to. However, efficiency is a goal for the later stages and so writing the whole thing in kernel space from the start might be a good idea (less stuff to rewrite or to fit in). About using FUSE. I have to say your arguments for its use are compelling. However, one of my hidden goals is to learn and practice kernel programming and if I'd use fuse, I'd be constrained to fuse programming which is also restricted to filesystems. The day I want to build something else I would have to start from square one with kernel programming. Also, another project I have in mind after this one is to implement the same concept, but using a block device instead (I think this comes very close to what iSCSI does). It would enable the use of RAID or LVM on top of that networked block device. I don't think this will be more efficient than my current project (specially for use on the internet), but it might be useful in other ways (for backups perhaps, or a raid mirror of local device with networked device tuned with --write-mostly and --write-behind). At least, mdadm can be used to handle disconnects/failures, to resync the devices, to manage what remote hosts are used. Anyway, this one is still a very foggy idea in my mind; it has the word perhaps stamped in red across it! ;) I'll weight the pros and cons for using fuse when I feel ready to start development. Thanks again, Simon ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel-space flirting with user-space
On 2013-01-10 16:09:49 (-0500), Simon turne...@gmail.com wrote: Firstly, I was wondering if it would be possible to implement the filesystem entirely as a kernel module. I would need TCP/UDP sockets. I think this is really not the recommended approach, but an advantage I see is it could be used to mount the root filesystem before calling init (that would be before user-space exists, right?). On the other hand, separating the work to have a program/daemon in user-space do the communication and processing would allow me to write that part in C++ (which I personally prefer). It's certainly possible to implement a network file system entirely in kernel space. That's where NFS is implemented. If you *really* care about performance it's pretty much the only way to go. If you don't you should keep as much as possible in user space. In either case I'd recommend you start your development on top of fuse though. That means you can implement all new code in user space, in whatever language you prefer. It'll be much easier to debug in any case. Think of it as a fast prototype, to validate your protocol. Kristof ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Kernel-space flirting with user-space
Hi there,
I'm rather new to kernel development, but I am somewhat experienced with
Linux, kernel compilation, etc. I'm just passed writing a helloworld
module, dummy filesystem (ramfs wrapper) and proc entry. I would like to
know how communication between kernel user-space is done (the standard)
for a high number of tiny and large messages going in both directions
(large messages would need to be transfered incrementally, in chunks, can
be sequential or random access). I plan on developping a network
filesystem. I would appreciate if someone could clarify where I'm wrong
and confirm where I'm right, and ideally suggest some vocabulary, source
files to lookup, online docs or even books. The minimum you can give will
be most appreciated! =)
Firstly, I was wondering if it would be possible to implement the
filesystem entirely as a kernel module. I would need TCP/UDP sockets. I
think this is really not the recommended approach, but an advantage I see
is it could be used to mount the root filesystem before calling init (that
would be before user-space exists, right?). On the other hand, separating
the work to have a program/daemon in user-space do the communication and
processing would allow me to write that part in C++ (which I personally
prefer).
Secondly, I wonder how I can bind a user-space program/daemon with the
kernel-space of the module.
Using procfs: It seems technically possible to achieve my goal with it,
but I feel this would not be the standard approach. I would rather keep it
for live config live status reporting.
Using sysfs: I've read too much misleading information that I'm not sure
anymore what it's for or even if it's still used or deprecated! Can
someone clarify? It looks technically identical to procfs, but it's
mission is a different one, correct?
Using pipes: This may be a good way, but again, I'm not sure if it's the
standard way.
Using IPC: As far as I understand, IPC would be the way if we'd be
talking of two threads within the same process, or at least within
kernel-space. Though I am likely to be wrong.
Using a char device: It seems technically possible as well, but it may
be difficult to deal with if I ever have more than one user-space process.
Using a block device: My file system works with files and their
metadata, but not with blocks, so this is not suitable. Though it might be
a nice experiment to build a network block device containing an ext2
filesystem (which I think would be similar to iSCSI).
Using net interface: Not applicable. And an experiment to try to make it
applicable doesn't even seem remotely fun. ;)
I've seen the functions copy_to/from_user() or something like that, but I
haven't seen the user-space counter-part of these. I think this may
actually be the best approach by far, but I really lack info/docs to go
that way. The truly best way might be to use shared memory to avoid
copying data internally, but I haven't looked into that much yet.
Finally, is it possible/correct/standard to do something similar to the
following for the local kernel user-space communication? (I haven't
reviewed the syntax, so take it as some kind of pseudo-code!)
struct foo {...} outgoingFoo;
// foo contains no pointers, so it is of a fixed size always.
fwrite(outgoingFoo, 1, sizeof(outgoingFoo), output);
[...then on the other end...]
if(sizeof(fread( buffer, 1, buffer_size, input);
struct foo * incomingFoo = (struct foo *) buffer;
Regards,
Simon
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel-space flirting with user-space
On Thu, Jan 10, 2013 at 4:09 PM, Simon turne...@gmail.com wrote:
Hi there,
I'm rather new to kernel development, but I am somewhat experienced with
Linux, kernel compilation, etc. I'm just passed writing a helloworld
module, dummy filesystem (ramfs wrapper) and proc entry. I would like to
know how communication between kernel user-space is done (the standard)
for a high number of tiny and large messages going in both directions (large
messages would need to be transfered incrementally, in chunks, can be
sequential or random access). I plan on developping a network filesystem.
I would appreciate if someone could clarify where I'm wrong and confirm
where I'm right, and ideally suggest some vocabulary, source files to
lookup, online docs or even books. The minimum you can give will be most
appreciated! =)
Firstly, I was wondering if it would be possible to implement the
filesystem entirely as a kernel module. I would need TCP/UDP sockets. I
think this is really not the recommended approach, but an advantage I see is
it could be used to mount the root filesystem before calling init (that
would be before user-space exists, right?). On the other hand, separating
the work to have a program/daemon in user-space do the communication and
processing would allow me to write that part in C++ (which I personally
prefer).
Secondly, I wonder how I can bind a user-space program/daemon with the
kernel-space of the module.
Using procfs: It seems technically possible to achieve my goal with it,
but I feel this would not be the standard approach. I would rather keep it
for live config live status reporting.
Using sysfs: I've read too much misleading information that I'm not sure
anymore what it's for or even if it's still used or deprecated! Can someone
clarify? It looks technically identical to procfs, but it's mission is a
different one, correct?
Using pipes: This may be a good way, but again, I'm not sure if it's the
standard way.
Using IPC: As far as I understand, IPC would be the way if we'd be
talking of two threads within the same process, or at least within
kernel-space. Though I am likely to be wrong.
Using a char device: It seems technically possible as well, but it may be
difficult to deal with if I ever have more than one user-space process.
Using a block device: My file system works with files and their metadata,
but not with blocks, so this is not suitable. Though it might be a nice
experiment to build a network block device containing an ext2 filesystem
(which I think would be similar to iSCSI).
Using net interface: Not applicable. And an experiment to try to make it
applicable doesn't even seem remotely fun. ;)
I've seen the functions copy_to/from_user() or something like that, but I
haven't seen the user-space counter-part of these. I think this may
actually be the best approach by far, but I really lack info/docs to go that
way. The truly best way might be to use shared memory to avoid copying data
internally, but I haven't looked into that much yet.
Finally, is it possible/correct/standard to do something similar to the
following for the local kernel user-space communication? (I haven't
reviewed the syntax, so take it as some kind of pseudo-code!)
struct foo {...} outgoingFoo;
// foo contains no pointers, so it is of a fixed size always.
fwrite(outgoingFoo, 1, sizeof(outgoingFoo), output);
[...then on the other end...]
if(sizeof(fread( buffer, 1, buffer_size, input);
struct foo * incomingFoo = (struct foo *) buffer;
Regards,
Simon
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
What about netlink and mmap?
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
