Re: How can I disable compile optimization in kernel for friendly debugging, Thanks

[Yubin Ruan]

On 2018-04-20 16:08, valdis.kletni...@vt.edu wrote:
> On Fri, 20 Apr 2018 23:39:10 +0800, Yubin Ruan said:
> > On 2018-04-19 13:28, valdis.kletni...@vt.edu wrote:
> > > On Thu, 19 Apr 2018 16:58:40 +0800, sizel said:
> > > >  How can I disable compile optimization in kernel for friendly  
> > > > debugging, Thanks
> > >
> > > First off, there are parts of the kernel that *WILL* explode if you try 
> > > to build
> > > with -O0 - in particular, any code that expects static inlines to be 
> > > treated as
> > > part of the unit they are inlined into for the purposes of 
> > > __builtin_return_address()
> > > and similar.
> >
> > Can you elaborate more on that?
> 
> grep for __builtin_return_address.  Look where it's used. What does it return
> if it's inlined? What does it return if it's called as  not inlined?
> 
> For a simple example, consider this code from lib/smp_processor_id.c: in
> function check_preemption_disabled():
> 
> printk(KERN_ERR "BUG: using %s%s() in preemptible [%08x] code: 
> %s/%d\n",
> what1, what2, preempt_count() - 1, current->comm, 
> current->pid);
> 
> printk("caller is %pS\n", __builtin_return_address(0));
> dump_stack();
> 
> If __builtin_return_address is not inlined, that call points at the printk()
> call. If it *is* inlined, it points at the return point in the function that
> called check_preemption_disabled().

Thanks! I tested it and found that __builtin_return_addres's behavior is
indeed as you described.

>From this, I have a related question, can I treat __builtin_return_address()
as a "builtin" stack unwinder (but only check for return addresses, or, where
a function is called).

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How can I disable compile optimization in kernel for friendly debugging, Thanks

[Yubin Ruan]

On 2018-04-19 13:28, valdis.kletni...@vt.edu wrote:
> On Thu, 19 Apr 2018 16:58:40 +0800, sizel said:
> >  How can I disable compile optimization in kernel for friendly  debugging, 
> > Thanks
> 
> First off, there are parts of the kernel that *WILL* explode if you try to 
> build
> with -O0 - in particular, any code that expects static inlines to be treated 
> as
> part of the unit they are inlined into for the purposes of 
> __builtin_return_address()
> and similar.

Can you elaborate more on that?
 
> Second, modern gdb is perfectly able to deal with -O2 optimization,
> especially if you build with -g.  (In the kernel build, CONFIG_DEBUG_INFO=y
> will do the right thing for this).

I don't think so. According to my experience, GDB is not good enough to deal
with -O2 optimization.

Maybe my gdb is not modern enough? I am using gdb7.11 (Ubuntu 
7.11.1-0ubuntu1~16.5)

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Set Location Counter to a new value in ld linker script

[Yubin Ruan]

On Thu, Apr 05, 2018 at 11:25:23AM +0800, Hao Lee wrote:
> Hi,
> 
> I'm confused about the location counter, especially when setting it to
> a new value in the output section.
> 
> I write a simple program and a linker script to make my question more
> clear. The code is as follows:
> 
> -test.s-
> section .text
> .globl _start
> _start:
> movq $1, %rax
> movq $0, %rbx
> int $0x80
> 
> --test.lds--
> SECTIONS
> {
> . = 0x10;
> label_1 = .;
> custom_section : {
> . = 0x20;
> label_2 = . ;
> label_3 = ABSOLUTE(.) ;
> *(.text) ;
> }
> }
> 
> After linking, use nm command to print symbol addresses:
> 
> 0010 T label_1
> 0030 T label_2
> 0030 A label_3
> 0030 T _start
> 
> I can't understand why lable_2 is 0x30. As the LD documentation
> says[1], if '.' is used inside a section description, it refers to the
> byte offset from the start of that section, not an absolute address.
> In the custom_section, '.' is set to 0x20 which is a relative offset,

So it is relative, and 0x10 + 0x20 = 0x30 .
What is the problem?

Yubin

> so I think label_2 should also be 0x20. The value of label_3 is
> reasonable because it's an absolute address.
>
> Could someone please explain why label_2 is 0x30? Thanks!
> 
> 
> [1] 
> https://sourceware.org/binutils/docs-2.30/ld/Location-Counter.html#Location-Counter
> 
> Regards,
> Hao Lee

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Gmail and mailing list (was Re: Regarding Linux kernel vs Android

[Yubin Ruan]

On Thu, Jan 18, 2018 at 11:11:45PM +0800, Yubin Ruan wrote:
> On Tue, Jan 16, 2018 at 04:24:41PM -0500, valdis.kletni...@vt.edu wrote:
> > On Tue, 16 Jan 2018 22:15:39 +0530, inventsekar said:
> > 
> > > PS - when I send mail to this DL, I thought I would receive my own
> > > mail(similar to Google groups DL), as I am already member of the DL, but I
> > > am not getting my own mail, so I get confusion whether my mail was sent or
> > > not. On the subscription page, I looked for any options, but no luck.
> > > Please suggest
> 
> I am using gmail but never notice the behaviors you stated. Thanks for your
> information. See below:
>  
> > The tl;dr: If you really care, you can have the list send you a "Your 
> > message was posted"
> > confirmation. Go here: 
> > https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> > and login (you'll need the password the list has for you - if you don't 
> > have it, it's
> > in the monthly reminder or you can ask it to send you a copy).
> > 
> > The gory details:
> > 
> > A long known issue with GMail.  Basically, what should happen is this:
> > 
> > 1) You send e-mail. At some point, a Message-ID header gets added, usually 
> > by
> > the "Mail submission server". For the one I'm replying to, the server was 
> > one
> > of Gmail's so it tacked on:
> > 
> > Message-id: 
> > 

Re: Gmail and mailing list (was Re: Regarding Linux kernel vs Android

[Yubin Ruan]

On Tue, Jan 16, 2018 at 04:24:41PM -0500, valdis.kletni...@vt.edu wrote:
> On Tue, 16 Jan 2018 22:15:39 +0530, inventsekar said:
> 
> > PS - when I send mail to this DL, I thought I would receive my own
> > mail(similar to Google groups DL), as I am already member of the DL, but I
> > am not getting my own mail, so I get confusion whether my mail was sent or
> > not. On the subscription page, I looked for any options, but no luck.
> > Please suggest

I am using gmail but never notice the behaviors you stated. Thanks for your
information. See below:
 
> The tl;dr: If you really care, you can have the list send you a "Your message 
> was posted"
> confirmation. Go here: 
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> and login (you'll need the password the list has for you - if you don't have 
> it, it's
> in the monthly reminder or you can ask it to send you a copy).
> 
> The gory details:
> 
> A long known issue with GMail.  Basically, what should happen is this:
> 
> 1) You send e-mail. At some point, a Message-ID header gets added, usually by
> the "Mail submission server". For the one I'm replying to, the server was one
> of Gmail's so it tacked on:
> 
> Message-id: 
> 

Re: Efficient management of emails

[Yubin Ruan]

On Mon, Jan 15, 2018 at 09:41:08PM +0800, Yubin Ruan wrote:
> On Mon, Jan 15, 2018 at 06:04:54PM +0530, Shyam Saini wrote:
> > Hi Everyone,
> > 
> > I have subscribed multiple mailing lists.
> > 
> > My question is  how do kernel developers and other users manage their
> > emails on daily basis
> > considering the fact that we receive hundreds of mails everyday.
> > 
> > One way is to tag each mails with their name for example "NetDev".
> > 
> > I'm curious is their any other way?
> > 
> > I would one really appreciate if someone will share their experience
> > and use case.
> 
> Gmail allows you to filter emails and place them into corresponding mailbox.
> For example, you can filter email by the "list" field and then tag them. For
> emails sent to linux-ker...@vger.kernel.org, I tag them with the tag
> "linux-kernel", and then Gmail will automatically create a folder (mailbox)
> called "linux-kernel" and places all emails from that mailing list there. When
> creating filter, you can also choose to remove those emails from INBOX so that
> they will not appear in two mailbox. In this way, all mails from all mailing
> list get sorted into different mailbox, making things clean.
> 
> And then I download them using offlineimap, and use mutt to view them.

BTW, there is a dedicated doc in the kernel's documentation:

https://www.kernel.org/doc/html/v4.11/process/email-clients.html

And if you want something easy to use without having to configure/tweak, maybe
you should give Thunderbird a try.

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Efficient management of emails

[Yubin Ruan]

On Mon, Jan 15, 2018 at 06:04:54PM +0530, Shyam Saini wrote:
> Hi Everyone,
> 
> I have subscribed multiple mailing lists.
> 
> My question is  how do kernel developers and other users manage their
> emails on daily basis
> considering the fact that we receive hundreds of mails everyday.
> 
> One way is to tag each mails with their name for example "NetDev".
> 
> I'm curious is their any other way?
> 
> I would one really appreciate if someone will share their experience
> and use case.

Gmail allows you to filter emails and place them into corresponding mailbox.
For example, you can filter email by the "list" field and then tag them. For
emails sent to linux-ker...@vger.kernel.org, I tag them with the tag
"linux-kernel", and then Gmail will automatically create a folder (mailbox)
called "linux-kernel" and places all emails from that mailing list there. When
creating filter, you can also choose to remove those emails from INBOX so that
they will not appear in two mailbox. In this way, all mails from all mailing
list get sorted into different mailbox, making things clean.

And then I download them using offlineimap, and use mutt to view them.

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: ReviewRequest of a booklet on System Concepts

[Yubin Ruan]

On Tue, Dec 26, 2017 at 10:20:21AM +0530, Rishi Agrawal wrote:
> Hi All,
> 
> I have written this booklet http://sys.readthedocs.io/en/latest/ explaining
> some concepts related to "glibc and system calls ". This will be useful for
> the newbies who are learning Linux System Programming.
> 
> I request the members of the group to kindly review it once and let me know
> if anything is incorrect or can be written and explained in a better way.
> If someone wants to add more contents he/she can surely do by raising a
> "pull request" in the github repo.
> 
> My apologies if this is not the right group for it as I do not know of any
> other group with this level of expertise.

Good job!

>From the chapter "Setting Up Arguments"[1], it is stated that:

From the objdump we saw that __libc_open was called. This called
__open_nocancel and it had a syscall instruction.

however I check the source code of __libc_open[2] and found that __libc_open()
does not directly call __open_nocancel():

26  /* Open FILE with access OFLAG.  If O_CREAT or O_TMPFILE is in OFLAG,
27 a third argument is the file protection.  */
28  int
29  __libc_open (const char *file, int oflag)
30  {
31int mode;
32  
33if (file == NULL)
34  {
35__set_errno (EINVAL);
36return -1;
37  }
38  
39if (__OPEN_NEEDS_MODE (oflag))
40  {
41va_list arg;
42va_start(arg, oflag);
43mode = va_arg(arg, int);
44va_end(arg);
45  }
46  
47__set_errno (ENOSYS);
48return -1;
49  }
50  libc_hidden_def (__libc_open)
51  weak_alias (__libc_open, __open)
52  libc_hidden_weak (__open)
53  weak_alias (__libc_open, open)

I note previously that there are some scripts used to handle this when
building glibc, but never be able to see how this go through (cannot get its
intermediate form). Do you know how to do that?

Yubin

[1]: https://sys.readthedocs.io/en/latest/doc/06_setting_up_arguements.html
[2]: https://code.woboq.org/userspace/glibc/io/open.c.html

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Linux release & branch management

[Yubin Ruan]

2017-11-16 9:27 GMT+08:00 Anand Arumugam :
> Hello All,
>
> I am looking for some good wiki or documentation about about Linux branches
> and release management. I have read in the web about the benevolent dictator
> model of git is used by Linus. But what I'm looking for is the actual
> logistical details like what are the official blessed repos/branches that
> Linus' lieutenants use and where does the patch submitted by any contributor
> get to and how does it get to, etc. The recent email thread on submitting
> patches to Linus triggered this question for me.

You might want to search for "Linux Kernel Development". Those are
talks from Greg KH, about the Linux kernel development. They are
freely available on Youtube. And here are those slides:
https://github.com/gregkh/kernel-development

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: atomic test and set

[Yubin Ruan]

2017-10-20 16:55 GMT+08:00 Tobin C. Harding <m...@tobin.cc>:
> On Fri, Oct 20, 2017 at 10:22:01PM +0800, Yubin Ruan wrote:
>> On Wed, Oct 18, 2017 at 11:40:44AM +1100, Tobin C. Harding wrote:
>> > Hi,
>> >
>> > Do we have an atomic test and set function in the kernel. I have tried
>> >
>> > const int KEY_FLAG_BIT = 1;
>> >
>> > ...
>> >
>> > static siphash_key_t ptr_secret __read_mostly;
>> > static unsigned long have_key = 0;
>> >
>> > if (test_and_set_bit(KEY_FLAG_BIT, _key))
>> > get_random_bytes(_secret, sizeof(ptr_secret));
>> >
>> >
>> > But that doesn't work.
>> >
>> > I looked in include/linux/atomic.h and thought about using
>> >
>> > static atomic_t have_key = ATOMIC_INIT(0);
>> >
>> > if (atomic_xchg(_key, 1) == 0)
>> > get_random_bytes(_secret, sizeof(ptr_secret));
>> >
>> >
>> > This works. My question is; does this code LOAD the value at have_key and 
>> > STORE the argument on
>> > every call? Or does it LOAD the value, check if it is the same as the 
>> > argument, and STORE _only_ if
>> > it is different?
>>
>> Yes it stores at every call. To check before load, you should use cmpxchg

sorry this should be "check before store" ...

>>
>> > (Is this whole discussion just premature optimization?)
>> >
>> > I cannot grok the macros in atomic.h, they seem circular. Here is the 
>> > macro definitions in call
>> > chain order starting with atomic_xchg()
>>
>> I think those macros look loverly ;-)
>
> Thanks Yubin, guess I need to work on my macro-foo
>
>> Maybe you should look into `Documentation/atomic_ops.txt' for help. Also, I
>> think see how xchg() and cmpxchg() is implemented in the kernel, e.g., at 
>> here:
>>
>> 
>> https://elixir.free-electrons.com/linux/latest/source/arch/x86/include/asm/cmpxchg.h

I really think the `cmpxchg.h' is valuable for learning. Please do
take a look at it.

And sorry again for so many typos.

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: help in the MM Area of the Linux Kernel

[Yubin Ruan]

2017-10-07 0:34 GMT+08:00  :
> On Thu, 05 Oct 2017 18:14:08 +0200, Damian Tometzki said:
>
>> i'am intrested in helping and Bug Fixing in the mm area of the linux
>> kernel.
>>
>> For driver development is it clear check in the staging area the
>> TODO's.
>>
>> And what is the process for other areas of the kernel for example mm
>> (Memory management X86) ?
>
> Rule 1 of kernel hacking:  Not every mechanic gets to work on Formula 1
> engines.
>
> For mm, you'll probably need to show some expertise in other kernel areas,
> *plus* have a deep understanding of memory management theory. That code has
> already been worked over by multiple professionals, which means pretty much 
> all
> the easy stuff has already been done.
>
> Oh, and you're probably going to also need knowledge of the kernel
> instrumentation - perf, tracepoints, and friends.
>
> If you manage to find an actual bug in that code, it is most likely going to 
> be
> some weird corner case, and *much* MM clue will be required to fix it without
> breaking some *other* more common corner case.  Remember that the same code 
> has
> to Do The Right Thing on everything from an embedded system with 32M of RAM 
> and
> only one major process running, to large mainframe class boxes with a terabyte
> of RAM, a large Oracle instance, and several hundred Apache / Tomcat / etc

um...terabyte of RAM? Can you name one of those machine with so much RAM?

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

version number of Linux kernel development

[Yubin Ruan]

Hi, 
I got a question regarding to Linux's version number. According to some talk
from Greg KH, there would be an accompanying stable release along with every
-rc release, that is, something like this:

 4.2.0
  |   \
4.3.1-rc4.2.1
  ||
4.3.2-rc 4.2.2
  ||
4.3.3-rc 4.2.3
  ||
4.3.4-rc 4.2.4
  ||
4.3.5-rc 4.2.5
  ||
4.3.6-rc 4.2.6
  ||
4.3.7-rc 4.2.7   <- this branch is thrown away after 4.4.0 is released
  |
4.4.0

So, in this example, 4.2.7 would be thrown away after 4.4.0 is released. Is
this the same for every major release?

I see a series of 4.4.x release, ranging from 4.4.1 ~ 4.4.49, at 
http://elixir.free-electrons.com/linux/latest/source
which really confused me. Accoding to the model above, there should be only
4.4.7. After that, it would 4.5.x.

Please correct me if I am wrong.

Thanks,
Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Need help to understand Linux kernel source versioning systems

[Yubin Ruan]

2017-07-24 20:56 GMT+08:00 vibnwis :
> Hi there,
>
> Being a newbie, I encountered the following scenarios when tracing
> build_kernel.sh script from eewiki. Would someone help me to understand why
> please? From my experience, I would to summarize the following scenario.
>
> 1. First, source is cloned from
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git and
> checked for release version, "v4.10.17". Obviously it has no kernel tag of
> "v4.10.17", but "v4.10"
>
> 2. As "v4.10.17" does not exist, it tried to pull from the origin/master by
> command, git pull --no-edit
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
> --tags
>
> 3. Kernel tag, of "v4.10.17" is checked being verified again, but failed.
> Hence, now it fetched from  git fetch
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> master --tags
>
> Now, the kernel, of v4.10.17" is found.
>
> My questions are:
>
> 1. One would know that clone of
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git would not
> produce kernel tag of , "v4.10.17", wouldn't it?
>
> 2. Similar by pull it again from
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git with
> option of master --tags would not produce kernel tag, "v4.10.17" as well,
> wouldn' it?
>
> 3. Since fetching with git fetch
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> master --tags would produce v4.10.17, why would not it clone it from
> .../stable/linux-stable,git?

Here is the basic Linux kernel development model: Before every "major" release,
there would be 7 "rc" release, and along with each rc release, there
is a stable release.
For example, let's say we are at 4.2 (or, 4.2.0) now. Before we move
to 4.3, there would
4.3.1-rc, 4.3.2-rc, ..., 4.3.7-rc. After 4.3.7-rc, we have 4.3 (or,
4.3.0). But, rc release is not
stable! So, in order to enable users to use/test the kernel, Greg KH
would maintain a
relatively stable kernel along with each rc release. In the example
above, there would be
4.2.1 along with 4.3.1-rc, 4.2.2 along with 4.3.2-rc, etc. Basically
it is something like this:

 4.2.0
  |  \
4.3.1-rc4.2.1
  ||
4.3.2-rc 4.2.2
  ||
4.3.3-rc 4.2.3
  ||
4.3.4-rc 4.2.4
  ||
4.3.5-rc 4.2.5
  ||
4.3.6-rc 4.2.6
  ||
4.3.7-rc 4.2.7   <- this branch is thrown away after 4.4.0 is released
  |
4.4.0

So, you see, you can only get v4.10.17 from the stable release.

see:
(1) https://github.com/gregkh/kernel-development
(2) https://www.kernel.org/doc/html/v4.11/process/2.Process.html

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: OOM killer hung the whole system

[Yubin Ruan]

2017-07-25 17:59 GMT+08:00 Rock Lee :
> Hi,
>
> I was using a program to test OOM killer, but OOM killer hung my board.  The
> test snippet is very simple, just malloc() without free().
>
>   6 int i=0;
>   7 char *ptrtest=NULL;
>   8  while (1)
>   9  {
>  10ptrtest = malloc(0x1000);
>  11if (ptrtest == NULL)
>  12printf("malloc failed.\n");
>  13  }
>  14  return 0;
>  15
>
> After I executed this program(a.out), there is no reaction of my shell. Only
> this messege was shown:
>
> [  180.138188] Out of memory: Kill process 2706 (qq) score 623 or sacrifice
> child
> [  180.164230] Killed process 2706 (qq) total-vm:98156kB, anon-rss:96400kB,
> file-rss:4kB
>
> I think it must be some problem with OOM killer, and tried a lot of way to
> get some infomation. And I got this:
>
> ===
> Process: a.out, cpu: 0 pid: 1463 start: 0xd5849b00
> =
> Task name: a.out pid: 1463 cpu: 0
> state: 0x2 exit_state: 0x0 stack base: 0xd619
> Stack:
> [] __schedule+0x2c8
> [] squashfs_cache_get+0x108
> [] squashfs_readpage_block+0x28
> [] squashfs_readpage+0x624
> [] __do_page_cache_readahead+0x228
> [] filemap_fault+0x1d4
> [] __do_fault+0x34
> [] do_read_fault+0x19c
> [] handle_mm_fault+0x468
> [] do_page_fault+0x11c
> [] do_PrefetchAbort+0x34
> [] ret_from_exception+0x0
>
> I am suprised, because a.out is in /tmp, there is no relation between a.out
> and squashfs(my rootfs). Could anyone give me a hint ? Is the problem of
> squashfs or OOM killer.

You say your rootfs is squashfs filesystem. Is /tmp using the same
device and same filesystem (i.e., squashfs) ?

Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Is vnode number also limit system-wide number of open file?

[Yubin Ruan]

On Wed, May 31, 2017 at 01:30:59PM -0400, Rik van Riel wrote:
> On Wed, 2017-05-31 at 21:37 +0800, Yubin Ruan wrote:
> > I notice that there is a 
> > 
> > unsigned long   i_ino;
> > 
> > in definition of `struct inode' [1], which is the virtual filesystem
> > inode.
> > Does that mean "inode number" and is it used for indexing in the
> > system-wide
> > inode table? 
> > 
> > If that is the case, would that limit the number of open file in
> > Linux?
> 
> Those numbers are unrelated.
> 
> The i_ino number is the inode number within each
> filesystem, and different filesystems can have
> inodes with the same inode numbers.

Regarding to inode number, I notice that nearly every filesystem has tree
representation of the inode number:
1. on-disk inode number
2. in-memory inode number
3. VFS inode number
How are these related? I mean, if they are all the same, then if filesystem A
and filesystem B both have some identical inode number, the VFS inode numbers
will conflict.
 
Yubin

> File descriptors (open files) point to a struct inode
> somewhere in memory. The same file can be opened many
> times (all programs opening libc.so). Many files will
> not be opened by any program at all.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Is vnode number also limit system-wide number of open file?

[Yubin Ruan]

On Wed, May 31, 2017 at 08:34:09AM +0100, Okash Khawaja wrote:
> 
> 
> > On 31 May 2017, at 14:37, Yubin Ruan <ablacktsh...@gmail.com> wrote:
> > 
> > I notice that there is a 
> > 
> >unsigned longi_ino;
> > 
> > in definition of `struct inode' [1], which is the virtual filesystem inode.
> > Does that mean "inode number" and is it used for indexing in the system-wide
> > inode table? 
> > 
> > If that is the case, would that limit the number of open file in Linux?
> > 
> > I know there *is* such a limit, and superusers can adjust that by
> > /proc/sys/fs/file-max. Currently I cannot raise that to too high, otherwise
> > the system would crash, which I think is because I have limited memory. But,
> > the point is, if I have lots of memory in my machine (say hunderds of
> > Gigabytes), would the number of open file system-wide limited by the `i_ino'
> > above? Since its type is "unsigned long", I guess I can only open
> > 2^(sizeof(unsigned long)) file simultaneously? 
> 
> 2^(8*sizeof(unsigned long))
 
 good catch.

> Which is big. Is there some use case for very large number of files?

No, I just wonder whether this is the limit...

--
Yubin

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Is vnode number also limit system-wide number of open file?

[Yubin Ruan]

I notice that there is a 

unsigned long   i_ino;

in definition of `struct inode' [1], which is the virtual filesystem inode.
Does that mean "inode number" and is it used for indexing in the system-wide
inode table? 

If that is the case, would that limit the number of open file in Linux?

I know there *is* such a limit, and superusers can adjust that by
/proc/sys/fs/file-max. Currently I cannot raise that to too high, otherwise
the system would crash, which I think is because I have limited memory. But,
the point is, if I have lots of memory in my machine (say hunderds of
Gigabytes), would the number of open file system-wide limited by the `i_ino'
above? Since its type is "unsigned long", I guess I can only open
2^(sizeof(unsigned long)) file simultaneously? 

--
Yubin


[1]: 
http://elixir.free-electrons.com/linux/latest/source/include/linux/fs.h#L575

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Kernelnewbies Digest, Vol 69, Issue 18

[Yubin Ruan]

> On Thu, Aug 11, 2016 at 07:25:45PM +, Matt M wrote:
> > I am just getting started with kernel module development, and I can't get
> > anything to compile.
> > 
> > I get the following errors:
> > 
> > In file included from main.c:1:0:
> > /usr/src/linux/include/linux/init.h:142:13: error: expected ?=?, ?,?, ?;?,
> > ?asm? or ?__attribute__? before ?load_default_modules?
> > ?void __init load_default_modules(void);
> > ? ? ? ? ? ? ?^~~~
> > /usr/src/linux/include/linux/init.h:143:12: error: expected ?=?, ?,?, ?;?,
> > ?asm? or ?__attribute__? before ?init_rootfs?
> > ?int __init init_rootfs(void);
> > ? ? ? ? ? ? ^~~
> > /usr/src/linux/include/linux/init.h:285:13: error: expected ?=?, ?,?, ?;?,
> > ?asm? or ?__attribute__? before ?parse_early_param?
> > ?void __init parse_early_param(void);
> > ? ? ? ? ? ? ?^
> > /usr/src/linux/include/linux/init.h:286:13: error: expected ?=?, ?,?, ?;?,
> > ?asm? or ?__attribute__? before ?parse_early_options?
> > ?void __init parse_early_options(char *cmdline);
> > 
> > 
> > The errors go on and on, but the majority look like that.
> > 
> > My code:
> > 
> > #include 
> > #include 
> > 
> > int main() {
> > return 0;
> > }
> > 
> > 
> > The compile command:
> > gcc -I /usr/src/linux/include/ main.c
> 
> That's not how you compile a Linux kernel module, sorry.
> 
> Please read the very good documentation for how to set up a Makefile for
> building a kernel module.  It's in the kernel's Documentation/
> directory.
> 
> best of luck,
> 
> greg k-h
 
Alternatively, try this doc on TLKD:
http://www.tldp.org/LDP/lkmpg/2.6/html/lkmpg.html
it's kind of outdated, but some code there still compile against the newest
kernel.

good luck,
Ruan.


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Questions about the LMA and VMA in a linker script

[Yubin Ruan]

On 2016年07月16日 02:42, Dave Hylands wrote:
>
> On Thu, Jul 14, 2016 at 1:45 AM, Yubin Ruan <ablacktsh...@gmail.com
> <mailto:ablacktsh...@gmail.com>> wrote:
>
> On Wed, Jul 13, 2016 at 7:33 PM, Yubin Ruan
> <ablacktsh...@gmail.com <mailto:ablacktsh...@gmail.com>
> <mailto:ablacktsh...@gmail.com <mailto:ablacktsh...@gmail.com>>>
> wrote:
>
>  Hi,
>  I got some question about the AT directive in linker
> script. I have
>  post this question to binutils{at}sourceware.org
> <http://sourceware.org>
>  <http://sourceware.org> with no reply.
>
>  Hopefully someone can help me out.
>
>  After some searching and asking, I finally know that the AT
> directive
>  tell the linker about LMA of  a section.
>
>  For example there is some linker script like this:
>
>   SECTIONS
>   {
>   . = 0X8010;
>   .text : AT(0x10) {
>   *(.text .stub .text.* .gnu.linkonce.t.*)
>   }
>
> ... blah blah ...
>   }
>
>  Now 0x801 is a VMA, and 0x10 is a LMA.
>
>  My question is, is LMA the same as the physical address in
> a ELF
>  program header ? A typical ELF declaration would be
> something like
>  this:
>
>  typedef struct
>  {
> Elf32_Wordp_type; /* Segment type */
> Elf32_Off p_offset;   /* Segment file
> offset */
> Elf32_Addrp_vaddr;/* Segment virtual
> address */
> Elf32_Addrp_paddr;/* Segment
> physical address */
> Elf32_Wordp_filesz;   /* Segment size in
> file */
> Elf32_Wordp_memsz;/* Segment size in
> memory */
> Elf32_Wordp_flags;/* Segment flags */
> Elf32_Wordp_align;/* Segment
> alignment */
>  } Elf32_Phdr;
>
>  Is LMA just **p_paddr** in the program header?
>
> ...snip...
>
> Thank you for replying. I think I understand what you mean.
> But I still want the answer to my question, that is, is LMA just
> **p_paddr** in the program header?
>
>
> I'm pretty sure that's the case. There are actually 2 sets of headers.
> You can use:
>
> objdump -p foo.elf
>
> to view the "private" headers which shows you the p_vaddr and p_paddr
> fields.
>
> And you can use
>
> objdump -h foo.elf
>
> to view the section headers.
>
> Here's some example output for a typical embedded program:
>
> 2216 >objdump -p firmware.elf
>
> firmware.elf: file format elf32-little
>
> Program Header:
>  LOAD off0x8000 vaddr 0x0800 paddr 0x0800 align 2**15
>   filesz 0x288c memsz 0x288c flags r-x
>  LOAD off0x0001 vaddr 0x0802 paddr 0x0802 align 2**15
>   filesz 0x00040470 memsz 0x00040470 flags r-x
>  LOAD off0x00058000 vaddr 0x2000 paddr 0x08060470 align 2**15
>   filesz 0x0108 memsz 0x64f0 flags rw-
>  LOAD off0x0005e4f0 vaddr 0x200064f0 paddr 0x08060578 align 2**15
>   filesz 0x memsz 0x4000 flags rw-
>  LOAD off0x0005a4f0 vaddr 0x2000a4f0 paddr 0x08060578 align 2**15
>   filesz 0x memsz 0x0800 flags rw-
>
> 2217 >objdump -h firmware.elf
>
> firmware.elf: file format elf32-little
>
> Sections:
> Idx Name  Size  VMA   LMA   File off  Algn
>0 .isr_vector   288c  0800  0800  8000  2**2
>CONTENTS, ALLOC, LOAD, READONLY, CODE
>1 .text 00040470  0802  0802  0001  2**2
>CONTENTS, ALLOC, LOAD, READONLY, CODE
>2 .data 0108  2000  08060470  00058000  2**2
>CONTENTS, ALLOC, LOAD, DATA
>3 .bss  63e8  2108  08060578  00058108  2**2
>ALLOC
>4 .heap 4000  200064f0  08060578  0005e4f0  2**0
>ALLOC
>5 .stack0800  2000a4f0  08060578  0005a4f0  2**0
>ALLOC
>6 .ARM.attributes 0037      00058108  2**0
>CONTENTS, READONLY
>7 .comment  00e0      0005813f  2**0
>CONTENTS, READONLY
>
> --
> Dave Hylands
> Shuswap, BC, Canada
> http://www.davehylands.com

Thanks :-)
Your reply really eliminate my doubts.

Regards,
Ruan.


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Questions about the LMA and VMA in a linker script

[Yubin Ruan]

> On Wed, Jul 13, 2016 at 7:33 PM, Yubin Ruan <ablacktsh...@gmail.com
> <mailto:ablacktsh...@gmail.com>> wrote:
>
> Hi,
> I got some question about the AT directive in linker script. I have
> post this question to binutils{at}sourceware.org
> <http://sourceware.org> with no reply.
> Hopefully someone can help me out.
>
> After some searching and asking, I finally know that the AT directive
> tell the linker about LMA of  a section.
>
> For example there is some linker script like this:
>
>  SECTIONS
>  {
>  . = 0X8010;
>  .text : AT(0x10) {
>  *(.text .stub .text.* .gnu.linkonce.t.*)
>  }
>
>... blah blah ...
>  }
>
> Now 0x801 is a VMA, and 0x10 is a LMA.
>
> My question is, is LMA the same as the physical address in a ELF
> program header ? A typical ELF declaration would be something like
> this:
>
> typedef struct
> {
>Elf32_Wordp_type; /* Segment type */
>Elf32_Off p_offset;   /* Segment file offset */
>Elf32_Addrp_vaddr;/* Segment virtual address */
>Elf32_Addrp_paddr;/* Segment physical address */
>Elf32_Wordp_filesz;   /* Segment size in file */
>Elf32_Wordp_memsz;/* Segment size in memory */
>Elf32_Wordp_flags;/* Segment flags */
>Elf32_Wordp_align;/* Segment alignment */
> } Elf32_Phdr;
>
> Is LMA just **p_paddr** in the program header?
>
> My understanding is, when the linker link all the object files
> together and then output a executable file of ELF format, those LMA
> declare in the linker script would be the **p_paddr** in the
> executable file, so the loader can correspondingly put that program on
> the physical address as declared by **p_paddr**. Is that correct?
> Please correct me if you may. I'm reading some low level code and is
> not really familiar with those low level stuff.
>
> Thanks in advance!
> Ruan.
>
> On 2016, July 14, at 14:07, Dave Hylands wrote:
> Replying to all this time.
>
> Currently, I'm most recently familiar with small non-MMU processors (not
> running linux), and in that case, the VMA is the final address
> (typically in RAM) that the section will be loaded to.
> The LMA is the address (typically in ROM) that the section will be at
> when the program starts execution.
>
> A typical example of this is initialized data. In the image, this
> "initialized data" section will be stored in ROM and copied to RAM by
> the C runtime library before calling main.
>
> In the linux kernel, the kernel image will be linked against its final
> virtual address (the VMA) and the LMA would correspond to the physical
> address that the kernel will be loaded at (since the MMU is typically
> off when the kernel image is loaded).
>
> There are lots of variations and reasons why things might not be exactly
> like I described (different architectures have different conventions),
> but that's the jist of things.
>
>
>
>
> --
> Dave Hylands
> Shuswap, BC, Canada
> http://www.davehylands.com

Thank you for replying. I think I understand what you mean.
But I still want the answer to my question, that is, is LMA just 
**p_paddr** in the program header?

Regards,
Ruan

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Is there any good literature explaining Linux's file system ?

[Yubin Ruan]

 Hi everybody,
 I'm really curious about whether there is any literature that 
describe Linux's file systems(e.g. ext2{3,4}, btrfs) deeply?
 I can't find much literature about Linux's filesystem. I have read 
some papers about Unix's file system. For example, **A Fast File System 
for UNIX**, which explain things in detail(though not so informative for 
me). I also read **Design and Implementation of the Second Extend 
Filesystem** by Remy, Theodore, et al. But those papers and blogs can't 
make thing clear for me, because there are so many things that they have 
not explained.
 I wonder is there any book that explain one of the Linux 
filesystem(design, implementation) in detail, like what Mal Gorman 
explain about Linux Memory Management in his books **Understand The 
Linux Memory Manager** and **Code Commentary On The Linux Virtual Memory 
Manager** ?

Thanks in advance,
Ruan.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies