Re: Kernel thread scheduling
And just for the record, I have no financial interest in Coverity or its parent company. I've been a user of their system at a couple of companies I've worked for now, but that and my usage of their free service is the only connection I have with them. Jeff Haran Jeff, FWIW, I have no objection to a financial interest. I want you to make as much many as you can This is not an issue of money. Ruben -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www2.mrbrklyn.com/resources - Unpublished Archive http://www.coinhangout.com - coins! http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013 ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
RE: Kernel thread scheduling
Am 2015-04-16 23:41, schrieb Jeff Haran: I often read emails on this list from people looking to help and get started in kernel development. Well, here's a good place to start. Submit patches to fix some of those Coverity identified kernel bugs. Some of them will be false positives, but some of them will be the real thing. I totally agree. If there are people that do not want to use non-free software for static code analysis (even as a service) they can instead go for smatch[0]. Smatch is part of the 0-day Kernel test infrastructure[1] of Intel and is open source software. I've written a short blog post on how to use smatch here[2]. Any feedback and/or criticism would be appreciated. Cheers, Silvan [0] http://smatch.sourceforge.net/ [1] https://lists.01.org/mailman/listinfo/kbuild [2] http://sillymon.ch/posts/smatchusage.html ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
That is written by my friend GREG!!
I know Greg. I've had many beers with Greg.
Ruben
On Sun, Apr 12, 2015 at 04:36:53AM +, Mohammed Ghriga wrote:
In addition to the suggestions that were offered, I recommend you try reading
Chapter 16: https://www.google.com/search?tbo=ptbm=bksq=isbn:0596554672
(pages: 267-272).
-Original Message-
From: nick [mailto:xerofo...@gmail.com]
Sent: Sunday, April 12, 2015 12:16 AM
To: Ruben Safir; kernelnewbies@kernelnewbies.org; Mohammed Ghriga
Subject: Re: Kernel thread scheduling
On 2015-04-11 11:02 PM, Ruben Safir wrote:
On 04/11/2015 10:21 PM, Ruben Safir wrote:
On 04/10/2015 09:09 AM, nick wrote:
On 2015-04-09 11:37 PM, Ruben Safir wrote:
On 04/09/2015 10:52 PM, nick wrote:
Before asking questions again like this please look into either
using lxr or ctags to navigate the kernel tree for answers as can
be faster then waiting for me or someone else to respond.
well, I reading the text is ctags aren't much value there.
Ctags is useful for searching the code, which is why I am recommending it.
Nick
I have it built into gvim, but you can't use it from a textbook. I'm
finding it is not as useful as it could be for the kernel code.
There are stacks of tags to get around. Another 2 days to learn to
get around tags in vi is not in the agenda right now. It is the tool
I have so I'll have to live with it right now.
I also have a question that is not obvious from the code I'm looking at.
I'm not sure how these structs are attached together. Or more
specifically, I'm not sure how pulling the correct sched_entity gets
one the coresponding task_entity
You have
struct task_struct with a
struct sched_entity
struct sched_enitities are nodes in the RB tree
which are a container for struct rb_node run_node.
So a look at sched_entity ... is in ../linux/sched.h
1161 struct sched_entity {
1162struct load_weight load;/* for load-balancing */
1163struct rb_noderun_node;
1164struct list_head group_node;
1165unsigned int on_rq;
1166
1167u64 exec_start;
1168u64 sum_exec_runtime;
1169u64 vruntime;
1170u64 prev_sum_exec_runtime;
1171
1172u64 nr_migrations;
1173
1174 #ifdef CONFIG_SCHEDSTATS
1175struct sched_statistics statistics;
1176 #endif
1177
1178 #ifdef CONFIG_FAIR_GROUP_SCHED
1179int depth;
1180struct sched_entity *parent;
1181/* rq on which this entity is (to be) queued: */
1182struct cfs_rq *cfs_rq;
1183/* rq owned by this entity/group: */
1184struct cfs_rq *my_q;
1185 #endif
1186
1187 #ifdef CONFIG_SMP
1188/* Per-entity load-tracking */
1189struct sched_avg avg;
1190 #endif
1191 };
I see no means of referencing a specific task from this struct that
forms the node. So when you pull the node with the smallest vruntime
from the left most postion of the RB tree, by calling
pick_next_task(),
static struct sched_entity *__pick_next_entity(struct sched_entity
*se) {
struct rb_node *next = rb_next(se-run_node);
This finds the next node in the red black tree for sched_enities.
Basically rb_next finds the next node in the tree. The argument is the
rb_node structure embedded in the structure using a red black tree.
if (!next)
return NULL;
If there is no runnable task return NULL and pick_next_task will run the
idle_task for this cpu.
return rb_entry(next, struct sched_entity, run_node); }
how do we know what task we are attached to?
Also try to read Chapter 6 of Linux Kernel Development as if have read that
chapter understanding how red black trees and other data structures work in
kernel code would make more sense.
Nick
Ruben
I'm still loss on how we know which taks_struct is being used but as a
side note, I found this also very puzzling
return rb_entry(next, struct sched_entity, run_node);
With help I ran it down to this:
http://lxr.free-electrons.com/source/include/linux/rbtree.h#L50
#define rb_entry(ptr, type, member) container_of(ptr, type, member)
which leads me to yet another macro
798 #define container_of(ptr, type, member) ({ \
799 const typeof( ((type *)0)-member ) *__mptr = (ptr);\
800 (type *)( (char *)__mptr - offsetof(type,member) );})
This is a use of macros I'd never seen before up close. If anyone could
help me understand it, I'd appreciate it.
Ruben
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org
Re: Kernel thread scheduling
looking at it know because I have no clue what that macro is.
It is not listed as possible in kings
On Sun, Apr 12, 2015 at 04:36:53AM +, Mohammed Ghriga wrote:
In addition to the suggestions that were offered, I recommend you try reading
Chapter 16: https://www.google.com/search?tbo=ptbm=bksq=isbn:0596554672
(pages: 267-272).
-Original Message-
From: nick [mailto:xerofo...@gmail.com]
Sent: Sunday, April 12, 2015 12:16 AM
To: Ruben Safir; kernelnewbies@kernelnewbies.org; Mohammed Ghriga
Subject: Re: Kernel thread scheduling
On 2015-04-11 11:02 PM, Ruben Safir wrote:
On 04/11/2015 10:21 PM, Ruben Safir wrote:
On 04/10/2015 09:09 AM, nick wrote:
On 2015-04-09 11:37 PM, Ruben Safir wrote:
On 04/09/2015 10:52 PM, nick wrote:
Before asking questions again like this please look into either
using lxr or ctags to navigate the kernel tree for answers as can
be faster then waiting for me or someone else to respond.
well, I reading the text is ctags aren't much value there.
Ctags is useful for searching the code, which is why I am recommending it.
Nick
I have it built into gvim, but you can't use it from a textbook. I'm
finding it is not as useful as it could be for the kernel code.
There are stacks of tags to get around. Another 2 days to learn to
get around tags in vi is not in the agenda right now. It is the tool
I have so I'll have to live with it right now.
I also have a question that is not obvious from the code I'm looking at.
I'm not sure how these structs are attached together. Or more
specifically, I'm not sure how pulling the correct sched_entity gets
one the coresponding task_entity
You have
struct task_struct with a
struct sched_entity
struct sched_enitities are nodes in the RB tree
which are a container for struct rb_node run_node.
So a look at sched_entity ... is in ../linux/sched.h
1161 struct sched_entity {
1162struct load_weight load;/* for load-balancing */
1163struct rb_noderun_node;
1164struct list_head group_node;
1165unsigned int on_rq;
1166
1167u64 exec_start;
1168u64 sum_exec_runtime;
1169u64 vruntime;
1170u64 prev_sum_exec_runtime;
1171
1172u64 nr_migrations;
1173
1174 #ifdef CONFIG_SCHEDSTATS
1175struct sched_statistics statistics;
1176 #endif
1177
1178 #ifdef CONFIG_FAIR_GROUP_SCHED
1179int depth;
1180struct sched_entity *parent;
1181/* rq on which this entity is (to be) queued: */
1182struct cfs_rq *cfs_rq;
1183/* rq owned by this entity/group: */
1184struct cfs_rq *my_q;
1185 #endif
1186
1187 #ifdef CONFIG_SMP
1188/* Per-entity load-tracking */
1189struct sched_avg avg;
1190 #endif
1191 };
I see no means of referencing a specific task from this struct that
forms the node. So when you pull the node with the smallest vruntime
from the left most postion of the RB tree, by calling
pick_next_task(),
static struct sched_entity *__pick_next_entity(struct sched_entity
*se) {
struct rb_node *next = rb_next(se-run_node);
This finds the next node in the red black tree for sched_enities.
Basically rb_next finds the next node in the tree. The argument is the
rb_node structure embedded in the structure using a red black tree.
if (!next)
return NULL;
If there is no runnable task return NULL and pick_next_task will run the
idle_task for this cpu.
return rb_entry(next, struct sched_entity, run_node); }
how do we know what task we are attached to?
Also try to read Chapter 6 of Linux Kernel Development as if have read that
chapter understanding how red black trees and other data structures work in
kernel code would make more sense.
Nick
Ruben
I'm still loss on how we know which taks_struct is being used but as a
side note, I found this also very puzzling
return rb_entry(next, struct sched_entity, run_node);
With help I ran it down to this:
http://lxr.free-electrons.com/source/include/linux/rbtree.h#L50
#define rb_entry(ptr, type, member) container_of(ptr, type, member)
which leads me to yet another macro
798 #define container_of(ptr, type, member) ({ \
799 const typeof( ((type *)0)-member ) *__mptr = (ptr);\
800 (type *)( (char *)__mptr - offsetof(type,member) );})
This is a use of macros I'd never seen before up close. If anyone could
help me understand it, I'd appreciate it.
Ruben
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http
Re: Kernel thread scheduling
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
How do you search these things out?
Ruben
On Thu, Apr 09, 2015 at 10:52:03PM -0400, nick wrote:
On 2015-04-09 10:12 PM, Ruben Safir wrote:
On 04/09/2015 10:00 PM, nick wrote:
On 2015-04-09 09:51 PM, Ruben Safir wrote:
It is passover so I've read over much of this text, but I have to say
that in general, I'm way ahead of this book. Although I have limited
knowledge of Kernel technology in the specific, the C code, data
structs, and programming concepts are spoon feed in this text and its
wasting too much time with words that are more easily explained with
coding examples and UML charts. I don't need a chapter explaining how
to use ps and the basis of Unix architecture. This text is targeted to
a different audience, and FWIW, I'm not certain it does a good job of
that either. The guys who write these texts fall in love with their own
voices. I know, I've suffered this disease myself when I've written
tech articles and books.
I can''t recommend this book to anyone. Anyone who doesn't understand
the basics of I/O processer blocks is not going to understand
static void update_curr(struct cfs_rq *cfs_rq)
and OTOH void update_curr(struct cfs_rq *cfs_rq) is not explained well
enough for coders unfamiliar with the kernel data structs of which BTW
struct cfs_rq is not one defined in the text.
:(
I'm looking for something more like this, but flushed out more as a
textbook
http://www.ibm.com/developerworks/library/l-completely-fair-scheduler/index.html,
and some mentoring, I hope.
Ruben
Ruben,
The book is for an intro to the kernel not a complete walk through of each
subsystem.
If that is the case,why not read the subsystem code and docs in the
kernel. I am a
novice myself in terms of patch and coding experience but will be glad to
explain the
code as I have read lots of it.
Thank you nick. Yes, I downloaded the entire source from Kernel.org and
it is sitting on both my laptop and in virtual machines where I have
already compiled some stuff and not broken my VMs yets :)
I'm looking over /home/ruben/linux-3.19.3/Documentation/scheduler
[ruben@stat13 scheduler]$ ls
00-INDEX sched-deadline.txt sched-rt-group.txt
media=legal -o sides=two-sided-long-edg sched-design-CFS.txt
sched-stats.txt sched-arch.txt sched-domains.txt
sched-bwc.txt sched-nice-design.txt
I also see in the code there is significant documentation.
Right now I am trying to figure out what is the relationship between
struct sched_entity and
struct cfs_rq and
struct rq_of
why do we have both??
The way that these structures are related is sched_enity is the entity for
the task's scheduling
information or the task_struct of the a excuting process to be exact.
Furthermore cfs_rq is the
runquenue on which the task is running or selected to run on by schedule,the
main scheduling function
of the kernel. Finally here is the function definition for rq_of to answer
your question,
static inline struct rq *rq_of(struct cfs_rq *cfs_rq)
{
return cfs_rq-rq;
}
Before asking questions again like this please look into either using lxr or
ctags
to navigate the kernel tree for answers as can be faster then waiting for me
or
someone else to respond.
Thanks,
Nick
there is a cast in update_curr
u64 now = rq_of(cfs_rq)-clock;
or is rq_of a function that returns a pointer is a struct that I missed?
Nick
On 03/22/2015 08:35 PM, nick wrote:
On 2015-03-22 08:05 PM, Ruben Safir wrote:
On 03/22/2015 07:30 PM, nick wrote:
I would recommend reading Chapters 3 and 4 of Linux Kernel
Development by Robert Love
as when I was learning the scheduler and process management
how much has the scheduler changed since then. It was completely
overhauled when the CFS was created
The 3rd edition of this book was written after CFS was in the kernel so
the chapters
are pretty up to date.
Nick
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
.
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
So many immigrant groups have swept through our town
that Brooklyn, like Atlantis, reaches mythological
Re: Kernel thread scheduling
On Thu, Apr 16, 2015 at 4:56 PM, Ruben Safir ru...@mrbrklyn.com wrote:
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
How do you search these things out?
When everything else fails, I use grep
ricardo@neopili:~/curro/qtec/qt5022/kernel-cesium$ git grep rb_node
{ include/
include/linux/rbtree.h:struct rb_node {
ricardo@neopili:~/curro/qtec/qt5022/kernel-cesium$
--
Ricardo Ribalda
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/16/2015 11:07 AM, Ricardo Ribalda Delgado wrote:
On Thu, Apr 16, 2015 at 4:56 PM, Ruben Safir ru...@mrbrklyn.com wrote:
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
How do you search these things out?
When everything else fails, I use grep
ricardo@neopili:~/curro/qtec/qt5022/kernel-cesium$ git grep rb_node
{ include/
include/linux/rbtree.h:struct rb_node {
ricardo@neopili:~/curro/qtec/qt5022/kernel-cesium$
yeah ...
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
I find that the free electrons LXR has the best search capabilities:
http://lxr.free-electrons.com/source/include/linux/rbtree.h#L35
-M
On Thu, Apr 16, 2015 at 10:56 AM, Ruben Safir ru...@mrbrklyn.com wrote:
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
How do you search these things out?
Ruben
On Thu, Apr 09, 2015 at 10:52:03PM -0400, nick wrote:
On 2015-04-09 10:12 PM, Ruben Safir wrote:
On 04/09/2015 10:00 PM, nick wrote:
On 2015-04-09 09:51 PM, Ruben Safir wrote:
It is passover so I've read over much of this text, but I have to say
that in general, I'm way ahead of this book. Although I have limited
knowledge of Kernel technology in the specific, the C code, data
structs, and programming concepts are spoon feed in this text and its
wasting too much time with words that are more easily explained with
coding examples and UML charts. I don't need a chapter explaining
how
to use ps and the basis of Unix architecture. This text is targeted
to
a different audience, and FWIW, I'm not certain it does a good job of
that either. The guys who write these texts fall in love with their
own
voices. I know, I've suffered this disease myself when I've written
tech articles and books.
I can''t recommend this book to anyone. Anyone who doesn't
understand
the basics of I/O processer blocks is not going to understand
static void update_curr(struct cfs_rq *cfs_rq)
and OTOH void update_curr(struct cfs_rq *cfs_rq) is not explained
well
enough for coders unfamiliar with the kernel data structs of which
BTW
struct cfs_rq is not one defined in the text.
:(
I'm looking for something more like this, but flushed out more as a
textbook
http://www.ibm.com/developerworks/library/l-completely-fair-scheduler/index.html
,
and some mentoring, I hope.
Ruben
Ruben,
The book is for an intro to the kernel not a complete walk through of
each subsystem.
If that is the case,why not read the subsystem code and docs in the
kernel. I am a
novice myself in terms of patch and coding experience but will be
glad to explain the
code as I have read lots of it.
Thank you nick. Yes, I downloaded the entire source from Kernel.org
and
it is sitting on both my laptop and in virtual machines where I have
already compiled some stuff and not broken my VMs yets :)
I'm looking over /home/ruben/linux-3.19.3/Documentation/scheduler
[ruben@stat13 scheduler]$ ls
00-INDEX sched-deadline.txt sched-rt-group.txt
media=legal -o sides=two-sided-long-edg sched-design-CFS.txt
sched-stats.txt sched-arch.txt sched-domains.txt
sched-bwc.txt sched-nice-design.txt
I also see in the code there is significant documentation.
Right now I am trying to figure out what is the relationship between
struct sched_entity and
struct cfs_rq and
struct rq_of
why do we have both??
The way that these structures are related is sched_enity is the entity
for the task's scheduling
information or the task_struct of the a excuting process to be exact.
Furthermore cfs_rq is the
runquenue on which the task is running or selected to run on by
schedule,the main scheduling function
of the kernel. Finally here is the function definition for rq_of to
answer your question,
static inline struct rq *rq_of(struct cfs_rq *cfs_rq)
{
return cfs_rq-rq;
}
Before asking questions again like this please look into either using
lxr or ctags
to navigate the kernel tree for answers as can be faster then waiting
for me or
someone else to respond.
Thanks,
Nick
there is a cast in update_curr
u64 now = rq_of(cfs_rq)-clock;
or is rq_of a function that returns a pointer is a struct that I
missed?
Nick
On 03/22/2015 08:35 PM, nick wrote:
On 2015-03-22 08:05 PM, Ruben Safir wrote:
On 03/22/2015 07:30 PM, nick wrote:
I would recommend reading Chapters 3 and 4 of Linux Kernel
Development by Robert Love
as when I was learning the scheduler and process management
how much has the scheduler changed since then. It was completely
overhauled when the CFS was created
The 3rd edition of this book was written after CFS was in the
kernel so the chapters
are pretty up to date.
Nick
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
.
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
Re: Kernel thread scheduling
On 04/16/2015 11:07 AM, Ricardo Ribalda Delgado wrote:
On Thu, Apr 16, 2015 at 4:56 PM, Ruben Safir ru...@mrbrklyn.com wrote:
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
How do you search these things out?
When everything else fails, I use grep
ricardo@neopili:~/curro/qtec/qt5022/kernel-cesium$ git grep rb_node
{ include/
include/linux/rbtree.h:struct rb_node {
ricardo@neopili:~/curro/qtec/qt5022/kernel-cesium$
although in theory that only works if all your defined files are in your
current subtree
I wish I understood what the tags file says. It is like autoconf. It
can take a lifetime to master the syntax.
Ruben
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
Please use this: Linux 2.6.11 and later http://lxr.linux.no/linux
At the top right there is a text box please type what your searching for,
in this rb_node. It will show you on the right the search results.
The Structure is defined here: include/linux/rbtree.h, line 35
http://lxr.linux.no/linux+*/include/linux/rbtree.h#L35
and here it is in case you have difficulties:
struct rb_node http://lxr.linux.no/linux+*/+code=rb_node { 36
http://lxr.linux.no/linux+*/include/linux/rbtree.h#L36
unsigned long __rb_parent_color
http://lxr.linux.no/linux+*/+code=__rb_parent_color; 37
http://lxr.linux.no/linux+*/include/linux/rbtree.h#L37struct
rb_node http://lxr.linux.no/linux+*/+code=rb_node *rb_right
http://lxr.linux.no/linux+*/+code=rb_right; 38
http://lxr.linux.no/linux+*/include/linux/rbtree.h#L38struct
rb_node http://lxr.linux.no/linux+*/+code=rb_node *rb_left
http://lxr.linux.no/linux+*/+code=rb_left; 39
http://lxr.linux.no/linux+*/include/linux/rbtree.h#L39}
__attribute__ http://lxr.linux.no/linux+*/+code=__attribute__((aligned
http://lxr.linux.no/linux+*/+code=aligned(sizeof(long; 40
http://lxr.linux.no/linux+*/include/linux/rbtree.h#L40/* The
alignment might seem pointless, but allegedly CRIS needs it */
Thank's - Aruna
Ruben
On Thu, Apr 09, 2015 at 10:52:03PM -0400, nick wrote:
On 2015-04-09 10:12 PM, Ruben Safir wrote:
On 04/09/2015 10:00 PM, nick wrote:
On 2015-04-09 09:51 PM, Ruben Safir wrote:
It is passover so I've read over much of this text, but I have to say
that in general, I'm way ahead of this book. Although I have limited
knowledge of Kernel technology in the specific, the C code, data
structs, and programming concepts are spoon feed in this text and its
wasting too much time with words that are more easily explained with
coding examples and UML charts. I don't need a chapter explaining
how
to use ps and the basis of Unix architecture. This text is targeted
to
a different audience, and FWIW, I'm not certain it does a good job of
that either. The guys who write these texts fall in love with their
own
voices. I know, I've suffered this disease myself when I've written
tech articles and books.
I can''t recommend this book to anyone. Anyone who doesn't
understand
the basics of I/O processer blocks is not going to understand
static void update_curr(struct cfs_rq *cfs_rq)
and OTOH void update_curr(struct cfs_rq *cfs_rq) is not explained
well
enough for coders unfamiliar with the kernel data structs of which
BTW
struct cfs_rq is not one defined in the text.
:(
I'm looking for something more like this, but flushed out more as a
textbook
http://www.ibm.com/developerworks/library/l-completely-fair-scheduler/index.html
,
and some mentoring, I hope.
Ruben
Ruben,
The book is for an intro to the kernel not a complete walk through of
each subsystem.
If that is the case,why not read the subsystem code and docs in the
kernel. I am a
novice myself in terms of patch and coding experience but will be
glad to explain the
code as I have read lots of it.
Thank you nick. Yes, I downloaded the entire source from Kernel.org
and
it is sitting on both my laptop and in virtual machines where I have
already compiled some stuff and not broken my VMs yets :)
I'm looking over /home/ruben/linux-3.19.3/Documentation/scheduler
[ruben@stat13 scheduler]$ ls
00-INDEX sched-deadline.txt sched-rt-group.txt
media=legal -o sides=two-sided-long-edg sched-design-CFS.txt
sched-stats.txt sched-arch.txt sched-domains.txt
sched-bwc.txt sched-nice-design.txt
I also see in the code there is significant documentation.
Right now I am trying to figure out what is the relationship between
struct sched_entity and
struct cfs_rq and
struct rq_of
why do we have both??
The way that these structures are related is sched_enity is the entity
for the task's scheduling
information or the task_struct of the a excuting process to be exact.
Furthermore cfs_rq is the
runquenue on which the task is running or selected to run on by
schedule,the main scheduling function
of the kernel. Finally here is the function definition for rq_of to
answer your question,
static inline struct rq *rq_of(struct cfs_rq *cfs_rq)
{
return cfs_rq-rq;
}
Before asking questions again like this please look into either using
lxr or ctags
to navigate the kernel tree for answers as can be faster then waiting
for me or
someone else to respond.
Thanks,
Nick
there is a cast in update_curr
u64 now = rq_of(cfs_rq)-clock;
or is rq_of a function that returns a pointer is a struct that I
missed?
Nick
On 03/22/2015 08:35 PM, nick wrote:
Re: Kernel thread scheduling
On Thu, Apr 16, 2015 at 5:12 PM, Ruben Safir ru...@mrbrklyn.com wrote:
On 04/16/2015 11:07 AM, Ricardo Ribalda Delgado wrote:
When everything else fails, I use grep
ricardo@neopili:~/curro/qtec/qt5022/kernel-cesium$ git grep rb_node
{ include/
include/linux/rbtree.h:struct rb_node {
ricardo@neopili:~/curro/qtec/qt5022/kernel-cesium$
although in theory that only works if all your defined files are in your
current subtree
git grep works recursively
git -r also
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/16/2015 11:10 AM, Aruna Hewapathirane wrote: I'm trying to find rb_node's structure and I can't find it with ctags or in the http://lxr.linux.no website. Please use this: Linux 2.6.11 and later http://lxr.linux.no/linux 91 responses is not a useful search... ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/16/2015 12:31 PM, Jeff Haran wrote: From: kernelnewbies-boun...@kernelnewbies.org [mailto:kernelnewbies-boun...@kernelnewbies.org] On Behalf Of Mark P Sent: Thursday, April 16, 2015 8:12 AM To: Ruben Safir Cc: nick; kernelnewbies@kernelnewbies.org Subject: Re: Kernel thread scheduling I find that the free electrons LXR has the best search capabilities: http://lxr.free-electrons.com/source/include/linux/rbtree.h#L35 -M Those interested in kernel source browsers might want to check out the code browser available at https://scan.coverity.com. Coverity does static code analysis and sells a product to do so, but they do regular scans of popular open source projects. Most of their focus is on finding and reporting defects, but the code browser they have created to do so is far beyond anything else I've found out there. Getting to it is a little awkward, you first need to sign up for an account (I got mine for free), then browse to the linux kernel project and select a defect to get into the browser. But once there, click the folder icon at the top left of the code window and select a source file. All of function names, variable names, structure names and structure field names are hyperlinks. Left click on one of them, click the little down arrow and select from the menu to list definitions, references, etc. You do need to find a reference to the token in question using some other browser like LXR, Coverity's doesn't seem to have a search button for that, but once located the cross-referencing provided is better than what I've seen in other text matching browsers like LXR, cscope, etc. It's particular good when you are trying to understand how a given field of a structure is used. Say you want to find out how a structure field named lock in some structure named foo is referenced. Find the definition of struct foo, click on the lock field and list references. It will show all the references to struct foo's lock but NOT show the thousands of references to all of the other fields named lock in other structures. That is something no other browser does, at least none that I am aware of. Jeff Haran Is it free software? Can I download it? How is it better than grep and sed? Does it work better than ctags? How can I get it to ingrate with vim? First off, if it is not available as free Software, then I won't use it. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
RE: Kernel thread scheduling
From: kernelnewbies-boun...@kernelnewbies.org [mailto:kernelnewbies-boun...@kernelnewbies.org] On Behalf Of Mark P Sent: Thursday, April 16, 2015 8:12 AM To: Ruben Safir Cc: nick; kernelnewbies@kernelnewbies.org Subject: Re: Kernel thread scheduling I find that the free electrons LXR has the best search capabilities: http://lxr.free-electrons.com/source/include/linux/rbtree.h#L35 -M Those interested in kernel source browsers might want to check out the code browser available at https://scan.coverity.com. Coverity does static code analysis and sells a product to do so, but they do regular scans of popular open source projects. Most of their focus is on finding and reporting defects, but the code browser they have created to do so is far beyond anything else I've found out there. Getting to it is a little awkward, you first need to sign up for an account (I got mine for free), then browse to the linux kernel project and select a defect to get into the browser. But once there, click the folder icon at the top left of the code window and select a source file. All of function names, variable names, structure names and structure field names are hyperlinks. Left click on one of them, click the little down arrow and select from the menu to list definitions, references, etc. You do need to find a reference to the token in question using some other browser like LXR, Coverity's doesn't seem to have a search button for that, but once located the cross-referencing provided is better than what I've seen in other text matching browsers like LXR, cscope, etc. It's particular good when you are trying to understand how a given field of a structure is used. Say you want to find out how a structure field named lock in some structure named foo is referenced. Find the definition of struct foo, click on the lock field and list references. It will show all the references to struct foo's lock but NOT show the thousands of references to all of the other fields named lock in other structures. That is something no other browser does, at least none that I am aware of. Jeff Haran ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
RE: Kernel thread scheduling
-Original Message- From: kernelnewbies-boun...@kernelnewbies.org [mailto:kernelnewbies- boun...@kernelnewbies.org] On Behalf Of Ruben Safir Sent: Thursday, April 16, 2015 10:09 AM To: kernelnewbies@kernelnewbies.org Subject: Re: Kernel thread scheduling On 04/16/2015 12:31 PM, Jeff Haran wrote: From: kernelnewbies-boun...@kernelnewbies.org [mailto:kernelnewbies-boun...@kernelnewbies.org] On Behalf Of Mark P Sent: Thursday, April 16, 2015 8:12 AM To: Ruben Safir Cc: nick; kernelnewbies@kernelnewbies.org Subject: Re: Kernel thread scheduling I find that the free electrons LXR has the best search capabilities: http://lxr.free-electrons.com/source/include/linux/rbtree.h#L35 -M Those interested in kernel source browsers might want to check out the code browser available at https://scan.coverity.com. Coverity does static code analysis and sells a product to do so, but they do regular scans of popular open source projects. Most of their focus is on finding and reporting defects, but the code browser they have created to do so is far beyond anything else I've found out there. Getting to it is a little awkward, you first need to sign up for an account (I got mine for free), then browse to the linux kernel project and select a defect to get into the browser. But once there, click the folder icon at the top left of the code window and select a source file. All of function names, variable names, structure names and structure field names are hyperlinks. Left click on one of them, click the little down arrow and select from the menu to list definitions, references, etc. You do need to find a reference to the token in question using some other browser like LXR, Coverity's doesn't seem to have a search button for that, but once located the cross-referencing provided is better than what I've seen in other text matching browsers like LXR, cscope, etc. It's particular good when you are trying to understand how a given field of a structure is used. Say you want to find out how a structure field named lock in some structure named foo is referenced. Find the definition of struct foo, click on the lock field and list references. It will show all the references to struct foo's lock but NOT show the thousands of references to all of the other fields named lock in other structures. That is something no other browser does, at least none that I am aware of. Jeff Haran Is it free software? Can I download it? The web site is free to use. The code that runs it is proprietary. How is it better than grep and sed? Does it work better than ctags? Grep, sed, ctags, cscope, etc. can only find references based on text matches. C allows duplicate field names across structure definitions. When using text based browsers to look for references to C structure field names on a large project like linux such searches yield lots of false positives, thousands of them in some cases. Coverity's browser on the other hand seems to understand C syntax. When looking for references so say struct foo's lock field, it won't show you references to struct bar's lock field. Other tools can't tell the difference. How can I get it to ingrate with vim? You can't so far as I know. First off, if it is not available as free Software, then I won't use it. That is of course your choice to make. Others may come to different conclusions. I use www.google.com on a regular basis even though I can't download it. Jeff Haran ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On Thu, Apr 16, 2015 at 02:42:07PM -0400, Ruben Safir wrote:
On 04/16/2015 02:32 PM, John de la Garza wrote:
On Thu, Apr 16, 2015 at 10:56:46AM -0400, Ruben Safir wrote:
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
How do you search these things out?
That is truly beautiful. Is that documented anywhere in plain english?
If you mean the tag search functionality, yes. See below.
I run:
make ctags
Documentation for this you can find by running make help at the root
of the Kernel source tree.
run vim
type:
:ts rb_node
This functionality is documented in Vim itself. To access the relevant
documentation just run
:help tags
There is also the cscope program that can give you information about
which functions call and are being called by which other functions. It
has its own man page.
Cheers,
Silvan
then I scanned the list for things that are from include/linux
and found this
20 F srb_node include/linux/rbtree.h
struct rb_node {
I select 20 and it takes me to include/linux/rbtree.h
and puts me at the line containing this:
struct rb_node {
unsigned long __rb_parent_color;
struct rb_node *rb_right;
struct rb_node *rb_left;
} __attribute__((aligned(sizeof(long;
/* The alignment might seem pointless, but allegedly CRIS needs it */
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
RE: Kernel thread scheduling
-Original Message- From: valdis.kletni...@vt.edu [mailto:valdis.kletni...@vt.edu] Sent: Thursday, April 16, 2015 11:48 AM To: Ruben Safir Cc: Jeff Haran; kernelnewbies@kernelnewbies.org Subject: Re: Kernel thread scheduling On Thu, 16 Apr 2015 14:28:59 -0400, Ruben Safir said: Others may come to different conclusions. I use www.google.com on a regular basis even though I can't download it. That is not even a good analogy. We're comparing closed Coverty-as-a-service with closed Google-as-a- service. Seems like a good analogy to me. I'm failing to see the moral difference between using Google SaaS to look to see if anybody else has reported a particular kernel error and using Coverty SaaS to examine kernel code. As you yourself said: If everyone depended on this Software as a Service then you would have a nice walled garden to the Kernel Code. Why do you object to Coverty when Google is almost certainly more depended on by kernel developers? Not only that, but demonizing Coverity over this is quite unfair. They provide for free to the world the results of their static code analysis of many open source projects, Linux included. Granted, Coverity's system isn't perfect, it does generate false positives but I know from personally quite embarrassing experience that it does find real bugs in code that I've written and looked at hundreds of times that I didn't see and its pointing out to anybody who would care to look thousands of potential kernel bugs. They also have this posterior-kicking code browser that's better than all of the free competition that I know of. If somebody knows of any better, please let me know. I often read emails on this list from people looking to help and get started in kernel development. Well, here's a good place to start. Submit patches to fix some of those Coverity identified kernel bugs. Some of them will be false positives, but some of them will be the real thing. And just for the record, I have no financial interest in Coverity or its parent company. I've been a user of their system at a couple of companies I've worked for now, but that and my usage of their free service is the only connection I have with them. Jeff Haran ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/16/2015 02:47 PM, valdis.kletni...@vt.edu wrote: We're comparing closed Coverty-as-a-service with closed Google-as-a-service. Seems like a good analogy to me. well it isn't. It is not demonizing them to state the facts. First of all, search engines do something you can't do with software, which is crawl the internet. The correct analogy is what google is doing with android, which indeed is repulsive because it is inhibiting access and development on the platform, and then worse than that, claiming that it is all open when really it is not. http://arstechnica.com/gadgets/2013/10/googles-iron-grip-on-android-controlling-open-source-by-any-means-necessary/ I have the source code for the kernel here. If Coventry would be interested in selling me a free software of their application, I might be interested. But I'm not interested in putting kernel tools behind slavewalls. That is exactly the opposite of what I want to do and defies my very Raison E'Stat for studying the Linux Kernel nono. If you find this Demonizing, it is not I who does this, it is facts stated forthrightly and dispassionately. Someone who uses closed proprietary software as a service to access the Linux Kernel Source is cutting off the very air to free software development by supporting non-free tools that compete with free tools in learning about the source code. Using free software tools is not a technological decision. It is a political decision (as all decisions ultimately are). It is a decision to exercise freedom over a __lack__ of freedom. Aside from that, the question of google is irrelevant to this conversation and is just a distraction. Tow wrongs don't make a right. A company that has a search engine that uses published free software would be much more desirable than one that doesn't do that. No one wants to penalize anyone for selling software services. But that is not reason to abandon free software tools for the studying of the kernel in order to adopt a proprietary scheme of secret software sold as a service. That would just be irrational. Ruben ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
First off, if it is not available as free Software, then I won't use it. That is of course your choice to make. And it is the correct decision to make. People who make another decision would not care about the damage they do to themselves and others by promoting the use of a proprietary system to access and learn about a free software project devised to intentionally do just the opposite and to give people access to code and ownership of their digital system. Why would someone sell that freedom for a minor convenience. This has been hashed. If everyone depended on this Software as a Service then you would have a nice walled garden to the Kernel Code. That would be a nice for you __maybe__, but it would be bad for the rest of us. https://www.gnu.org/philosophy/who-does-that-server-really-serve.html Others may come to different conclusions. I use www.google.com on a regular basis even though I can't download it. That is not even a good analogy. But for what it is worth, SOS is killing free software on the google platform altogether and it is a tragedy... it is killing Free Software and choking the future of development. Jeff Haran ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www2.mrbrklyn.com/resources - Unpublished Archive http://www.coinhangout.com - coins! http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013 ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On Thu, Apr 16, 2015 at 10:56:46AM -0400, Ruben Safir wrote:
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
How do you search these things out?
I run:
make ctags
run vim
type:
:ts rb_node
then I scanned the list for things that are from include/linux
and found this
20 F srb_node include/linux/rbtree.h
struct rb_node {
I select 20 and it takes me to include/linux/rbtree.h
and puts me at the line containing this:
struct rb_node {
unsigned long __rb_parent_color;
struct rb_node *rb_right;
struct rb_node *rb_left;
} __attribute__((aligned(sizeof(long;
/* The alignment might seem pointless, but allegedly CRIS needs it */
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/16/2015 02:32 PM, John de la Garza wrote:
On Thu, Apr 16, 2015 at 10:56:46AM -0400, Ruben Safir wrote:
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
THANK YOU - This is for my archive of useful Linux Hints!!
How do you search these things out?
I run:
make ctags
run vim
type:
:ts rb_node
then I scanned the list for things that are from include/linux
and found this
20 F srb_node include/linux/rbtree.h
struct rb_node {
I select 20 and it takes me to include/linux/rbtree.h
and puts me at the line containing this:
struct rb_node {
unsigned long __rb_parent_color;
struct rb_node *rb_right;
struct rb_node *rb_left;
} __attribute__((aligned(sizeof(long;
/* The alignment might seem pointless, but allegedly CRIS needs it */
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/16/2015 02:32 PM, John de la Garza wrote:
On Thu, Apr 16, 2015 at 10:56:46AM -0400, Ruben Safir wrote:
I'm trying to find rb_node's structure and I can't find it with ctags or
in the http://lxr.linux.no website.
How do you search these things out?
That is truly beautiful. Is that documented anywhere in plain english?
Ruben
I run:
make ctags
run vim
type:
:ts rb_node
then I scanned the list for things that are from include/linux
and found this
20 F srb_node include/linux/rbtree.h
struct rb_node {
I select 20 and it takes me to include/linux/rbtree.h
and puts me at the line containing this:
struct rb_node {
unsigned long __rb_parent_color;
struct rb_node *rb_right;
struct rb_node *rb_left;
} __attribute__((aligned(sizeof(long;
/* The alignment might seem pointless, but allegedly CRIS needs it */
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On Thu, 16 Apr 2015 14:28:59 -0400, Ruben Safir said: Others may come to different conclusions. I use www.google.com on a regular basis even though I can't download it. That is not even a good analogy. We're comparing closed Coverty-as-a-service with closed Google-as-a-service. Seems like a good analogy to me. I'm failing to see the moral difference between using Google SaaS to look to see if anybody else has reported a particular kernel error and using Coverty SaaS to examine kernel code. As you yourself said: If everyone depended on this Software as a Service then you would have a nice walled garden to the Kernel Code. Why do you object to Coverty when Google is almost certainly more depended on by kernel developers? pgpTD697fDT0c.pgp Description: PGP signature ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On April 12, 2015 1:06:57 AM EDT, Ruben Safir ru...@mrbrklyn.com wrote: On 04/12/2015 12:36 AM, Mohammed Ghriga wrote: In addition to the suggestions that were offered, I recommend you try reading Chapter 16: https://www.google.com/search?tbo=ptbm=bksq=isbn:0596554672 (pages: 267-272). http://it-ebooks.info/book/481/ FWIW I found this lovely wait example in the other kernel text under the scheduling section Sleeping and Waking Up page 58 in Robert Love, Linux Kernel Development http://www.it-ebooks.info/book/819/ I recommended this book as that was how I learn the basics of kernel development. Again if your interested in certain areas I will be glad to link/explain it to you to the best of my knowledge. Nick Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/12/2015 12:16 AM, nick wrote:
This finds the next node in the red black tree for sched_enities.
Basically rb_next finds the next node in the tree. The argument is
the rb_node structure embedded in the structure using a red black
tree.
The problem is this
struct mystruc {
int a;
char * string;
} wonderful;
char * string2 = wonderful.string;
There is no way to know that string2 comes from wonderful.
That is the problem I have with the nodes
The node is an instance of sched_entity. But unless there is a field
that says what task_entity it is embedded in, in of itself, there is no
way to know. There is no 'this' C.
Obviously if does know. I'll look at chapter 6
I'm also looking at this
static struct sched_entity *__pick_next_entity(struct sched_entity *se)
{
struct rb_node *next = rb_next(se-run_node);
if (!next)
return NULL;
return rb_entry(next, struct sched_entity, run_node); ==macro
}
The macro is: I think:
http://lxr.free-electrons.com/source/include/linux/rbtree.h#L50
50 #define rb_entry(ptr, type, member) container_of(ptr, type, member)
container_of is
http://lxr.free-electrons.com/source/include/linux/kernel.h#L798
791 /**
792 * container_of - cast a member of a structure out to the containing
structure
793 * @ptr:the pointer to the member.
794 * @type: the type of the container struct this is embedded in.
795 * @member: the name of the member within the struct.
796 *
797 */
798 #define container_of(ptr, type, member) ({ \
799 const typeof( ((type *)0)-member ) *__mptr = (ptr);\
800 (type *)( (char *)__mptr - offsetof(type,member) );})
Now I have embedded macros which I thought you couldn't even do...
In trying to understand this I stumbled on
http://linuxkernel51.blogspot.com/2011/02/how-containerof-macro-works-example.html
How container_of macro works, an Example
Here iam giving a small code snippet that gives and idea about working
of container_of, this posed me little difficulty in understanding,
after google-ing i got some examples and after working on that i wrote a
simple C application that depicts its working. here i have defined two
macros offsetof and container_of which i have extracted from
kernel.h header.
Please interpret this code and try some trick to understand
container_of.
container_of macro is defined in linux/kernel.h
syntax: container_of( pointer, container_type, container_field );
This macro takes a pointer to a filed name container_field, within a
structure of type container_type, and returns a pointer to the
containing structure .
simply this is a convenience macro that may be used to obtain a pointer
to a structure from a pointer to some other structure contained with in it.
Code :
#include stdio.h
#include stdlib.h
#define offsetof(TYPE, MEMBER) ((size_t) ((TYPE *)0)-MEMBER)
#define container_of(ptr, type, member) ({\
const typeof( ((type *)0)-member ) *__mptr = (ptr);\
(type *)( (char *)__mptr - offsetof(type,member) );})
struct test1 {
int a;
};
struct test2 {
int b;
struct test1 z;
int c;
};
int main()
{
/* existing structure */
struct test2 *obj;
obj = malloc(sizeof(struct test2));
if(obj == NULL){
printf(Error: Memory not allocated...!\n);
}
obj-z.a = 51;
obj-b = 43;
obj-c = 53;
/* pointer to existing entry */
struct test1 *obj1 = obj-z; //both of type test1
struct test2 *obj2 = container_of(obj1, struct test2, z);
printf(obj2-b = %d\n, obj2-b); ///notsure what this does,need a nap
return EXIT_SUCCESS;
}
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/11/2015 10:21 PM, Ruben Safir wrote:
On 04/10/2015 09:09 AM, nick wrote:
On 2015-04-09 11:37 PM, Ruben Safir wrote:
On 04/09/2015 10:52 PM, nick wrote:
Before asking questions again like this please look into either using lxr
or ctags
to navigate the kernel tree for answers as can be faster then waiting for
me or
someone else to respond.
well, I reading the text is ctags aren't much value there.
Ctags is useful for searching the code, which is why I am recommending it.
Nick
I have it built into gvim, but you can't use it from a textbook. I'm
finding it is not as useful as it could be for the kernel code. There
are stacks of tags to get around. Another 2 days to learn to get around
tags in vi is not in the agenda right now. It is the tool I have so
I'll have to live with it right now.
I also have a question that is not obvious from the code I'm looking at.
I'm not sure how these structs are attached together. Or more
specifically, I'm not sure how pulling the correct sched_entity gets one
the coresponding task_entity
You have
struct task_struct with a
struct sched_entity
struct sched_enitities are nodes in the RB tree
which are a container for struct rb_node run_node.
So a look at sched_entity ... is in ../linux/sched.h
1161 struct sched_entity {
1162struct load_weight load;/* for load-balancing */
1163struct rb_noderun_node;
1164struct list_head group_node;
1165unsigned int on_rq;
1166
1167u64 exec_start;
1168u64 sum_exec_runtime;
1169u64 vruntime;
1170u64 prev_sum_exec_runtime;
1171
1172u64 nr_migrations;
1173
1174 #ifdef CONFIG_SCHEDSTATS
1175struct sched_statistics statistics;
1176 #endif
1177
1178 #ifdef CONFIG_FAIR_GROUP_SCHED
1179int depth;
1180struct sched_entity *parent;
1181/* rq on which this entity is (to be) queued: */
1182struct cfs_rq *cfs_rq;
1183/* rq owned by this entity/group: */
1184struct cfs_rq *my_q;
1185 #endif
1186
1187 #ifdef CONFIG_SMP
1188/* Per-entity load-tracking */
1189struct sched_avg avg;
1190 #endif
1191 };
I see no means of referencing a specific task from this struct that
forms the node. So when you pull the node with the smallest vruntime
from the left most postion of the RB tree, by calling pick_next_task(),
static struct sched_entity *__pick_next_entity(struct sched_entity *se)
{
struct rb_node *next = rb_next(se-run_node);
if (!next)
return NULL;
return rb_entry(next, struct sched_entity, run_node);
}
how do we know what task we are attached to?
Ruben
I'm still loss on how we know which taks_struct is being used but as a
side note, I found this also very puzzling
return rb_entry(next, struct sched_entity, run_node);
With help I ran it down to this:
http://lxr.free-electrons.com/source/include/linux/rbtree.h#L50
#define rb_entry(ptr, type, member) container_of(ptr, type, member)
which leads me to yet another macro
798 #define container_of(ptr, type, member) ({ \
799 const typeof( ((type *)0)-member ) *__mptr = (ptr);\
800 (type *)( (char *)__mptr - offsetof(type,member) );})
This is a use of macros I'd never seen before up close. If anyone could
help me understand it, I'd appreciate it.
Ruben
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/12/2015 12:36 AM, Mohammed Ghriga wrote: In addition to the suggestions that were offered, I recommend you try reading Chapter 16: https://www.google.com/search?tbo=ptbm=bksq=isbn:0596554672 (pages: 267-272). http://it-ebooks.info/book/481/ FWIW I found this lovely wait example in the other kernel text under the scheduling section Sleeping and Waking Up page 58 in Robert Love, Linux Kernel Development http://www.it-ebooks.info/book/819/ ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 2015-04-11 11:02 PM, Ruben Safir wrote:
On 04/11/2015 10:21 PM, Ruben Safir wrote:
On 04/10/2015 09:09 AM, nick wrote:
On 2015-04-09 11:37 PM, Ruben Safir wrote:
On 04/09/2015 10:52 PM, nick wrote:
Before asking questions again like this please look into either using lxr
or ctags
to navigate the kernel tree for answers as can be faster then waiting for
me or
someone else to respond.
well, I reading the text is ctags aren't much value there.
Ctags is useful for searching the code, which is why I am recommending it.
Nick
I have it built into gvim, but you can't use it from a textbook. I'm
finding it is not as useful as it could be for the kernel code. There
are stacks of tags to get around. Another 2 days to learn to get around
tags in vi is not in the agenda right now. It is the tool I have so
I'll have to live with it right now.
I also have a question that is not obvious from the code I'm looking at.
I'm not sure how these structs are attached together. Or more
specifically, I'm not sure how pulling the correct sched_entity gets one
the coresponding task_entity
You have
struct task_struct with a
struct sched_entity
struct sched_enitities are nodes in the RB tree
which are a container for struct rb_node run_node.
So a look at sched_entity ... is in ../linux/sched.h
1161 struct sched_entity {
1162struct load_weight load;/* for load-balancing */
1163struct rb_noderun_node;
1164struct list_head group_node;
1165unsigned int on_rq;
1166
1167u64 exec_start;
1168u64 sum_exec_runtime;
1169u64 vruntime;
1170u64 prev_sum_exec_runtime;
1171
1172u64 nr_migrations;
1173
1174 #ifdef CONFIG_SCHEDSTATS
1175struct sched_statistics statistics;
1176 #endif
1177
1178 #ifdef CONFIG_FAIR_GROUP_SCHED
1179int depth;
1180struct sched_entity *parent;
1181/* rq on which this entity is (to be) queued: */
1182struct cfs_rq *cfs_rq;
1183/* rq owned by this entity/group: */
1184struct cfs_rq *my_q;
1185 #endif
1186
1187 #ifdef CONFIG_SMP
1188/* Per-entity load-tracking */
1189struct sched_avg avg;
1190 #endif
1191 };
I see no means of referencing a specific task from this struct that
forms the node. So when you pull the node with the smallest vruntime
from the left most postion of the RB tree, by calling pick_next_task(),
static struct sched_entity *__pick_next_entity(struct sched_entity *se)
{
struct rb_node *next = rb_next(se-run_node);
This finds the next node in the red black tree for sched_enities.
Basically rb_next finds the next node in the tree. The argument is
the rb_node structure embedded in the structure using a red black
tree.
if (!next)
return NULL;
If there is no runnable task return NULL and pick_next_task will run the
idle_task for this cpu.
return rb_entry(next, struct sched_entity, run_node);
}
how do we know what task we are attached to?
Also try to read Chapter 6 of Linux Kernel Development as if have read that
chapter understanding how red black trees and other data structures work in
kernel code would make more sense.
Nick
Ruben
I'm still loss on how we know which taks_struct is being used but as a
side note, I found this also very puzzling
return rb_entry(next, struct sched_entity, run_node);
With help I ran it down to this:
http://lxr.free-electrons.com/source/include/linux/rbtree.h#L50
#define rb_entry(ptr, type, member) container_of(ptr, type, member)
which leads me to yet another macro
798 #define container_of(ptr, type, member) ({ \
799 const typeof( ((type *)0)-member ) *__mptr = (ptr);\
800 (type *)( (char *)__mptr - offsetof(type,member) );})
This is a use of macros I'd never seen before up close. If anyone could
help me understand it, I'd appreciate it.
Ruben
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 04/10/2015 09:09 AM, nick wrote:
On 2015-04-09 11:37 PM, Ruben Safir wrote:
On 04/09/2015 10:52 PM, nick wrote:
Before asking questions again like this please look into either using lxr
or ctags
to navigate the kernel tree for answers as can be faster then waiting for
me or
someone else to respond.
well, I reading the text is ctags aren't much value there.
Ctags is useful for searching the code, which is why I am recommending it.
Nick
I have it built into gvim, but you can't use it from a textbook. I'm
finding it is not as useful as it could be for the kernel code. There
are stacks of tags to get around. Another 2 days to learn to get around
tags in vi is not in the agenda right now. It is the tool I have so
I'll have to live with it right now.
I also have a question that is not obvious from the code I'm looking at.
I'm not sure how these structs are attached together. Or more
specifically, I'm not sure how pulling the correct sched_entity gets one
the coresponding task_entity
You have
struct task_struct with a
struct sched_entity
struct sched_enitities are nodes in the RB tree
which are a container for struct rb_node run_node.
So a look at sched_entity ... is in ../linux/sched.h
1161 struct sched_entity {
1162struct load_weight load;/* for load-balancing */
1163struct rb_noderun_node;
1164struct list_head group_node;
1165unsigned int on_rq;
1166
1167u64 exec_start;
1168u64 sum_exec_runtime;
1169u64 vruntime;
1170u64 prev_sum_exec_runtime;
1171
1172u64 nr_migrations;
1173
1174 #ifdef CONFIG_SCHEDSTATS
1175struct sched_statistics statistics;
1176 #endif
1177
1178 #ifdef CONFIG_FAIR_GROUP_SCHED
1179int depth;
1180struct sched_entity *parent;
1181/* rq on which this entity is (to be) queued: */
1182struct cfs_rq *cfs_rq;
1183/* rq owned by this entity/group: */
1184struct cfs_rq *my_q;
1185 #endif
1186
1187 #ifdef CONFIG_SMP
1188/* Per-entity load-tracking */
1189struct sched_avg avg;
1190 #endif
1191 };
I see no means of referencing a specific task from this struct that
forms the node. So when you pull the node with the smallest vruntime
from the left most postion of the RB tree, by calling pick_next_task(),
static struct sched_entity *__pick_next_entity(struct sched_entity *se)
{
struct rb_node *next = rb_next(se-run_node);
if (!next)
return NULL;
return rb_entry(next, struct sched_entity, run_node);
}
how do we know what task we are attached to?
Ruben
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 03/22/2015 07:30 PM, nick wrote: I would recommend reading Chapters 3 and 4 of Linux Kernel Development by Robert Love as when I was learning the scheduler and process management how much has the scheduler changed since then. It was completely overhauled when the CFS was created ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 2015-03-22 08:05 PM, Ruben Safir wrote: On 03/22/2015 07:30 PM, nick wrote: I would recommend reading Chapters 3 and 4 of Linux Kernel Development by Robert Love as when I was learning the scheduler and process management how much has the scheduler changed since then. It was completely overhauled when the CFS was created The 3rd edition of this book was written after CFS was in the kernel so the chapters are pretty up to date. Nick ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
Thank you for the example. I understand what are the scheduling mechanics depending on task-state. But suppose another situation. Let's say I change current state to TASK_INTERRUPTIBLE. If I start now some long operation, then shouldn't the scheduler kick in as soon as he can and put my thread asleep? Or should I call necessarily schedule()? And if so, why? ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
On 2015-03-22 07:14 PM, Vincenzo Scotti wrote: Thank you for the example. I understand what are the scheduling mechanics depending on task-state. But suppose another situation. Let's say I change current state to TASK_INTERRUPTIBLE. If I start now some long operation, then shouldn't the scheduler kick in as soon as he can and put my thread asleep? Or should I call necessarily schedule()? And if so, why? Greetings Vincenzo. I would recommend reading Chapters 3 and 4 of Linux Kernel Development by Robert Love as when I was learning the scheduler and process management for the kernel,I found these chapters to be very useful.Further more for user space you are correct to my knowledge schedule is called for internal tasks that are taking too long in kernel land or to reschedule to another task,not for user space based tasks are TASK_INTERRUPTIBLE works fine there. Hope this Helps, Nick ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel thread scheduling
Hi Jeff,
kernel_thread just like daemon process. These are self monitoring process.
These process get scheduled by the kernel when their is any state change
from TASK_INTERRUPTIBLE to TASK_RUNNING or their is some condition that get
full field.
If you don't schedule() the kernel thread state is not get updated by the
scheduler.
Below example can help clear the concept.
http://lxr.free-electrons.com/source/drivers/vhost/vhost.c#L225
-Anand Moon
On Saturday, March 21, 2015 4:58 AM, Jeff Haran jeff.ha...@citrix.com wrote:
-Original Message-
From: kernelnewbies-boun...@kernelnewbies.org
[mailto:kernelnewbies-boun...@kernelnewbies.org] On Behalf Of Vincenzo Scotti
Sent: Friday, March 20, 2015 4:20 PM
To: kernelnewbies@kernelnewbies.org
Subject: Kernel thread scheduling
Hello,
I am actually studying kernel threads, and I have some doubts about them.
Let's take for example this snippet of code
static int thread_function(void *data)
{
while (!kthread_should_stop()) {
schedule();
}
pr_err(Stopped);
return 0;
}
This way it works just fine, and waits until I call kthread_stop on it.
But if I comment out that schedule() call, it just hangs my system when I load
it (it is part of a module). I see that the loop-schedule-wakeup pattern is
used among all the others kernel threads. But I don't get why I need to call
the scheduler explicitly.
I know that the kernel is fully preemptible, and in my interpretation I thought
that it could stop every running thread, even in kernel space, using a
timer-based interrupt handler, to give cpu to other threads. Doesn't this
pattern resemble a voluntary preemption model?
Where am I wrong?
Are you sure your kernel is configured with kernel preemption on? It is a
configurable option. Grep for PREEMPT in your .config file.
Jeff
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
RE: Kernel thread scheduling
-Original Message-
From: kernelnewbies-boun...@kernelnewbies.org
[mailto:kernelnewbies-boun...@kernelnewbies.org] On Behalf Of Vincenzo Scotti
Sent: Friday, March 20, 2015 4:20 PM
To: kernelnewbies@kernelnewbies.org
Subject: Kernel thread scheduling
Hello,
I am actually studying kernel threads, and I have some doubts about them.
Let's take for example this snippet of code
static int thread_function(void *data)
{
while (!kthread_should_stop()) {
schedule();
}
pr_err(Stopped);
return 0;
}
This way it works just fine, and waits until I call kthread_stop on it.
But if I comment out that schedule() call, it just hangs my system when I load
it (it is part of a module). I see that the loop-schedule-wakeup pattern is
used among all the others kernel threads. But I don't get why I need to call
the scheduler explicitly.
I know that the kernel is fully preemptible, and in my interpretation I thought
that it could stop every running thread, even in kernel space, using a
timer-based interrupt handler, to give cpu to other threads. Doesn't this
pattern resemble a voluntary preemption model?
Where am I wrong?
Are you sure your kernel is configured with kernel preemption on? It is a
configurable option. Grep for PREEMPT in your .config file.
Jeff
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
