Emulating linux timer.. syscalls

[SHAIKH AYEED]

Hello Sir,

Well, I was very much interested to start with this project could you
please guide me with some more details to start with.

Regards
AYEED SHAIKH

-- 



To know more on us, Please click on any of the following links;




 
 
 
 
 
 




Disclaimer

This email and any files 
transmitted with it are confidential and intended solely for the use of the 
individual or entity to whom they are addressed. It may also contain 
privileged or right protected information / work.  Use discretion in their 
appropriate use and sharing.  When in doubt, personally confirm with the 
sender. Although Acharya Institutes has taken reasonable precautions to 
ensure no viruses are present in this email, it cannot accept 
responsibility for any loss or damage arising from the use of this email or 
attachments. The recipient should adequate 'virus-check' before view / 
download / use.

Re: pg_jobc going negative?

[Mouse]

>> [...] I'd be astonished if there aren't at least a few programs
>> [in pkgsrc] that grub around in things like this.
> No question, there are, but this particular field seems very unlikely
> to have any users - really really unlikely.

Especially since, come to think of it, the value is incorrect often
enough for those asserts to fire, so anything depending on it is
already broken.

Good point.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: pg_jobc going negative?

[Christos Zoulas]

In article <27763.1594388...@jinx.noi.kre.to>,
Robert Elz   wrote:
>Date:Tue, 9 Jun 2020 08:23:19 - (UTC)
>From:mlel...@serpens.de (Michael van Elst)
>Message-ID:  
>
>I have spent a little time looking at this now, and I think
>it is just all a mess.
>
>  | pg_jobc is not a reference counter.
>
>Maybe not technically a "reference" counter, as what it is counting isn't
>strictly references, but anything that has x++ and if (--x == 0) do_stuff()
>is essentially a reference counter.   What it is counting references to
>isn't clear (particularly here), but that is what it is doing, or trying
>to do (it has all the same issues as things which really are ref counters).
>
>  | The assertion probably stopped
>  | a bug in a different place by coincidence.
>
>I doubt that, this code is not at all good.   There is no question but
>that the counter does not count properly.
>
>As best I can work out, and someone correct me if I'm wrong,
>the whole purpose of pg_jobc is so that orphanpg() can be called
>when a process group is orphaned (no longer has a session leader).
>
>If it has any other use, I cannot see it.
>
>What's there now simply doesn't work for this purpose.   It was
>suggested that the FreeBSD code has been modified from what we
>have, and that simply adopting that might work.   I went to look
>at their code, but before I did that, I saw that a month ago
>(that is, just around the time of the original discussion here)
>they copied maxv's KASSERTs into their code.  A week ago they removed
>them again, as they were occasionally firing.   That tells me,
>even without looking at their code, that they (still) have similar
>bugs to what we do, and thus that just importing their code won't
>help us.
>
>I see 3 ways forward...   simply drop the KASSERT the way that FreeBSD
>have done, and let things return to the semi-broken but rarely bothersome
>code that was there before.   That's not really a good idea, as the
>sanitizers apparently find problems with the code the way it was (not
>too surprising, deleting the KASSERT won't fix the bugs, it just stops
>noticing them explicitly).
>
>Or, we could properly define what pg_jobc is counting, and then make sure
>that it counts whatever that is properly - is incremented in all the
>appropriate places, and decremented properly as well.   Currently
>the comment about it in proc.h is:
>/*
> * Number of processes qualifying
>* pgrp for job control 
> */
>which makes it clear that it is a reference counter (not necessarily
>counting the number of something which exists, so that something can be 
>deleted, but it is counting references to processes).   Unfortunately
>I have no idea what "qualifying pgrp for job control" is supposed to mean.
>
>That could be done, but it seems like a lot of work to me, and not easy.
>
>Another (more radical) approach would be to simply drop orphanpg()
>completely, and thus no longer need pg_jobc at all.   The system
>wouldn't be bothered by this at all - all orphanpg() does is locate
>stopped members of the process group, and send then SIGCONT (to restart)
>and SIGHUP (to probably kill them - or at least inform them they their
>environment has changed).   If the system wasn't doing this, users manually
>(or a script run from cron or something) could do it instead,  If not done
>at all, badly behaving session leaders (processes which don't clean up
>their stopped jobs before exiting - including ones with bugs that causes
>them to abort) would over time cause a growing number of stopped jobs to
>simply clutter the system, consuming various resources, but otherwise
>harmless (there is nothing special about the processes, they can be killed,
>or continued - it is just that the process which would normally do that
>is no longer around).
>
>Third, and the option I'd suggest, is to revert to more basic principles,
>remove the pg_jobc attempt to detect when a session leader has exited,
>or changed to a different process group, and instead at candidate events
>(any time a process leaves a process group, for any reason) check if that
>process was the session leader, and if it is, clean up any now orphaned
>stopped processes.   This is likely to be slower than the current attempt,
>but is much easier to make correct (and much less likely to cause problems,
>other than dangling orphaned stopped processes, if incorrect).
>
>As best I can tell, all the data needed exists already, all that will be
>needed is to modify the code.   We can even leave pg_jobc in the pgrp
>struct, to avoid needing a kernel version bump (and for reasons I cannot
>imagine, pg_jobc is copied into kinfo and einfo structs for sysctl and /proc
>access to the process data, so leaving it around avoids needing to version
>those interfaces as well ... the value would be a meaningless 0, always, but
>I really find it hard to believe that anything would ever care, or even 
>notice).
>
>Opinions on any of this before I start 

Re: pg_jobc going negative?

[Robert Elz]

Date:Fri, 10 Jul 2020 16:47:28 +0200
From:Rhialto 
Message-ID:  <20200710144728.gy3...@falu.nl>

  | It also seems to be involved in deciding wether to send a SIGTTOU or
  | SIGTTIN to a process

Ah, right, thanks ... when I was reviewing uses in the kernel I
was concentrating on places where pg_jobc changes, and just sort of
dismissed the places where it was simply examined   then I never
went back to them again.   I will make sure my plans keep this working
(I knew that orphaned pg processes no longer do any kind of auto-stop,
though they can stil be sent SIGSTOP I believe).


  | I found this above fixjobc() which goes into a bit more detail what is
  | being counted:

Yes, I've been looking at that - that's where things are going wrong I
think, when looking at the children each child can decrement pg_jobc
but it only seems to get incremented once.   Easy to see how it becomes 
negative.

mo...@rodents-montreal.org said:
  | But each of those steps involves some winnowing-down.

Yes, again speculating, but my guess would be (for libkvm)
"if someone uses it, we make it available, if no-one does,
we don't" and quite probably ps was using it via /dev/mem
already (though what use that was supposed to have I cannot guess,
perhaps just for debugging the kernel implementation)

After that it appears that everything was just copied to each new
interface, based upon the philosophy of phasing out uses of libkvm,
which would mean making sure that the alternate interface could do
everything libkvm could do, otherwise someone would find the missing
data a justification for sticking with libkvm.

mo...@rodents-montreal.org said:
  | Did your userland sweep include pkgsrc? 

No, I don't have the resources to do that.  I don't use many packages
(so don't have many distfiles) and have even fewer of those unpacked.
My test system for HEAD has none of it at all.

mo...@rodents-montreal.org said:
  | [...] I'd be astonished if there aren't at least a few
  | programs there that grub around in things like this.

No question, there are, but this particular field seems very unlikely
to have any users - really really unlikely.

kre

Re: pg_jobc going negative?

[Rhialto]

On Fri 10 Jul 2020 at 20:41:40 +0700, Robert Elz wrote:
> As best I can work out, and someone correct me if I'm wrong,
> the whole purpose of pg_jobc is so that orphanpg() can be called
> when a process group is orphaned (no longer has a session leader).
> 
> If it has any other use, I cannot see it.

It also seems to be involved in deciding wether to send a SIGTTOU or
SIGTTIN to a process (i.e., to suspend a process if it tries to
read from the tty, but it is in the background).
In sys/kern/tty.c and sys/kern/tty_pty.c.

> Or, we could properly define what pg_jobc is counting, and then make sure
> that it counts whatever that is properly - is incremented in all the
> appropriate places, and decremented properly as well.   Currently
> the comment about it in proc.h is:
> /*
>  * Number of processes qualifying
>* pgrp for job control 
>  */
> which makes it clear that it is a reference counter (not necessarily
> counting the number of something which exists, so that something can be 
> deleted, but it is counting references to processes).   Unfortunately
> I have no idea what "qualifying pgrp for job control" is supposed to mean.

I found this above fixjobc() which goes into a bit more detail what is
being counted:

/*
 * Adjust pgrp jobc counters when specified process changes process group.
 * We count the number of processes in each process group that "qualify"
 * the group for terminal job control (those with a parent in a different
 * process group of the same session).  If that count reaches zero, the
 * process group becomes orphaned.  Check both the specified process'
 * process group and that of its children.

-Olaf.
-- 
Olaf 'Rhialto' Seibert -- rhialto at falu dot nl
___  Anyone who is capable of getting themselves made President should on
\X/  no account be allowed to do the job.   --Douglas Adams, "THGTTG"


signature.asc
Description: PGP signature

Re: pg_jobc going negative?

[Kamil Rytarowski]

On 10.07.2020 15:41, Robert Elz wrote:
> Unfortunately
> I have no idea what "qualifying pgrp for job control" is supposed to mean.

The Design and implementation of 4.4book phrases it as: number of
processes with parent controlling terminal.

Unfortunately the book does not explain whether pg_jobc can go bellow 0.



signature.asc
Description: OpenPGP digital signature

Re: pg_jobc going negative?

[Mouse]

> [... pg_jobc ...]

> I see 3 ways forward...

I count 4, but maybe kre is counting two of them as subclasses of a
single one.

> simply drop the KASSERT the way that FreeBSD have done, and let
> things return to the semi-broken but rarely bothersome code that was
> there before.

> Or, we could properly define what pg_jobc is counting, and then make
> sure that it counts whatever that is properly [...]

> Another (more radical) approach would be to simply drop orphanpg()
> completely, and thus no longer need pg_jobc at all.

> Third, and the option I'd suggest, is to revert to more basic
> principles, remove the pg_jobc attempt to detect when a session
> leader has exited, or changed to a different process group, and
> instead at candidate events (any time a process leaves a process
> group, for any reason) check if that process was the session leader,
> and if it is, clean up any now orphaned stopped processes.

Not surprisingly in view of who put it forward, I would agree with this
suggestion.

But that alone isn't worth an email.  The main thing prompting me to
write this mail is

> We can even leave pg_jobc in the pgrp struct, to avoid needing a
> kernel version bump (and for reasons I cannot imagine, pg_jobc is
> copied into kinfo and einfo structs for sysctl and /proc access to
> the process data, so leaving it around avoids needing to version
> those interfaces as well ... the value would be a meaningless 0,
> always, but I really find it hard to believe that anything would ever
> care, or even notice).

It seems to me that if pg_jobc is exported, someone presumably once
cared and there's thus a decent chance someone still cares.

Did you do a sweep for userland references to it?  It seems plausible
to me that it's used, at least for zero/nonzero, by userland tools that
are interested in process groups.

Is there any record of who added the code to export pg_jobc to
userland?  If so, and if that person is still around, it might be worth
sending a question thataway to see if any explanation might be
forthcoming.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: pg_jobc going negative?

[Robert Elz]

Date:Tue, 9 Jun 2020 08:23:19 - (UTC)
From:mlel...@serpens.de (Michael van Elst)
Message-ID:  

I have spent a little time looking at this now, and I think
it is just all a mess.

  | pg_jobc is not a reference counter.

Maybe not technically a "reference" counter, as what it is counting isn't
strictly references, but anything that has x++ and if (--x == 0) do_stuff()
is essentially a reference counter.   What it is counting references to
isn't clear (particularly here), but that is what it is doing, or trying
to do (it has all the same issues as things which really are ref counters).

  | The assertion probably stopped
  | a bug in a different place by coincidence.

I doubt that, this code is not at all good.   There is no question but
that the counter does not count properly.

As best I can work out, and someone correct me if I'm wrong,
the whole purpose of pg_jobc is so that orphanpg() can be called
when a process group is orphaned (no longer has a session leader).

If it has any other use, I cannot see it.

What's there now simply doesn't work for this purpose.   It was
suggested that the FreeBSD code has been modified from what we
have, and that simply adopting that might work.   I went to look
at their code, but before I did that, I saw that a month ago
(that is, just around the time of the original discussion here)
they copied maxv's KASSERTs into their code.  A week ago they removed
them again, as they were occasionally firing.   That tells me,
even without looking at their code, that they (still) have similar
bugs to what we do, and thus that just importing their code won't
help us.

I see 3 ways forward...   simply drop the KASSERT the way that FreeBSD
have done, and let things return to the semi-broken but rarely bothersome
code that was there before.   That's not really a good idea, as the
sanitizers apparently find problems with the code the way it was (not
too surprising, deleting the KASSERT won't fix the bugs, it just stops
noticing them explicitly).

Or, we could properly define what pg_jobc is counting, and then make sure
that it counts whatever that is properly - is incremented in all the
appropriate places, and decremented properly as well.   Currently
the comment about it in proc.h is:
/*
 * Number of processes qualifying
 * pgrp for job control 
 */
which makes it clear that it is a reference counter (not necessarily
counting the number of something which exists, so that something can be 
deleted, but it is counting references to processes).   Unfortunately
I have no idea what "qualifying pgrp for job control" is supposed to mean.

That could be done, but it seems like a lot of work to me, and not easy.

Another (more radical) approach would be to simply drop orphanpg()
completely, and thus no longer need pg_jobc at all.   The system
wouldn't be bothered by this at all - all orphanpg() does is locate
stopped members of the process group, and send then SIGCONT (to restart)
and SIGHUP (to probably kill them - or at least inform them they their
environment has changed).   If the system wasn't doing this, users manually
(or a script run from cron or something) could do it instead,  If not done
at all, badly behaving session leaders (processes which don't clean up
their stopped jobs before exiting - including ones with bugs that causes
them to abort) would over time cause a growing number of stopped jobs to
simply clutter the system, consuming various resources, but otherwise
harmless (there is nothing special about the processes, they can be killed,
or continued - it is just that the process which would normally do that
is no longer around).

Third, and the option I'd suggest, is to revert to more basic principles,
remove the pg_jobc attempt to detect when a session leader has exited,
or changed to a different process group, and instead at candidate events
(any time a process leaves a process group, for any reason) check if that
process was the session leader, and if it is, clean up any now orphaned
stopped processes.   This is likely to be slower than the current attempt,
but is much easier to make correct (and much less likely to cause problems,
other than dangling orphaned stopped processes, if incorrect).

As best I can tell, all the data needed exists already, all that will be
needed is to modify the code.   We can even leave pg_jobc in the pgrp
struct, to avoid needing a kernel version bump (and for reasons I cannot
imagine, pg_jobc is copied into kinfo and einfo structs for sysctl and /proc
access to the process data, so leaving it around avoids needing to version
those interfaces as well ... the value would be a meaningless 0, always, but
I really find it hard to believe that anything would ever care, or even 
notice).

Opinions on any of this before I start banging on the code?

kre

ps: I don't believe that any of the problems here are race conditions,
the counter is simply not maintained correctly (which isn't to