Re: a problem with linux 2.6.11 and sa

[George Georgalis]

On Wed, Mar 16, 2005 at 05:37:59PM -0500, Paul Jarc wrote:
>"George Georgalis" <[EMAIL PROTECTED]> wrote:
>> On Wed, Mar 09, 2005 at 06:28:35PM -0500, Paul Jarc wrote:
>>> To simplify, what about these two:
>>> mplayer foo.mpg
>>> mplayer foo.mpg < mediafiles.txt
>>
>> The particular host does not have X support so mpg is out.
>
>Well, use any one of the files listed in mediafiles.txt.  I expect the
>first one would behave the same as your for loop, and the second would
>behave the same as your while loop.

zz.mtest contains the full path to 3 ogg files on 3 lines, no funny
characters, the following is one of them.

$ mplayer 
/usr/nfs/sandbox/media/audio/_the-party-has-just-begun/Lebanese_Blonde.ogg

plays fine, as expected. however, per your test:

$ mplayer 
/usr/nfs/sandbox/media/audio/_the-party-has-just-begun/Lebanese_Blonde.ogg 
176400 (112.0 kbit)
Selected audio codec: [vorbis] afm:libvorbis (OggVorbis Audio Decoder)
==
Checking audio filter chain for 44100Hz/2ch/16bit -> 44100Hz/2ch/16bit...
AF_pre: af format: 2 bps, 2 ch, 44100 hz, little endian signed int 
AF_pre: 44100Hz 2ch Signed 16-bit (Little-Endian)
AO: [oss] 44100Hz 2ch Signed 16-bit (Little-Endian) (2 bps)
Building audio filter chain for 44100Hz/2ch/16bit -> 44100Hz/2ch/16bit...
Video: no video
Starting playback...
No bind found for key _ 
A:   0.1 (00.1) ??,?%   
No bind found for key L 
No bind found for key _ 
No bind found for key B 
No bind found for key l 
A:   0.8 (00.8)  4.3%   
  =  PAUSE  =

Exiting... (End of file)


program crashes quickly, without any keyboard interaction.

>> I'm not sure that that test would work as mplayer requires filenames
>> as command arguments not stdin (exclusivly, I think);
>
>Note that I said to redirect input from mediafiles.txt, not from any
>of the filenames listed in it, but one of the files listed in it
>should also be passed ion the command line in both cases.
>
>Your test also had mplayer's stdin connected to mediafiles.txt.  It
>was just less explicit - mplayer inherits stdin from surrounding loop.
>So I'm suggesting simplifying the test so that stdin is the *only*
>difference between the two cases, and that will show whether it's
>relevant.  OTOH, if you can't reproduce the problem with the
>simplified pair of tests, then some interaction with the shell loops
>must be involved.

per above, the problem is reproduced with your example.

v>> this works fine
>> mplayer `cat zz.mtest `
>
>> Then I tried
>> while read file; do mplayer "$file" ; done 
>What's in zz.mtest?  E.g., if it contains a line "-", then that will
>tell mplayer to play the file on stdin, which in this case is
>zz.mtest.  Choosing one of the listed files and testing with that, as
>I suggested above, will eliminate this possibility.

zz.mtest is just 3 ogg files like the one above in my first run. Me
throws up hands, I know it is kernel api change, me thinks Linux is not
posix anymore (per lkml followup). Big concern is not my ability to play
songs, but *complex* scripts to check spam during smtp are broke in
2.6.11 (rc?) and forward.

tmp="${scq}/`safecat "${scq}/tmp" "${scq}" http://galis.org/george/ cell:646-331-2027 mailto:[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[Paul Jarc]

"George Georgalis" <[EMAIL PROTECTED]> wrote:
> On Wed, Mar 09, 2005 at 06:28:35PM -0500, Paul Jarc wrote:
>> To simplify, what about these two:
>> mplayer foo.mpg
>> mplayer foo.mpg < mediafiles.txt
>
> The particular host does not have X support so mpg is out.

Well, use any one of the files listed in mediafiles.txt.  I expect the
first one would behave the same as your for loop, and the second would
behave the same as your while loop.

> I'm not sure that that test would work as mplayer requires filenames
> as command arguments not stdin (exclusivly, I think);

Note that I said to redirect input from mediafiles.txt, not from any
of the filenames listed in it, but one of the files listed in it
should also be passed ion the command line in both cases.

Your test also had mplayer's stdin connected to mediafiles.txt.  It
was just less explicit - mplayer inherits stdin from surrounding loop.
So I'm suggesting simplifying the test so that stdin is the *only*
difference between the two cases, and that will show whether it's
relevant.  OTOH, if you can't reproduce the problem with the
simplified pair of tests, then some interaction with the shell loops
must be involved.

> this works fine
> mplayer `cat zz.mtest `
>
> Then I tried
> mplayer /dev/stdin  Then I tried
> while read file; do mplayer "$file" ; done http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[George Georgalis]

On Wed, Mar 09, 2005 at 06:28:35PM -0500, Paul Jarc wrote:
>"George Georgalis" <[EMAIL PROTECTED]> wrote:
>> It (Gerrit Pape's technique) very defiantly stopped working a few revs
>> back (2.6.7?). I'm seeing a similar failed read from /dev/rtc and
>> mplayer with 2.6.10, now too.
>
>The /proc/kmsg problem happens because the kernel now checks for
>permission at read() instead of open().  The /dev/rtc problem seems to
>be a different beast.

Thanks for the kmsg clairfication, Paul.

>> while read file; do mplayer $file ; done >
>> Failed to open /dev/rtc: Permission denied
>>
>> for file in `cat mediafiles.txt`; do mplayer $file ; done
>>
>> works.
>
>To simplify, what about these two:
>mplayer foo.mpg
>mplayer foo.mpg < mediafiles.txt
>
>You might try strace'ing both cases and see how they compare.

The particular host does not have X support so mpg is out.
I'm not sure that that test would work as mplayer requires filenames
as command arguments not stdin (exclusivly, I think); my guess
is mplayer would try to decode stdin.

this works fine
mplayer `cat zz.mtest `

Then I tried
mplayer /dev/stdin  /proc/sys/dev/rtc/max-user-freq" to your system startup 
scripts.

the file almost played though...
Playing 
/usr/nfs/sandbox/media/audio/_the-party-has-just-begun/Lebanese_Blonde.ogg.
Ogg file format detected.
...

But it seemed to take keyboard commands from the binary
No bind found for key _ 
A:   0.1 (00.1) ??,?%   
No bind found for key R 
A:   0.8 (00.8)  4.2%   

and quit.  I tried the sysctl suggestion, no change, whenever a file list
is redirected to stdin, and a filename argument is given to mplayer, eg

while read file; do mplayer "$file" ; done http://galis.org/george/ cell:646-331-2027 mailto:[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[Nix]

On Wed, 09 Mar 2005, Paul Jarc uttered the following:
> "George Georgalis" <[EMAIL PROTECTED]> wrote:
>> It (Gerrit Pape's technique) very defiantly stopped working a few revs
>> back (2.6.7?). I'm seeing a similar failed read from /dev/rtc and
>> mplayer with 2.6.10, now too.
> 
> The /proc/kmsg problem happens because the kernel now checks for
> permission at read() instead of open().

Am I the only person who thinks `eee-ick, how utterly non-POSIX' at
that?

-- 
> ...Hires Root Beer...
What we need these days is a stable, fast, anti-aliased root beer
with dynamic shading. Not that you can let just anybody have root.
 --- John M. Ford
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[Paul Jarc]

"George Georgalis" <[EMAIL PROTECTED]> wrote:
> It (Gerrit Pape's technique) very defiantly stopped working a few revs
> back (2.6.7?). I'm seeing a similar failed read from /dev/rtc and
> mplayer with 2.6.10, now too.

The /proc/kmsg problem happens because the kernel now checks for
permission at read() instead of open().  The /dev/rtc problem seems to
be a different beast.

> while read file; do mplayer $file ; done 
> Failed to open /dev/rtc: Permission denied
>
> for file in `cat mediafiles.txt`; do mplayer $file ; done
>
> works.

To simplify, what about these two:
mplayer foo.mpg
mplayer foo.mpg < mediafiles.txt

You might try strace'ing both cases and see how they compare.


paul
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[George Georgalis]

On Wed, Mar 09, 2005 at 01:06:11PM +, Nix wrote:

>> An interesting technique that allows a program (such as a log writer)
>> to run as an unprivileged user, while receiving privileged data. (taken
>> almost verbatim from Gerrit Pape's socklog)
>> 
>> #!/bin/sh
>> exec > exec 2>&1
>> exec softlimit -m 200 setuidgid nobody socklog ucspi
>> 
>> This script, run by root takes its stdin from /proc/kmsg then combines
>> its stdout and stderr, and exec-switches to the socklog program run
>> as an ucspi application listening to the domain stream socket, as
>> nobody:nogroup, with memory consumption limited to 2Mb. (and sends
>> log to stdout)
>
>This is definitely redirection, not piping. As far as I know the
>implementation of redirection in the kernel remains unchanged: certainly
>the need to buffer piped data doesn't exist in this case, and since the
>redesign was of the buffering, this is probably not your problem :)
>
>> It worked flawlessly until several kernel revs back when the kernel
>> started protecting kmsg and wouldn't allow the user program to receive
>> it,
>
>Indeed.
>
>>   result: nothing sent to the logging program and no error. The fix
>> was to run socklog as root instead of nobody.
>
>You should be able to open it as root and read from it as another user:
>i.e., your technique above shouldn't break. (I'd hope.)

Here is a nice proof that kmsg did become a problem around 2.6.0
http://article.gmane.org/gmane.comp.misc.pape.general/595
http://thread.gmane.org/gmane.comp.misc.pape.general/590


It (Gerrit Pape's technique) very defiantly stopped working a few revs
back (2.6.7?). I'm seeing a similar failed read from /dev/rtc and
mplayer with 2.6.10, now too.

http://lkml.org/lkml/2005/3/8/226

while read file; do mplayer $file ; done http://galis.org/george/ cell:646-331-2027 mailto:[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[Nix]

On Tue, 8 Mar 2005, George Georgalis announced authoritatively:
> Here's what I'm doing that is broken. I use tcpserver (functionally
> similar to inetd) to receive an incoming smtp connection. While the
> smtp session is still open, the message is piped to a temp file which
> is then scanned for spam, if it passes the temp file is piped to my

Both of these sound like redirection, not piping.

>>(I don't see what you mean by `a pipe rom /proc/kmsg', though:
>>pipes connect processes, not files. File redirections are
>>quite different and should work unchanged in 2.6.11.)
> 
> An interesting technique that allows a program (such as a log writer)
> to run as an unprivileged user, while receiving privileged data. (taken
> almost verbatim from Gerrit Pape's socklog)
> 
> #!/bin/sh
> exec  exec 2>&1
> exec softlimit -m 200 setuidgid nobody socklog ucspi
> 
> This script, run by root takes its stdin from /proc/kmsg then combines
> its stdout and stderr, and exec-switches to the socklog program run
> as an ucspi application listening to the domain stream socket, as
> nobody:nogroup, with memory consumption limited to 2Mb. (and sends
> log to stdout)

This is definitely redirection, not piping. As far as I know the
implementation of redirection in the kernel remains unchanged: certainly
the need to buffer piped data doesn't exist in this case, and since the
redesign was of the buffering, this is probably not your problem :)

> It worked flawlessly until several kernel revs back when the kernel
> started protecting kmsg and wouldn't allow the user program to receive
> it,

Indeed.

>   result: nothing sent to the logging program and no error. The fix
> was to run socklog as root instead of nobody.

You should be able to open it as root and read from it as another user:
i.e., your technique above shouldn't break. (I'd hope.)

-- 
> ...Hires Root Beer...
What we need these days is a stable, fast, anti-aliased root beer
with dynamic shading. Not that you can let just anybody have root.
 --- John M. Ford
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[Andre Tomt]

George Georgalis wrote:
Here is a problem with 2.6.10:
while read file; do mplayer $file ; done 
or
tail -n93  mediafiles.txt | while read file; do mplayer $file ; done
for each file path in that text file I get:
Failed to open /dev/rtc: Permission denied (it should be readable by the user.)
^- This is also emitted by mplayer. It usually does this on any standard 
system.

In addition the audio pcm level is set to zero (presumably by mplayer).
This does work:
for file in `cat mediafiles.txt`; do mplayer $file ; done
but discovering and fixing code now broke will be unpleasent.
What exactly is going on? 

// George
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[George Georgalis]

On Tue, 8 Mar 2005 12:19:53 -0500, George Georgalis <[EMAIL PROTECTED]> wrote:
> On Tue, Mar 08, 2005 at 11:58:14AM -0500, George Georgalis wrote:
> >On Tue, Mar 08, 2005 at 01:37:03PM +, Nix wrote:
> >>On Thu, 3 Mar 2005, George Georgalis uttered the following:
> >>> I recall a problem a while back with a pipe from
> >>> /proc/kmsg that was sent by root to a program with a
> >>> user uid. The fix was to run the logging program as
> >>> root. Has that protected pipe method been extended
> >>> since 2.6.8.1?
> >>
> >>The entire implementation of pipes has been radically revised between
> >>2.6.10 and 2.6.11: see, e.g.,  and
> >>.
> >>
> >>Bugs have been spotted in this area in 2.6.10: this may be
> >>another one.
> >
> >Thanks, my issue is clearly between 2.6.10 and 2.6.11; though I won't be
> >able to drill down anything more specific, for a while. The links
> >do look relevant but I cannot say for sure.

Here is a problem with 2.6.10:

while read file; do mplayer $file ; done http://galis.org/george/ cell:646-331-2027 mailto:[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[George Georgalis]

On Tue, Mar 08, 2005 at 11:58:14AM -0500, George Georgalis wrote:
>On Tue, Mar 08, 2005 at 01:37:03PM +, Nix wrote:
>>On Thu, 3 Mar 2005, George Georgalis uttered the following:
>>> I recall a problem a while back with a pipe from
>>> /proc/kmsg that was sent by root to a program with a
>>> user uid. The fix was to run the logging program as
>>> root. Has that protected pipe method been extended
>>> since 2.6.8.1?
>>
>>The entire implementation of pipes has been radically revised between
>>2.6.10 and 2.6.11: see, e.g.,  and
>>.
>>
>>Bugs have been spotted in this area in 2.6.10: this may be
>>another one.
>
>Thanks, my issue is clearly between 2.6.10 and 2.6.11; though I won't be
>able to drill down anything more specific, for a while. The links
>do look relevant but I cannot say for sure.

My last post didn't actually describe what the problem is, which is
spamassassin always scores 0/0 under 2.6.11 but functions properly
(scoring x/5) under 2.6.10.

More details are in the thread of this post.
http://lkml.org/lkml/2005/3/3/513

// George

-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: a problem with linux 2.6.11 and sa

[George Georgalis]

On Tue, Mar 08, 2005 at 01:37:03PM +, Nix wrote:
>On Thu, 3 Mar 2005, George Georgalis uttered the following:
>> I recall a problem a while back with a pipe from
>> /proc/kmsg that was sent by root to a program with a
>> user uid. The fix was to run the logging program as
>> root. Has that protected pipe method been extended
>> since 2.6.8.1?
>
>The entire implementation of pipes has been radically revised between
>2.6.10 and 2.6.11: see, e.g.,  and
>.
>
>Bugs have been spotted in this area in 2.6.10: this may be
>another one.

Thanks, my issue is clearly between 2.6.10 and 2.6.11; though I won't be
able to drill down anything more specific, for a while. The links
do look relevant but I cannot say for sure.

Here's what I'm doing that is broken. I use tcpserver (functionally
similar to inetd) to receive an incoming smtp connection. While the
smtp session is still open, the message is piped to a temp file which
is then scanned for spam, if it passes the temp file is piped to my
local delivery program. If it doesn't pass the spam test or the delivery
program fails (disk full etc), the respective error code, if any,
is passed to tcpserver. The corresponding accepted, temporary reject or
permanently reject signal is passed to the remote sender.

The temp file is then removed or, for spam, it is cataloged for
statistics and/or abuse reporting. An additional copy is kept in a
traditional maildir to check for false positives.

#!/bin/bash
# exit 31 = permanently refuse
# exit 71 = temporarily refuse
# pwd is /var/qmail
echo $0 # for the logs
scq="spamc-queue" # a maildir with qmaild write perms
tmp="${scq}/`safecat "${scq}/tmp" "${scq}" >$sipd/date # keep track of when they 
came
maildir "${sipd}" >/dev/null <"$tmp" # keep a copy for reporting
maildir "${scq}"  >/dev/null <"$tmp" # save it to verify no falseys
rm "$tmp"
exit 31
;;
*) # spamc error, 
echo "$0 error, spamc exit $sce"
exit 71
esac
exit 81 # Internal bug



>If you can reproduce it consistently, *please* report
>this to the linux-kernel list!

I did, but have had no response to my followup:

Date: Fri, 4 Mar 2005 15:58:43 -0500
Subject: Re: problem with linux 2.6.11 and sa


>(I don't see what you mean by `a pipe rom /proc/kmsg', though:
>pipes connect processes, not files. File redirections are
>quite different and should work unchanged in 2.6.11.)


An interesting technique that allows a program (such as a log writer)
to run as an unprivileged user, while receiving privileged data. (taken
almost verbatim from Gerrit Pape's socklog)

#!/bin/sh
exec &1
exec softlimit -m 200 setuidgid nobody socklog ucspi

This script, run by root takes its stdin from /proc/kmsg then combines
its stdout and stderr, and exec-switches to the socklog program run
as an ucspi application listening to the domain stream socket, as
nobody:nogroup, with memory consumption limited to 2Mb. (and sends
log to stdout)

It worked flawlessly until several kernel revs back when the kernel
started protecting kmsg and wouldn't allow the user program to receive
it, result: nothing sent to the logging program and no error. The fix
was to run socklog as root instead of nobody.

// George



-- 
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:[EMAIL PROTECTED]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/