Re: Thread issue - Possible fix
"Marcelo Ferreira" <[EMAIL PROTECTED]> wrote: > I tested the 20020203 snapshot with the _r() changes. > I ran then server without the -s option (radiusd -p 1812 ) and it cored > dump: These problems are a real pain to track down. > If I run it with -s (radiusd -s -p 1812) works fine. Yeah, it's probably a thread issue. The rest of the thread-unsafe functions should be replaced with the thread-safe version, but I haven't had time to do that yet. As always, patches are welcome. > btw.. did you have (extra) time to revise the rlm_radutmp patch that I sent No, not yet. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
Alan, I tested the 20020203 snapshot with the _r() changes. I ran then server without the -s option (radiusd -p 1812 ) and it cored dump: #0 pairmove (to=0x38, from=0x42964990) at valuepair.c:186 #1 0x08051614 in radius_exec_program (cmd=0x80fde28 "/usr/local/freeradius/auth/bin/autenticadbm %u %f %w", request=0x888, exec_wait=1, user_msg=0x42966e78) at exec.c:305 #2 0x0805259c in rad_authenticate (request=0x888) at auth.c:832 #3 0x0804dcd0 in rad_respond (request=0x888, fun=0x8051d50 ) at radiusd.c:1472 #4 0x08056e04 in request_handler_thread (arg=0x80e4b08) at threads.c:169 #5 0x40061b9c in pthread_start_thread (arg=0x42967be0) at manager.c:274 #6 0x40061c7f in pthread_start_thread_event (arg=0x42967be0) at manager.c:298 If I run it with -s (radiusd -s -p 1812) works fine. []s btw.. did you have (extra) time to revise the rlm_radutmp patch that I sent ? Marcelo Ferreira Canbras TVA Cabo Ltda Canbras Acesso - STA Phone: +5511-4993-8728 - Original Message - From: <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Saturday, February 02, 2002 2:02 PM Subject: Re: Thread issue - Possible fix > "Miquel van Smoorenburg" <[EMAIL PROTECTED]> wrote: > > Well, you should be able to link against thread-safe variants of > > those functions, right? I mean, strtok() is obviously not > > thread safe, but localtime() should be. > > Nope. Log into a *BSD box. The thread-safe variant of localtime() > is localtime_r(). Localtime returns a pointer to static data, which > can be over-written in subsequent calls. > > I've *never* understood why people designed functions which returned > pointers to static data. They don't even scale on *one* process, much > less multi-threaded. > > e.g. Try to keep two different pointers to data returned by two > subsequent calls to localtime(), for two different dates. You can't. > You get the same pointer both times. So the second call over-writes > the data from the first call. > > So the only sane way to use that function even in a process with one > thread of execution, is to copy the results to another buffer. But if > that's true, why the heck didn't localtime just take a pointer to the > place to store the returned data? > > But I digress... > > > It's probably just a matter of linking against the right library. > > libc_r instead of libc, most likely. > > libc_r usually has the foo_r() functions. libc() doesn't. > > libc_r usually *also* has the unsafe foo() functions, so people who > want to shoot themselves in the foot can do so. > > > I think Linux turns on thread-safety if you define -DREENTRANT > > and link with -pthreads. Other systems might need other > > magic incantations. Like FreeBSD, and probably Solaris. > > glibc does *insane* things to be thread-safe while returning > pointers to static data. That's why Linux doesn't have the foo_r() > functions. > > It's also why glibc is so much larger and more complicated than the > *BSD C libraries. They're trying to be smart & cute, instead of > writing simple, clean, code. > > > Don't just substitute all library functions for their _r equivalents, > > just find out how to link against the thread-safe libc variant. > > The server has been linking that way for a while, and it isn't good > enough. People other than the glibc hackers run away screaming when > asked to make thread-safe versions of localtime(), etc. > > The simplest thing to do is to *always* use the localtime_r() > functions, and to have a trivial wrapper around localtime() using > 'memcpy' to get the same effect. I've made that change in the server > for ctime_r() and for localtime_r(). If it *creates* problems, I'd > love to know. > > IMHO, *all* of the functions returning pointers to static data > should just go away. They're badly designed. > > Alan DeKok. > > - > List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html > - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
In article <[EMAIL PROTECTED]>, <[EMAIL PROTECTED]> wrote: >"Miquel van Smoorenburg" <[EMAIL PROTECTED]> wrote: >> Well, you should be able to link against thread-safe variants of >> those functions, right? I mean, strtok() is obviously not >> thread safe, but localtime() should be. > > Nope. Log into a *BSD box. The thread-safe variant of localtime() >is localtime_r(). Localtime returns a pointer to static data, which >can be over-written in subsequent calls. Ah, you're right. I think I spent too much time in perl. Mike. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
"Miquel van Smoorenburg" <[EMAIL PROTECTED]> wrote: > Well, you should be able to link against thread-safe variants of > those functions, right? I mean, strtok() is obviously not > thread safe, but localtime() should be. Nope. Log into a *BSD box. The thread-safe variant of localtime() is localtime_r(). Localtime returns a pointer to static data, which can be over-written in subsequent calls. I've *never* understood why people designed functions which returned pointers to static data. They don't even scale on *one* process, much less multi-threaded. e.g. Try to keep two different pointers to data returned by two subsequent calls to localtime(), for two different dates. You can't. You get the same pointer both times. So the second call over-writes the data from the first call. So the only sane way to use that function even in a process with one thread of execution, is to copy the results to another buffer. But if that's true, why the heck didn't localtime just take a pointer to the place to store the returned data? But I digress... > It's probably just a matter of linking against the right library. > libc_r instead of libc, most likely. libc_r usually has the foo_r() functions. libc() doesn't. libc_r usually *also* has the unsafe foo() functions, so people who want to shoot themselves in the foot can do so. > I think Linux turns on thread-safety if you define -DREENTRANT > and link with -pthreads. Other systems might need other > magic incantations. Like FreeBSD, and probably Solaris. glibc does *insane* things to be thread-safe while returning pointers to static data. That's why Linux doesn't have the foo_r() functions. It's also why glibc is so much larger and more complicated than the *BSD C libraries. They're trying to be smart & cute, instead of writing simple, clean, code. > Don't just substitute all library functions for their _r equivalents, > just find out how to link against the thread-safe libc variant. The server has been linking that way for a while, and it isn't good enough. People other than the glibc hackers run away screaming when asked to make thread-safe versions of localtime(), etc. The simplest thing to do is to *always* use the localtime_r() functions, and to have a trivial wrapper around localtime() using 'memcpy' to get the same effect. I've made that change in the server for ctime_r() and for localtime_r(). If it *creates* problems, I'd love to know. IMHO, *all* of the functions returning pointers to static data should just go away. They're badly designed. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
In article <[EMAIL PROTECTED]>, <[EMAIL PROTECTED]> wrote: >Eddie Stassen <[EMAIL PROTECTED]> wrote: >> It appears that the use of non thread-safe library functions are >> responsible for this. > > Uh, yeah. Why didn't I think of that. > Sorry... Well, you should be able to link against thread-safe variants of those functions, right? I mean, strtok() is obviously not thread safe, but localtime() should be. It's probably just a matter of linking against the right library. libc_r instead of libc, most likely. I think Linux turns on thread-safety if you define -DREENTRANT and link with -pthreads. Other systems might need other magic incantations. Like FreeBSD, and probably Solaris. > The changes MUSt be made for safety. I'll take a look at doing some >of the work over the next few days. Don't just substitute all library functions for their _r equivalents, just find out how to link against the thread-safe libc variant. Mike. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
On Fri, Feb 01, 2002 at 10:52:49AM -0500, [EMAIL PROTECTED] wrote: > Eddie Stassen <[EMAIL PROTECTED]> wrote: > > It appears that the use of non thread-safe library functions are > > responsible for this. [...] > The changes MUSt be made for safety. I'll take a look at doing some > of the work over the next few days. yay! Maybe this will solve the solaris proxy problems that seem to still be present? Sounds like it's time for 0.5. :-) /fc - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
Eddie Stassen <[EMAIL PROTECTED]> wrote: > I don't have the time right now to make up the diffs (its Friday afternoon > and 30 deg C outside :)), That's OK, it's -4C and raining here. > I changed all the localtime & ctime references in the following > files: > ... I've made the changes myself, and commited the code to CVS. So the latest snapshot should have the fixes. You'll have to download it, configure, make, and make install. > I am not too familiar with linux, but I suspect you may need some extra > #defines to enable the POSIX functions. I don't think so. glibc does it's own weirdness to get per-thread memory for the non thread-safe functions. So it's OK. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
Eddie Stassen <[EMAIL PROTECTED]> wrote: > It appears that the use of non thread-safe library functions are > responsible for this. Uh, yeah. Why didn't I think of that. Sorry... > I don't know how applicable these changes are to other OS's (Linux), but > perhaps someone running FR on linux could try this and see if it helps. The changes MUSt be made for safety. I'll take a look at doing some of the work over the next few days. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
At 12:33 PM 02/02/01 -0200, you wrote: >can you send the diff of all files that you fix ? > >[]s I don't have the time right now to make up the diffs (its Friday afternoon and 30 deg C outside :)), but I changed all the localtime & ctime references in the following files: src/main/xlat.c src/lib/valuepair.c src/main/log.c src/modules/rlm_detail/rlm_detail.c You have to change the 'struct tm *TM' declarations to 'struct tm TM' and the 'TM = localtime(&time)' calls to 'localtime_r(&time,&TM)'. Similar for ctime (you may have to create some temp variables though). I am not too familiar with linux, but I suspect you may need some extra #defines to enable the POSIX functions. Regards, Eddie > >Marcelo Ferreira >Canbras TVA Cabo Ltda >Canbras Acesso - STA >Phone: +5511-4993-8728 > > >- Original Message - >From: "Eddie Stassen" <[EMAIL PROTECTED]> >To: <[EMAIL PROTECTED]> >Sent: Friday, February 01, 2002 2:25 PM >Subject: Re: Thread issue - Possible fix > > > > I have been experiencing crashes as well under high accounting load > > (Solaris 7 , mysql accounting). It appears that the use of non thread-safe > > library functions are responsible for this. I have replaced all the > > localtime() and ctime() calls with their POSIX thread safe counterparts > > (localtime_r() and ctime_r()) and radiusd has been running perfectly for > > the last 2 hours under constant heavy load from radclient (1000+ req/sec) > > whereas before it would crash within a few minutes. > > > > There are a few other instances of non thread-safe library functions in >the > > source (gmtime, rand and strtok), but they are used mainly at startup so > > they should not affect the stability, although to be correct, I think they > > should be replaced with the *_r versions as well. > > > > I don't know how applicable these changes are to other OS's (Linux), but > > perhaps someone running FR on linux could try this and see if it helps. > > > > Eddie > > > > > > - > > List info/subscribe/unsubscribe? See >http://www.freeradius.org/list/users.html > > > > >- >List info/subscribe/unsubscribe? See >http://www.freeradius.org/list/users.html - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
can you send the diff of all files that you fix ? []s Marcelo Ferreira Canbras TVA Cabo Ltda Canbras Acesso - STA Phone: +5511-4993-8728 - Original Message - From: "Eddie Stassen" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, February 01, 2002 2:25 PM Subject: Re: Thread issue - Possible fix > I have been experiencing crashes as well under high accounting load > (Solaris 7 , mysql accounting). It appears that the use of non thread-safe > library functions are responsible for this. I have replaced all the > localtime() and ctime() calls with their POSIX thread safe counterparts > (localtime_r() and ctime_r()) and radiusd has been running perfectly for > the last 2 hours under constant heavy load from radclient (1000+ req/sec) > whereas before it would crash within a few minutes. > > There are a few other instances of non thread-safe library functions in the > source (gmtime, rand and strtok), but they are used mainly at startup so > they should not affect the stability, although to be correct, I think they > should be replaced with the *_r versions as well. > > I don't know how applicable these changes are to other OS's (Linux), but > perhaps someone running FR on linux could try this and see if it helps. > > Eddie > > > - > List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html > - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue - Possible fix
I have been experiencing crashes as well under high accounting load (Solaris 7 , mysql accounting). It appears that the use of non thread-safe library functions are responsible for this. I have replaced all the localtime() and ctime() calls with their POSIX thread safe counterparts (localtime_r() and ctime_r()) and radiusd has been running perfectly for the last 2 hours under constant heavy load from radclient (1000+ req/sec) whereas before it would crash within a few minutes. There are a few other instances of non thread-safe library functions in the source (gmtime, rand and strtok), but they are used mainly at startup so they should not affect the stability, although to be correct, I think they should be replaced with the *_r versions as well. I don't know how applicable these changes are to other OS's (Linux), but perhaps someone running FR on linux could try this and see if it helps. Eddie - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue
"Marcelo Ferreira" <[EMAIL PROTECTED]> wrote: > I tested an unpatched version of FR, snapshot-20020130, and > the CPU utilization issue occurred again. Ok.. > What informations can I provide to help you? > As I said before, I assume that is a thread problem. If you run the server with '-s', and the problem doesn't appear, then yes, it is a thread problem. The only thing I can suggest is to use 'gdb' to attach to the running process, to figure out where it's looping. If you're willing to give me an account on your machine, I may have time to track it down. Email me privately at the address in the CREDITS file, and we'll see. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue
Alan,
I tested an unpatched version of FR, snapshot-20020130, and
the CPU utilization issue occurred again.
What informations can I provide to help you?
As I said before, I assume that is a thread problem.
regards
Marcelo Ferreira
Canbras TVA Cabo Ltda
Canbras Acesso - STA
Phone: +5511-4993-8728
- Original Message -
From: "Tan Hwee Hong" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, January 29, 2002 12:58 PM
Subject: Re: Thread issue
> Hi,
>
> that's bad.
> it means there is something not quite right in the code.
>
> /tan
>
> On Mon, 28 Jan 2002, Marcelo Ferreira wrote:
>
> > Hi Tan,
> >
> >
> > I'm running FR with the -s option and the problem disappear..
> >
> > []s
> >
> >
> > Marcelo Ferreira
> > Canbras TVA Cabo Ltda
> > Canbras Acesso - STA
> > Phone: +5511-4993-8728
> >
> >
> > - Original Message -----
> > From: "Tan Hwee Hong" <[EMAIL PROTECTED]>
> > To: <[EMAIL PROTECTED]>
> > Sent: Sunday, January 27, 2002 1:56 AM
> > Subject: Re: Thread issue
> >
> >
> > > Hi Alan,
> > >
> > > I have encourtered similiar crashes.
> > > below is a crash dump and is reproducible. normally when that happens,
the
> > > logfile will produce the following:
> > >
> > > Error: Received Accounting-Response packet from 192.168.2.21 with
invalid
> > > signature!
> > > (note: secret was set correctly)
> > >
> > > here's the setup:
> > >
> > > OS: RedHat 7.1
> > > Freeradius version: cvs dated Jan 26 2002
> > >
> > > 192.168.2.10+
> > > 192.168.2.11---> 192.168.2.20 ---> 192.168.2.21
> > >
> > > 2 clients were used to generate the acct packets to feed the radius
server
> > > which in turn proxied the request to another server.
> > >
> > >
> > > config for proxy.conf
> > > realm testdomain.com {
> > > type= radius
> > > authhost= 192.168.2.21:1812
> > > accthost= 192.168.2.21:1813
> > > secret = secret
> > > nostrip
> > > }
> > >
> > >
> > > eg:
> > > ./bin/radclient -q -f account-pkt 192.168.2.20:1813 acct secret
> > >
> > > account-pkt was generated with the following:
> > >
> > > i=0;
> > > while [ $i -lt 2 ]; do
> > > (
> > > echo "User-Name = \"tst$[EMAIL PROTECTED]\""
> > > echo "NAS-Identifier = \"TESTING\""
> > > echo "Service-Type = Framed-User"
> > > echo "Framed-Protocol = PPP"
> > > echo "NAS-Port = 999$i"
> > > echo "Acct-Session-Id = \"$i-\""
> > > echo "Acct-Authentic = RADIUS"
> > > echo "Acct-Status-Type = Stop"
> > > echo "Session-Timeout = 22000"
> > > echo "Framed-IP-Address = 192.168.2.1"
> > > echo "Acct-Input-Octets = 212"
> > > echo "Acct-Output-Octets = 14332"
> > > echo "Acct-Input-Packets = 10"
> > > echo "Acct-Output-Packets = 141"
> > > echo "Acct-Session-Time = 254"
> > > echo ""
> > > )
> > > i=`expr $i + 1`;
> > > done
> > >
> > > $ gdb sbin/radiusd core
> > >
> > > GNU gdb Red Hat Linux (5.1-0.71)
> > > Copyright 2001 Free Software Foundation, Inc.
> > > GDB is free software, covered by the GNU General Public License, and
you
> > are
> > > welcome to change it and/or distribute copies of it under certain
> > conditions.
> > > Type "show copying" to see the conditions.
> > > There is absolutely no warranty for GDB. Type "show warranty" for
> > details.
> > > This GDB was configured as "i386-redhat-linux"...
> > > Core was generated by `sbin/radiusd'.
> > > Program terminated with signal 11, Segmentation fault.
> > > Reading symbols from /lib/libnsl.so.1...done.
> > > Loaded symbols for /lib/libnsl.so.1
> > > Reading symbols from /lib/libresolv.so.2...done.
> > > Loaded symbols for /lib/libresolv.so.2
> > > R
Re: Thread issue
Tan,
I assume that the problem is located at pthread functions (accounting
modules).
After I enable the -s option, FR hasn't crashed anymore.
regards
Marcelo Ferreira
Canbras TVA Cabo Ltda
Canbras Acesso - STA
Phone: +5511-4993-8728
- Original Message -
From: "Tan Hwee Hong" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, January 29, 2002 12:58 PM
Subject: Re: Thread issue
> Hi,
>
> that's bad.
> it means there is something not quite right in the code.
>
> /tan
>
> On Mon, 28 Jan 2002, Marcelo Ferreira wrote:
>
> > Hi Tan,
> >
> >
> > I'm running FR with the -s option and the problem disappear..
> >
> > []s
> >
> >
> > Marcelo Ferreira
> > Canbras TVA Cabo Ltda
> > Canbras Acesso - STA
> > Phone: +5511-4993-8728
> >
> >
> > - Original Message -
> > From: "Tan Hwee Hong" <[EMAIL PROTECTED]>
> > To: <[EMAIL PROTECTED]>
> > Sent: Sunday, January 27, 2002 1:56 AM
> > Subject: Re: Thread issue
> >
> >
> > > Hi Alan,
> > >
> > > I have encourtered similiar crashes.
> > > below is a crash dump and is reproducible. normally when that happens,
the
> > > logfile will produce the following:
> > >
> > > Error: Received Accounting-Response packet from 192.168.2.21 with
invalid
> > > signature!
> > > (note: secret was set correctly)
> > >
> > > here's the setup:
> > >
> > > OS: RedHat 7.1
> > > Freeradius version: cvs dated Jan 26 2002
> > >
> > > 192.168.2.10+
> > > 192.168.2.11---> 192.168.2.20 ---> 192.168.2.21
> > >
> > > 2 clients were used to generate the acct packets to feed the radius
server
> > > which in turn proxied the request to another server.
> > >
> > >
> > > config for proxy.conf
> > > realm testdomain.com {
> > > type= radius
> > > authhost= 192.168.2.21:1812
> > > accthost= 192.168.2.21:1813
> > > secret = secret
> > > nostrip
> > > }
> > >
> > >
> > > eg:
> > > ./bin/radclient -q -f account-pkt 192.168.2.20:1813 acct secret
> > >
> > > account-pkt was generated with the following:
> > >
> > > i=0;
> > > while [ $i -lt 2 ]; do
> > > (
> > > echo "User-Name = \"tst$[EMAIL PROTECTED]\""
> > > echo "NAS-Identifier = \"TESTING\""
> > > echo "Service-Type = Framed-User"
> > > echo "Framed-Protocol = PPP"
> > > echo "NAS-Port = 999$i"
> > > echo "Acct-Session-Id = \"$i-\""
> > > echo "Acct-Authentic = RADIUS"
> > > echo "Acct-Status-Type = Stop"
> > > echo "Session-Timeout = 22000"
> > > echo "Framed-IP-Address = 192.168.2.1"
> > > echo "Acct-Input-Octets = 212"
> > > echo "Acct-Output-Octets = 14332"
> > > echo "Acct-Input-Packets = 10"
> > > echo "Acct-Output-Packets = 141"
> > > echo "Acct-Session-Time = 254"
> > > echo ""
> > > )
> > > i=`expr $i + 1`;
> > > done
> > >
> > > $ gdb sbin/radiusd core
> > >
> > > GNU gdb Red Hat Linux (5.1-0.71)
> > > Copyright 2001 Free Software Foundation, Inc.
> > > GDB is free software, covered by the GNU General Public License, and
you
> > are
> > > welcome to change it and/or distribute copies of it under certain
> > conditions.
> > > Type "show copying" to see the conditions.
> > > There is absolutely no warranty for GDB. Type "show warranty" for
> > details.
> > > This GDB was configured as "i386-redhat-linux"...
> > > Core was generated by `sbin/radiusd'.
> > > Program terminated with signal 11, Segmentation fault.
> > > Reading symbols from /lib/libnsl.so.1...done.
> > > Loaded symbols for /lib/libnsl.so.1
> > > Reading symbols from /lib/libresolv.so.2...done.
> > > Loaded symbols for /lib/libresolv.so.2
> > > Reading symbols from /lib/libpthread.so.0...done.
> > &g
Re: Thread issue
Hi,
that's bad.
it means there is something not quite right in the code.
/tan
On Mon, 28 Jan 2002, Marcelo Ferreira wrote:
> Hi Tan,
>
>
> I'm running FR with the -s option and the problem disappear..
>
> []s
>
>
> Marcelo Ferreira
> Canbras TVA Cabo Ltda
> Canbras Acesso - STA
> Phone: +5511-4993-8728
>
>
> - Original Message -
> From: "Tan Hwee Hong" <[EMAIL PROTECTED]>
> To: <[EMAIL PROTECTED]>
> Sent: Sunday, January 27, 2002 1:56 AM
> Subject: Re: Thread issue
>
>
> > Hi Alan,
> >
> > I have encourtered similiar crashes.
> > below is a crash dump and is reproducible. normally when that happens, the
> > logfile will produce the following:
> >
> > Error: Received Accounting-Response packet from 192.168.2.21 with invalid
> > signature!
> > (note: secret was set correctly)
> >
> > here's the setup:
> >
> > OS: RedHat 7.1
> > Freeradius version: cvs dated Jan 26 2002
> >
> > 192.168.2.10+
> > 192.168.2.11---> 192.168.2.20 ---> 192.168.2.21
> >
> > 2 clients were used to generate the acct packets to feed the radius server
> > which in turn proxied the request to another server.
> >
> >
> > config for proxy.conf
> > realm testdomain.com {
> > type= radius
> > authhost= 192.168.2.21:1812
> > accthost= 192.168.2.21:1813
> > secret = secret
> > nostrip
> > }
> >
> >
> > eg:
> > ./bin/radclient -q -f account-pkt 192.168.2.20:1813 acct secret
> >
> > account-pkt was generated with the following:
> >
> > i=0;
> > while [ $i -lt 2 ]; do
> > (
> > echo "User-Name = \"tst$[EMAIL PROTECTED]\""
> > echo "NAS-Identifier = \"TESTING\""
> > echo "Service-Type = Framed-User"
> > echo "Framed-Protocol = PPP"
> > echo "NAS-Port = 999$i"
> > echo "Acct-Session-Id = \"$i-\""
> > echo "Acct-Authentic = RADIUS"
> > echo "Acct-Status-Type = Stop"
> > echo "Session-Timeout = 22000"
> > echo "Framed-IP-Address = 192.168.2.1"
> > echo "Acct-Input-Octets = 212"
> > echo "Acct-Output-Octets = 14332"
> > echo "Acct-Input-Packets = 10"
> > echo "Acct-Output-Packets = 141"
> > echo "Acct-Session-Time = 254"
> > echo ""
> > )
> > i=`expr $i + 1`;
> > done
> >
> > $ gdb sbin/radiusd core
> >
> > GNU gdb Red Hat Linux (5.1-0.71)
> > Copyright 2001 Free Software Foundation, Inc.
> > GDB is free software, covered by the GNU General Public License, and you
> are
> > welcome to change it and/or distribute copies of it under certain
> conditions.
> > Type "show copying" to see the conditions.
> > There is absolutely no warranty for GDB. Type "show warranty" for
> details.
> > This GDB was configured as "i386-redhat-linux"...
> > Core was generated by `sbin/radiusd'.
> > Program terminated with signal 11, Segmentation fault.
> > Reading symbols from /lib/libnsl.so.1...done.
> > Loaded symbols for /lib/libnsl.so.1
> > Reading symbols from /lib/libresolv.so.2...done.
> > Loaded symbols for /lib/libresolv.so.2
> > Reading symbols from /lib/libpthread.so.0...done.
> >
> > warning: Unable to set global thread event mask: generic error
> > [New Thread 1024 (LWP 8641)]
> > Error while reading shared library symbols:
> > Can't attach LWP 8641: No such process
> > Reading symbols from /lib/libcrypt.so.1...done.
> > Loaded symbols for /lib/libcrypt.so.1
> > Reading symbols from /usr/lib/libltdl.so.0...done.
> > Loaded symbols for /usr/lib/libltdl.so.0
> > Reading symbols from /lib/libc.so.6...done.
> > Loaded symbols for /lib/libc.so.6
> > Reading symbols from /lib/libdl.so.2...done.
> > Loaded symbols for /lib/libdl.so.2
> > Reading symbols from /lib/ld-linux.so.2...done.
> > Loaded symbols for /lib/ld-linux.so.2
> > Reading symbols from /lib/libnss_files.so.2...done.
> > Loaded symbols for /lib/libnss_files.so.2
> > Reading symbols from /lib/libnss_dns.so.2...done.
> > Loaded symbols for /lib/libnss_dns.so.2
> > Reading symbols from /usr/freeradius/lib/rlm_unix.so.0...done.
> > Loaded symbols for /usr/freeradius/lib/r
Re: Thread issue
Hi Tan,
I'm running FR with the -s option and the problem disappear..
[]s
Marcelo Ferreira
Canbras TVA Cabo Ltda
Canbras Acesso - STA
Phone: +5511-4993-8728
- Original Message -
From: "Tan Hwee Hong" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Sunday, January 27, 2002 1:56 AM
Subject: Re: Thread issue
> Hi Alan,
>
> I have encourtered similiar crashes.
> below is a crash dump and is reproducible. normally when that happens, the
> logfile will produce the following:
>
> Error: Received Accounting-Response packet from 192.168.2.21 with invalid
> signature!
> (note: secret was set correctly)
>
> here's the setup:
>
> OS: RedHat 7.1
> Freeradius version: cvs dated Jan 26 2002
>
> 192.168.2.10+
> 192.168.2.11---> 192.168.2.20 ---> 192.168.2.21
>
> 2 clients were used to generate the acct packets to feed the radius server
> which in turn proxied the request to another server.
>
>
> config for proxy.conf
> realm testdomain.com {
> type= radius
> authhost= 192.168.2.21:1812
> accthost= 192.168.2.21:1813
> secret = secret
> nostrip
> }
>
>
> eg:
> ./bin/radclient -q -f account-pkt 192.168.2.20:1813 acct secret
>
> account-pkt was generated with the following:
>
> i=0;
> while [ $i -lt 2 ]; do
> (
> echo "User-Name = \"tst$[EMAIL PROTECTED]\""
> echo "NAS-Identifier = \"TESTING\""
> echo "Service-Type = Framed-User"
> echo "Framed-Protocol = PPP"
> echo "NAS-Port = 999$i"
> echo "Acct-Session-Id = \"$i-\""
> echo "Acct-Authentic = RADIUS"
> echo "Acct-Status-Type = Stop"
> echo "Session-Timeout = 22000"
> echo "Framed-IP-Address = 192.168.2.1"
> echo "Acct-Input-Octets = 212"
> echo "Acct-Output-Octets = 14332"
> echo "Acct-Input-Packets = 10"
> echo "Acct-Output-Packets = 141"
> echo "Acct-Session-Time = 254"
> echo ""
> )
> i=`expr $i + 1`;
> done
>
> $ gdb sbin/radiusd core
>
> GNU gdb Red Hat Linux (5.1-0.71)
> Copyright 2001 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you
are
> welcome to change it and/or distribute copies of it under certain
conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for
details.
> This GDB was configured as "i386-redhat-linux"...
> Core was generated by `sbin/radiusd'.
> Program terminated with signal 11, Segmentation fault.
> Reading symbols from /lib/libnsl.so.1...done.
> Loaded symbols for /lib/libnsl.so.1
> Reading symbols from /lib/libresolv.so.2...done.
> Loaded symbols for /lib/libresolv.so.2
> Reading symbols from /lib/libpthread.so.0...done.
>
> warning: Unable to set global thread event mask: generic error
> [New Thread 1024 (LWP 8641)]
> Error while reading shared library symbols:
> Can't attach LWP 8641: No such process
> Reading symbols from /lib/libcrypt.so.1...done.
> Loaded symbols for /lib/libcrypt.so.1
> Reading symbols from /usr/lib/libltdl.so.0...done.
> Loaded symbols for /usr/lib/libltdl.so.0
> Reading symbols from /lib/libc.so.6...done.
> Loaded symbols for /lib/libc.so.6
> Reading symbols from /lib/libdl.so.2...done.
> Loaded symbols for /lib/libdl.so.2
> Reading symbols from /lib/ld-linux.so.2...done.
> Loaded symbols for /lib/ld-linux.so.2
> Reading symbols from /lib/libnss_files.so.2...done.
> Loaded symbols for /lib/libnss_files.so.2
> Reading symbols from /lib/libnss_dns.so.2...done.
> Loaded symbols for /lib/libnss_dns.so.2
> Reading symbols from /usr/freeradius/lib/rlm_unix.so.0...done.
> Loaded symbols for /usr/freeradius/lib/rlm_unix.so.0
> Reading symbols from /usr/freeradius/lib/rlm_preprocess.so.0...done.
> Loaded symbols for /usr/freeradius/lib/rlm_preprocess.so.0
> Reading symbols from /usr/freeradius/lib/rlm_realm.so.0...done.
> Loaded symbols for /usr/freeradius/lib/rlm_realm.so.0
> Reading symbols from /usr/freeradius/lib/rlm_files.so.0...done.
> Loaded symbols for /usr/freeradius/lib/rlm_files.so.0
> Reading symbols from /usr/freeradius/lib/rlm_detail.so.0...done.
> Loaded symbols for /usr/freeradius/lib/rlm_detail.so.0
> #0 0x40173292 in __libc_sendto () from /lib/libc.so.6
> (gdb) bt
> #0 0x40173292 in __libc_sendto () from /lib/libc.so.6
> #1 0x4004f38e in sendto (fd=6, buf=0x40203ae0, n=144
Re: Thread issue
Hi Alan,
I have encourtered similiar crashes.
below is a crash dump and is reproducible. normally when that happens, the
logfile will produce the following:
Error: Received Accounting-Response packet from 192.168.2.21 with invalid
signature!
(note: secret was set correctly)
here's the setup:
OS: RedHat 7.1
Freeradius version: cvs dated Jan 26 2002
192.168.2.10+
192.168.2.11---> 192.168.2.20 ---> 192.168.2.21
2 clients were used to generate the acct packets to feed the radius server
which in turn proxied the request to another server.
config for proxy.conf
realm testdomain.com {
type= radius
authhost= 192.168.2.21:1812
accthost= 192.168.2.21:1813
secret = secret
nostrip
}
eg:
./bin/radclient -q -f account-pkt 192.168.2.20:1813 acct secret
account-pkt was generated with the following:
i=0;
while [ $i -lt 2 ]; do
(
echo "User-Name = \"tst$[EMAIL PROTECTED]\""
echo "NAS-Identifier = \"TESTING\""
echo "Service-Type = Framed-User"
echo "Framed-Protocol = PPP"
echo "NAS-Port = 999$i"
echo "Acct-Session-Id = \"$i-\""
echo "Acct-Authentic = RADIUS"
echo "Acct-Status-Type = Stop"
echo "Session-Timeout = 22000"
echo "Framed-IP-Address = 192.168.2.1"
echo "Acct-Input-Octets = 212"
echo "Acct-Output-Octets = 14332"
echo "Acct-Input-Packets = 10"
echo "Acct-Output-Packets = 141"
echo "Acct-Session-Time = 254"
echo ""
)
i=`expr $i + 1`;
done
$ gdb sbin/radiusd core
GNU gdb Red Hat Linux (5.1-0.71)
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
Core was generated by `sbin/radiusd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libpthread.so.0...done.
warning: Unable to set global thread event mask: generic error
[New Thread 1024 (LWP 8641)]
Error while reading shared library symbols:
Can't attach LWP 8641: No such process
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /usr/lib/libltdl.so.0...done.
Loaded symbols for /usr/lib/libltdl.so.0
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_dns.so.2...done.
Loaded symbols for /lib/libnss_dns.so.2
Reading symbols from /usr/freeradius/lib/rlm_unix.so.0...done.
Loaded symbols for /usr/freeradius/lib/rlm_unix.so.0
Reading symbols from /usr/freeradius/lib/rlm_preprocess.so.0...done.
Loaded symbols for /usr/freeradius/lib/rlm_preprocess.so.0
Reading symbols from /usr/freeradius/lib/rlm_realm.so.0...done.
Loaded symbols for /usr/freeradius/lib/rlm_realm.so.0
Reading symbols from /usr/freeradius/lib/rlm_files.so.0...done.
Loaded symbols for /usr/freeradius/lib/rlm_files.so.0
Reading symbols from /usr/freeradius/lib/rlm_detail.so.0...done.
Loaded symbols for /usr/freeradius/lib/rlm_detail.so.0
#0 0x40173292 in __libc_sendto () from /lib/libc.so.6
(gdb) bt
#0 0x40173292 in __libc_sendto () from /lib/libc.so.6
#1 0x4004f38e in sendto (fd=6, buf=0x40203ae0, n=144, flags=0, addr=
{__sockaddr__ = 0xbf7ff9dc, __sockaddr_at__ = 0xbf7ff9dc, __sockaddr_ax25__ =
0xbf7ff9dc, __sockaddr_dl__ = 0xbf7ff9dc, __sockaddr_eon__ = 0xbf7ff9dc,
__sockaddr_in__ = 0xbf7ff9dc, __sockaddr_in6__ = 0xbf7ff9dc, __sockaddr_inarp__ =
0xbf7ff9dc, __sockaddr_ipx__ = 0xbf7ff9dc, __sockaddr_iso__ = 0xbf7ff9dc,
__sockaddr_ns__ = 0xbf7ff9dc, __sockaddr_un__ = 0xbf7ff9dc, __sockaddr_x25__ =
0xbf7ff9dc}, addr_len=16)
at wrapsyscall.c:222
#2 0x080591fc in rad_send (packet=0x40203958, original=0x0,
secret=0x80b4368 "secret") at radius.c:522
#3 0x08050a18 in proxy_send (request=0x40201b90) at proxy.c:307
#4 0x0804d83c in rad_respond (request=0x40201b90, fun=0x804f36c )
at radiusd.c:1527
#5 0x0805671c in request_handler_thread (arg=0x80bc420) at threads.c:169
#6 0x40049f87 in pthread_start_thread (arg=0xbf7ffc00) at manager.c:284
(gdb)
let me know if you need more info.
regards,
Tan
On Fri, 25 Jan 2002, [EMAIL PROTECTED] wrote:
> "Marcelo Ferreira" <[EMAIL PROTECTED]> wrote:
> > I got this error message this morning at a production test:
> >
> >
> > #0 0x401eea01 in __kill () from /lib/i686/libc.so.6
> > #1 0x400645bb in raise (sig=6) at signals.c:65
> > #2 0x401eff82 in abort () at ../s
Re: Thread issue
My authenticantion method has a particularity that has to ignore the accept/reject reply from proxy-radius and check the username in a cross-table locally. we call it "Experience Period" for new users. After 10 days, I set a "check-reply" flag at my local table to check if the proxy-radius reply with an accept packet. []s Marcelo Ferreira Canbras TVA Cabo Ltda Canbras Acesso - STA Phone: +5511-4993-8728 - Original Message - From: <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, January 25, 2002 2:22 PM Subject: Re: Thread issue > "Marcelo Ferreira" <[EMAIL PROTECTED]> wrote: > > I have the ignore_proxy_reply patch applied at auth.c, xlat.c and radius.h > > Then why didn't you say so at the start? I was trying in good faith > to answer your questions about FreeRADIUS. In turn, you did NOT tell > me you were running a patched version of the server. > > That's rude. > > > it isn't hard-complex code.. just variables values changes... > > That's irrelevant. > > > After a quick look, I can't find anything like the patch you're > referring to in the archives. I don't know what it does, so I won't > discuss ANYTHING related to it. > > Please install a LEGITIMATE, and UN-PATCHED version of the server. > If that works, then the problem is in the patch, and NOT in the > server. > > > > Please do not discuss this subject again UNLESS you can reproduce > the problem with an un-patched version of the server. > > Alan DeKok. > > - > List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html > - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue
Sorry Alan, my fault. regards Marcelo Ferreira Canbras TVA Cabo Ltda Canbras Acesso - STA Phone: +5511-4993-8728 - Original Message - From: <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, January 25, 2002 2:22 PM Subject: Re: Thread issue > "Marcelo Ferreira" <[EMAIL PROTECTED]> wrote: > > I have the ignore_proxy_reply patch applied at auth.c, xlat.c and radius.h > > Then why didn't you say so at the start? I was trying in good faith > to answer your questions about FreeRADIUS. In turn, you did NOT tell > me you were running a patched version of the server. > > That's rude. > > > it isn't hard-complex code.. just variables values changes... > > That's irrelevant. > > > After a quick look, I can't find anything like the patch you're > referring to in the archives. I don't know what it does, so I won't > discuss ANYTHING related to it. > > Please install a LEGITIMATE, and UN-PATCHED version of the server. > If that works, then the problem is in the patch, and NOT in the > server. > > > > Please do not discuss this subject again UNLESS you can reproduce > the problem with an un-patched version of the server. > > Alan DeKok. > > - > List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html > - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue
"Marcelo Ferreira" <[EMAIL PROTECTED]> wrote: > I have the ignore_proxy_reply patch applied at auth.c, xlat.c and radius.h Then why didn't you say so at the start? I was trying in good faith to answer your questions about FreeRADIUS. In turn, you did NOT tell me you were running a patched version of the server. That's rude. > it isn't hard-complex code.. just variables values changes... That's irrelevant. After a quick look, I can't find anything like the patch you're referring to in the archives. I don't know what it does, so I won't discuss ANYTHING related to it. Please install a LEGITIMATE, and UN-PATCHED version of the server. If that works, then the problem is in the patch, and NOT in the server. Please do not discuss this subject again UNLESS you can reproduce the problem with an un-patched version of the server. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue
I have the ignore_proxy_reply patch applied at auth.c, xlat.c and radius.h it isn't hard-complex code.. just variables values changes... if you want I can send my autch.c, xlat.c, radius.h, radiusd.conf, proxy.conf, clients.conf, hints, etc to you. regards Marcelo Ferreira Canbras TVA Cabo Ltda Canbras Acesso - STA Phone: +5511-4993-8728 - Original Message - From: <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, January 25, 2002 1:38 PM Subject: Re: Thread issue > "Marcelo Ferreira" <[EMAIL PROTECTED]> wrote: > > I got this error message this morning at a production test: > > > > > > #0 0x401eea01 in __kill () from /lib/i686/libc.so.6 > > #1 0x400645bb in raise (sig=6) at signals.c:65 > > #2 0x401eff82 in abort () at ../sysdeps/generic/abort.c:88 > > #3 0x0804db55 in rad_respond (request=0x8122848, fun=0x804f90c > > ) at radiusd.c:1423 > > Ouch. That's a definite bug. > > > Is there any bug at thread or pthread API's at RedHat Linux 7.2 or at > > Freeradius? > > It's an assertion in FreeRADIUS, that everything's OK. > > Since it dies, that means it found some internal inconsistency > problem, and didn't know what else to do, but kill itself. > > > What else are you doing? While I admit it's a bug in the server, if > no one else is seeing the problem, that means that your local > configuration must have something unique which makes it trigger the > bug. > > Alan DeKok. > > - > List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html > - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: Thread issue
"Marcelo Ferreira" <[EMAIL PROTECTED]> wrote: > I got this error message this morning at a production test: > > > #0 0x401eea01 in __kill () from /lib/i686/libc.so.6 > #1 0x400645bb in raise (sig=6) at signals.c:65 > #2 0x401eff82 in abort () at ../sysdeps/generic/abort.c:88 > #3 0x0804db55 in rad_respond (request=0x8122848, fun=0x804f90c > ) at radiusd.c:1423 Ouch. That's a definite bug. > Is there any bug at thread or pthread API's at RedHat Linux 7.2 or at > Freeradius? It's an assertion in FreeRADIUS, that everything's OK. Since it dies, that means it found some internal inconsistency problem, and didn't know what else to do, but kill itself. What else are you doing? While I admit it's a bug in the server, if no one else is seeing the problem, that means that your local configuration must have something unique which makes it trigger the bug. Alan DeKok. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
