Re: agent crash when proxying for Squid
Wes Hardaker wrote: On Mon, 02 May 2005 16:44:37 -0700, John Hardin <[EMAIL PROTECTED]> said: John> trace: init_agent_snmp_session(): snmp_agent.c, 1244: John> snmp_agent: agent_sesion 0x80f18c0 created John> Segmentation fault Can you run it in gdb and send us the results of a "where" call when it crashes? # gdb snmpd ... gdb> r -f -Le ... [boom ] ... gdb> where [useful stuff here] Here ya go: #0 0x4207a251 in chunk_alloc () from /lib/i686/libc.so.6 #1 0x4207a058 in malloc () from /lib/i686/libc.so.6 #2 0x40184639 in _clone_pdu_header (pdu=0x80f17c8) at snmp_client.c:327 #3 0x4018489c in _clone_pdu (pdu=0x80f17c8, drop_err=0) at snmp_client.c:509 #4 0x401848dc in snmp_clone_pdu (pdu=0x80f17c8) at snmp_client.c:537 #5 0x401390e6 in init_agent_snmp_session (session=0x81265c8, pdu=0x80f17c8) at snmp_agent.c:1246 #6 0x4013a18c in handle_snmp_packet (op=1, session=0x81265c8, reqid=1830018929, pdu=0x80f17c8, magic=0x0) at snmp_agent.c:1738 #7 0x401a84a5 in _sess_process_packet (sessp=0x8122550, sp=0x81265c8, isp=0x811d9f8, transport=0x81228e0, opaque=0x81367c0, olength=16, packetptr=0x81267b8 "0;\002\001\001\004\006public\u.\002\004m\023\uq\002\001\002\002\001", length=61) at snmp_api.c:5208 #8 0x401a932b in _sess_read (sessp=0x8122550, fdset=0xbfffe410) at snmp_api.c:5606 #9 0x401a9374 in snmp_sess_read (sessp=0x8122550, fdset=0xbfffe410) at snmp_api.c:5625 #10 0x401a85ee in snmp_read (fdset=0xbfffe410) at snmp_api.c:5260 #11 0x0804b776 in receive () at snmpd.c:1149 #12 0x0804b1d1 in main (argc=8, argv=0xb774) at snmpd.c:993 #13 0x42017499 in __libc_start_main () from /lib/i686/libc.so.6 --- This SF.Net email is sponsored by: NEC IT Guy Games. Get your fingers limbered up and give it your best shot. 4 great events, 4 opportunities to win big! Highest score wins.NEC IT Guy Games. Play to win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20 ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
Re: agent crash when proxying for Squid
> On Mon, 02 May 2005 16:44:37 -0700, John Hardin <[EMAIL PROTECTED]> said: John> trace: init_agent_snmp_session(): snmp_agent.c, 1244: John> snmp_agent: agent_sesion 0x80f18c0 created John> Segmentation fault Can you run it in gdb and send us the results of a "where" call when it crashes? # gdb snmpd ... gdb> r -f -Le ... [boom ] ... gdb> where [useful stuff here] -- Wes Hardaker Sparta, Inc. --- This SF.Net email is sponsored by: NEC IT Guy Games. Get your fingers limbered up and give it your best shot. 4 great events, 4 opportunities to win big! Highest score wins.NEC IT Guy Games. Play to win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20 ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
Re: agent crash when proxying for Squid
John Hardin wrote:
Wes Hardaker wrote:
On Mon, 02 May 2005 13:12:47 -0700, John Hardin
<[EMAIL PROTECTED]> said:
John> It looks like the agent then segfaults when the snmpbulkget
retries.
This code should have been executed in that case. Did you see this
message in your log:
if (request || var) {
/*
* ack, this is bad. The # of varbinds don't match and
* there is no way to fix the problem */
if (pdu)
snmp_free_pdu(pdu);
snmp_log(LOG_ERR,
"response to proxy request illegal. We're
screwed.\n");
netsnmp_set_request_error(cache->reqinfo, requests,
SNMP_ERR_GENERR);
}
Yes, I do see this. (It's there in my perhaps overly lengthy 1st msg.)
I've been looking at this code for something left in a strange state but
I don't see anything yet.
I again but without using the bulkget. Instead I sent an SNMPv1
getnext for the same two OIDs to snmpd. Same problem. Here's a
trace from just before the "we're screwed" log:
trace: proxy_got_response(): ucd-snmp/proxy.c, 519:
proxy: got response... SNMPv2-SMI::enterprises.3495.1.1.1.0
response to proxy request illegal. We're screwed.
trace: _sess_read(): snmp_api.c, 5289:
sess_read: not reading 8 (fdset 0xbfffe0f0 set 0)
trace: _sess_read(): snmp_api.c, 5289:
sess_read: not reading 6 (fdset 0xbfffe0f0 set 0)
trace: _sess_read(): snmp_api.c, 5289:
sess_read: not reading 4 (fdset 0xbfffe0f0 set 0)
trace: snmp_sess_select_info(): snmp_api.c, 5701:
sess_select: for all sessions: 12 10 11 8 6 4
sess_select: next alarm -1073750920.1075483508 sec
sess_select: blocking:no session requests or alarms.
trace: receive(): snmpd.c, 1114:
snmpd/select: select( numfds=13, ..., tvp=(nil))
trace: receive(): snmpd.c, 1119:
snmpd/select: returned, count = 1
trace: netsnmp_udp_recv(): snmpUDPDomain.c, 132:
netsnmp_udp: recvfrom fd 12 got 61 bytes (from UDP: [127.0.0.1]:32776)
trace: _sess_process_packet(): snmp_api.c, 4971:
sess_process_packet: session 0x8100940 fd 12 pkt 0x81267b0 length 61
Received 61 bytes from UDP: [127.0.0.1]:32776
: 30 3B 02 01 00 04 06 70 75 62 6C 69 63 A1 2E 020;.public...
0016: 04 47 CD 03 D5 02 01 00 02 01 00 30 20 30 0E 06.G.0 0..
0032: 0A 2B 06 01 04 01 9B 27 01 01 01 05 00 30 0E 06.+.'.0..
0048: 0A 2B 06 01 04 01 9B 27 01 01 02 05 00 .+.'.
Received SNMP packet(s) from UDP: [127.0.0.1]:32776
dumpx_recv:02 01 00
dumpv_recv: Integer: 0 (0x00)
trace: _snmp_parse(): snmp_api.c, 4024:
snmp_api: Parsing SNMPv1 message...
trace: _snmp_parse(): snmp_api.c, 4031:
dumph_recv: SNMPv1 message
trace: snmp_comstr_parse(): snmp_auth.c, 132:
dumph_recv: SNMP version
dumpx_recv:02 01 00
dumpv_recv: Integer: 0 (0x00)
trace: snmp_comstr_parse(): snmp_auth.c, 144:
dumph_recv: community string
dumpx_recv:04 06 70 75 62 6C 69 63
dumpv_recv: String:public
trace: _snmp_parse(): snmp_api.c, 4074:
dumph_recv: PDU
trace: snmp_pdu_parse(): snmp_api.c, 4260:
dumph_recv: request_id
dumpx_recv: 02 04 47 CD 03 D5
dumpv_recv:Integer: 1204618197 (0x47CD03D5)
trace: snmp_pdu_parse(): snmp_api.c, 4271:
dumph_recv: error status
dumpx_recv: 02 01 00
dumpv_recv:Integer: 0 (0x00)
trace: snmp_pdu_parse(): snmp_api.c, 4282:
dumph_recv: error index
dumpx_recv: 02 01 00
dumpv_recv:Integer: 0 (0x00)
trace: snmp_pdu_parse(): snmp_api.c, 4300:
dumph_recv: VarBindList
trace: snmp_pdu_parse(): snmp_api.c, 4330:
dumph_recv: VarBind
trace: snmp_parse_var_op(): snmp.c, 166:
dumph_recv: Name
dumpx_recv: 06 0A 2B 06 01 04 01 9B 27 01 01 01
dumpv_recv:ObjID: SNMPv2-SMI::enterprises.3495.1.1.1
trace: snmp_pdu_parse(): snmp_api.c, 4339:
dumph_recv: Value
trace: snmp_pdu_parse(): snmp_api.c, 4330:
dumph_recv: VarBind
trace: snmp_parse_var_op(): snmp.c, 166:
dumph_recv: Name
dumpx_recv: 06 0A 2B 06 01 04 01 9B 27 01 01 02
dumpv_recv:ObjID: SNMPv2-SMI::enterprises.3495.1.1.2
trace: snmp_pdu_parse(): snmp_api.c, 4339:
dumph_recv: Value
GETNEXT message
-- SNMPv2-SMI::enterprises.3495.1.1.1
-- SNMPv2-SMI::enterprises.3495.1.1.2
trace: init_agent_snmp_session(): snmp_agent.c, 1244:
snmp_agent: agent_sesion 0x80f18c0 created
Segmentation fault
---
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, 4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20
___
Net-snmp-users mailing list
Net-snmp-users@lists.sourceforge.net
Please see the following page to unsubscribe or change other options:
https://lists.sourceforge.net/lists/listinf
Re: agent crash when proxying for Squid
Wes Hardaker wrote:
On Mon, 02 May 2005 13:12:47 -0700, John Hardin <[EMAIL PROTECTED]> said:
John> It looks like the agent then segfaults when the snmpbulkget retries.
This code should have been executed in that case. Did you see this
message in your log:
if (request || var) {
/*
* ack, this is bad. The # of varbinds don't match and
* there is no way to fix the problem
*/
if (pdu)
snmp_free_pdu(pdu);
snmp_log(LOG_ERR,
"response to proxy request illegal. We're screwed.\n");
netsnmp_set_request_error(cache->reqinfo, requests,
SNMP_ERR_GENERR);
}
Yes, I do see this. (It's there in my perhaps overly lengthy 1st msg.)
I've been looking at this code for something left in a strange state but
I don't see anything yet.
---
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, 4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20
___
Net-snmp-users mailing list
Net-snmp-users@lists.sourceforge.net
Please see the following page to unsubscribe or change other options:
https://lists.sourceforge.net/lists/listinfo/net-snmp-users
Re: agent crash when proxying for Squid
Wes Hardaker wrote: On Mon, 02 May 2005 13:12:47 -0700, John Hardin <[EMAIL PROTECTED]> said: John> Apparently the BULK-GET received by snmpd is sent to Squid as John> a GET-NEXT with two OIDs. Ethereal reports this: John> Version: 1 (0) John> Community: public John> PDU type: GET-NEXT (1) John> Request Id: 0x60962325 John> Error Status: NO ERROR (0) John> Error Index: 0 John> Object identifier 1: 1.3.6.1.4.1.3495.1.1.1 John> (SNMPv2-SMI::enterprises.3495.1.1.1) John> Value: NULL John> Object identifier 2: 1.3.6.1.4.1.3495.1.1.2 John> (SNMPv2-SMI::enterprises.3495.1.1.2) John> Value: NULL John> Squid responds with this: John> Version: 1 (0) John> Community: public John> PDU type: RESPONSE (2) John> Request Id: 0x60962325 John> Error Status: NO ERROR (0) John> Error Index: 0 John> Object identifier 1: 1.3.6.1.4.1.3495.1.1.1.0 John> (SNMPv2-SMI::enterprises.3495.1.1.1.0) John> Value: INTEGER: 84 John> It looks like the agent then segfaults when the snmpbulkget John> retries. hmmm... lame that it's only returning one varbind when 2 were requested! not that anything should crash when this happens, mind you... Yes, lame. I've found the code in Squid that responds to SNMP requests and, sure enough, if it's a GET it will loop through all the OIDs but if it's a GETNEXT it just assumes a single OID. If I get the fix working I'll submit it the the Squid folks. --- This SF.Net email is sponsored by: NEC IT Guy Games. Get your fingers limbered up and give it your best shot. 4 great events, 4 opportunities to win big! Highest score wins.NEC IT Guy Games. Play to win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20 ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
Re: agent crash when proxying for Squid
> On Mon, 02 May 2005 13:12:47 -0700, John Hardin <[EMAIL PROTECTED]> said:
John> It looks like the agent then segfaults when the snmpbulkget retries.
This code should have been executed in that case. Did you see this
message in your log:
if (request || var) {
/*
* ack, this is bad. The # of varbinds don't match and
* there is no way to fix the problem
*/
if (pdu)
snmp_free_pdu(pdu);
snmp_log(LOG_ERR,
"response to proxy request illegal. We're screwed.\n");
netsnmp_set_request_error(cache->reqinfo, requests,
SNMP_ERR_GENERR);
}
--
Wes Hardaker
Sparta, Inc.
---
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, 4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20
___
Net-snmp-users mailing list
Net-snmp-users@lists.sourceforge.net
Please see the following page to unsubscribe or change other options:
https://lists.sourceforge.net/lists/listinfo/net-snmp-users
Re: agent crash when proxying for Squid
> On Mon, 02 May 2005 13:12:47 -0700, John Hardin <[EMAIL PROTECTED]> said: John> Apparently the BULK-GET received by snmpd is sent to Squid as John> a GET-NEXT with two OIDs. Ethereal reports this: John> Version: 1 (0) John> Community: public John> PDU type: GET-NEXT (1) John> Request Id: 0x60962325 John> Error Status: NO ERROR (0) John> Error Index: 0 John> Object identifier 1: 1.3.6.1.4.1.3495.1.1.1 John> (SNMPv2-SMI::enterprises.3495.1.1.1) John> Value: NULL John> Object identifier 2: 1.3.6.1.4.1.3495.1.1.2 John> (SNMPv2-SMI::enterprises.3495.1.1.2) John> Value: NULL John> Squid responds with this: John> Version: 1 (0) John> Community: public John> PDU type: RESPONSE (2) John> Request Id: 0x60962325 John> Error Status: NO ERROR (0) John> Error Index: 0 John> Object identifier 1: 1.3.6.1.4.1.3495.1.1.1.0 John> (SNMPv2-SMI::enterprises.3495.1.1.1.0) John> Value: INTEGER: 84 John> It looks like the agent then segfaults when the snmpbulkget John> retries. hmmm... lame that it's only returning one varbind when 2 were requested! not that anything should crash when this happens, mind you... -- Wes Hardaker Sparta, Inc. --- This SF.Net email is sponsored by: NEC IT Guy Games. Get your fingers limbered up and give it your best shot. 4 great events, 4 opportunities to win big! Highest score wins.NEC IT Guy Games. Play to win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20 ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
Re: agent crash when proxying for Squid
John Hardin wrote: I have a system where I set up Net-SNMP to proxy for Squid (the caching software). Squid is running on the same machine as Net-SNMP. Its agent is integrated into its cache daemon and only understands SNMPv1, so I have Squid's agent listening on 127.0.0.1:3401. The snmpd.conf file I give to the Net-SNMP agent contains this line: proxy -v 1 -c public 127.0.0.1:3401 .1.3.6.1.4.1.3495 The problem occurs when I do a bulk get for a couple of objects from Squid. It seems to be any two or more objects. (A single object will work ok, but why do buld get on one object. :-) Below is a trace from the agent. I'm running the agent on port 32000 instead of 161 only to isolate it from any other SNMP requests (for example HP OV doing discovery). I'm runing the agent on a Linux box running kernel 2.4.26. $ /usr/sbin/snmpd -c /etc/snmp/snmpd.conf -C -f -d -p -Lo 10.1.0.36:32000 NET-SNMP version 5.2.1 Received 61 bytes from UDP: [10.1.2.10]:36951 : 30 3B 02 01 01 04 06 70 75 62 6C 69 63 A5 2E 02 0;.public... 0016: 04 20 90 02 54 02 01 02 02 01 00 30 20 30 0E 06. ..T..0 0.. 0032: 0A 2B 06 01 04 01 9B 27 01 01 01 05 00 30 0E 06 .+.'.0.. 0048: 0A 2B 06 01 04 01 9B 27 01 01 02 05 00 .+.'. Received SNMP packet(s) from UDP: [10.1.2.10]:36951 GETBULK message, non-rep=2, max_rep=0 -- SNMPv2-SMI::enterprises.3495.1.1.1 -- SNMPv2-SMI::enterprises.3495.1.1.2 Sending 61 bytes to UDP: [127.0.0.1]:3401 : 30 3B 02 01 00 04 06 70 75 62 6C 69 63 A1 2E 02 0;.public... 0016: 04 63 1C A3 89 02 01 00 02 01 00 30 20 30 0E 06 .c.0 0.. 0032: 0A 2B 06 01 04 01 9B 27 01 01 01 05 00 30 0E 06 .+.'.0.. 0048: 0A 2B 06 01 04 01 9B 27 01 01 02 05 00 .+.'. Received 55 bytes from UDP: [127.0.0.1]:3401 : 30 82 00 33 02 01 00 04 06 70 75 62 6C 69 63 A2 0..3.public. 0016: 82 00 24 02 04 63 1C A3 89 02 01 00 02 01 00 30 ..$..c.0 0032: 82 00 14 30 82 00 10 06 0B 2B 06 01 04 01 9B 27 ...0.+.' 0048: 01 01 01 00 02 01 54 ..T response to proxy request illegal. We're screwed. Received 61 bytes from UDP: [10.1.2.10]:36951 : 30 3B 02 01 01 04 06 70 75 62 6C 69 63 A5 2E 02 0;.public... 0016: 04 20 90 02 54 02 01 02 02 01 00 30 20 30 0E 06. ..T..0 0.. 0032: 0A 2B 06 01 04 01 9B 27 01 01 01 05 00 30 0E 06 .+.'.0.. 0048: 0A 2B 06 01 04 01 9B 27 01 01 02 05 00 .+.'. Received SNMP packet(s) from UDP: [10.1.2.10]:36951 Segmentation fault Heres the command I use to reproduce this: snmpbulkget -c public -v2c -Cn2 -Cr0 myserver:32000 \ SNMPv2-SMI::enterprises.3495.1.1.1 \ SNMPv2-SMI::enterprises.3495.1.1.2 I haven't hand-decoded the traffic above, yet, but thought I'd share what I have so far. Please let me know if this is a known issue. Thanks. -John Apparently the BULK-GET received by snmpd is sent to Squid as a GET-NEXT with two OIDs. Ethereal reports this: Version: 1 (0) Community: public PDU type: GET-NEXT (1) Request Id: 0x60962325 Error Status: NO ERROR (0) Error Index: 0 Object identifier 1: 1.3.6.1.4.1.3495.1.1.1 (SNMPv2-SMI::enterprises.3495.1.1.1) Value: NULL Object identifier 2: 1.3.6.1.4.1.3495.1.1.2 (SNMPv2-SMI::enterprises.3495.1.1.2) Value: NULL Squid responds with this: Version: 1 (0) Community: public PDU type: RESPONSE (2) Request Id: 0x60962325 Error Status: NO ERROR (0) Error Index: 0 Object identifier 1: 1.3.6.1.4.1.3495.1.1.1.0 (SNMPv2-SMI::enterprises.3495.1.1.1.0) Value: INTEGER: 84 It looks like the agent then segfaults when the snmpbulkget retries. --- This SF.Net email is sponsored by: NEC IT Guy Games. Get your fingers limbered up and give it your best shot. 4 great events, 4 opportunities to win big! Highest score wins.NEC IT Guy Games. Play to win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20 ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
agent crash when proxying for Squid
I have a system where I set up Net-SNMP to proxy for Squid (the caching software). Squid is running on the same machine as Net-SNMP. Its agent is integrated into its cache daemon and only understands SNMPv1, so I have Squid's agent listening on 127.0.0.1:3401. The snmpd.conf file I give to the Net-SNMP agent contains this line: proxy -v 1 -c public 127.0.0.1:3401 .1.3.6.1.4.1.3495 The problem occurs when I do a bulk get for a couple of objects from Squid. It seems to be any two or more objects. (A single object will work ok, but why do buld get on one object. :-) Below is a trace from the agent. I'm running the agent on port 32000 instead of 161 only to isolate it from any other SNMP requests (for example HP OV doing discovery). I'm runing the agent on a Linux box running kernel 2.4.26. $ /usr/sbin/snmpd -c /etc/snmp/snmpd.conf -C -f -d -p -Lo 10.1.0.36:32000 NET-SNMP version 5.2.1 Received 61 bytes from UDP: [10.1.2.10]:36951 : 30 3B 02 01 01 04 06 70 75 62 6C 69 63 A5 2E 020;.public... 0016: 04 20 90 02 54 02 01 02 02 01 00 30 20 30 0E 06. ..T..0 0.. 0032: 0A 2B 06 01 04 01 9B 27 01 01 01 05 00 30 0E 06.+.'.0.. 0048: 0A 2B 06 01 04 01 9B 27 01 01 02 05 00 .+.'. Received SNMP packet(s) from UDP: [10.1.2.10]:36951 GETBULK message, non-rep=2, max_rep=0 -- SNMPv2-SMI::enterprises.3495.1.1.1 -- SNMPv2-SMI::enterprises.3495.1.1.2 Sending 61 bytes to UDP: [127.0.0.1]:3401 : 30 3B 02 01 00 04 06 70 75 62 6C 69 63 A1 2E 020;.public... 0016: 04 63 1C A3 89 02 01 00 02 01 00 30 20 30 0E 06.c.0 0.. 0032: 0A 2B 06 01 04 01 9B 27 01 01 01 05 00 30 0E 06.+.'.0.. 0048: 0A 2B 06 01 04 01 9B 27 01 01 02 05 00 .+.'. Received 55 bytes from UDP: [127.0.0.1]:3401 : 30 82 00 33 02 01 00 04 06 70 75 62 6C 69 63 A20..3.public. 0016: 82 00 24 02 04 63 1C A3 89 02 01 00 02 01 00 30..$..c.0 0032: 82 00 14 30 82 00 10 06 0B 2B 06 01 04 01 9B 27...0.+.' 0048: 01 01 01 00 02 01 54 ..T response to proxy request illegal. We're screwed. Received 61 bytes from UDP: [10.1.2.10]:36951 : 30 3B 02 01 01 04 06 70 75 62 6C 69 63 A5 2E 020;.public... 0016: 04 20 90 02 54 02 01 02 02 01 00 30 20 30 0E 06. ..T..0 0.. 0032: 0A 2B 06 01 04 01 9B 27 01 01 01 05 00 30 0E 06.+.'.0.. 0048: 0A 2B 06 01 04 01 9B 27 01 01 02 05 00 .+.'. Received SNMP packet(s) from UDP: [10.1.2.10]:36951 Segmentation fault Heres the command I use to reproduce this: snmpbulkget -c public -v2c -Cn2 -Cr0 myserver:32000 \ SNMPv2-SMI::enterprises.3495.1.1.1 \ SNMPv2-SMI::enterprises.3495.1.1.2 I haven't hand-decoded the traffic above, yet, but thought I'd share what I have so far. Please let me know if this is a known issue. Thanks. -John --- This SF.Net email is sponsored by: NEC IT Guy Games. Get your fingers limbered up and give it your best shot. 4 great events, 4 opportunities to win big! Highest score wins.NEC IT Guy Games. Play to win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20 ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
