I'm having issues with sending traps using send_v2trap() from an AgentX
subagent on Solaris 9.
When the subagent sends series of traps, the subagent and master agent
both get blocked on the sendto() call. It does not matter if there is any
delay between sending traps. But interestingly blocking happens at the
exact number of traps each time (34th trap in my case).. is the domain
socket's buffer getting full? When the problem happens, obviously master
agent does not respond to any snmp queries.
To isolate the problem, I wrote a simple test subagent which only sends nothing
but traps. Enterprise trap PDU is about 330 bytes.
If subagent is killed, I can see the master agent sends another 34 traps.
MY workaround is to create a separate session for sending traps, construct
the trap PDUs myself and send them, which works. But I rather not do that
if possible.
Any insight into this problem will be appreciated.
Some debug information is shown below:
===============================================================================
OS: Solaris 9
Net-SNMP versions: 5.4.2.1 and 5.5
Communication: unix domain socket. But problem happens if TCP is used as well.
Subagent's pid: 13375
# pstack 13375
ff19cf54 sendto (7, 3493d0, 290, 0, 0, 0)
000c563c _sess_async_send (25e618, 33eec0, 48080, 0, 2710, 0) + f3c
000c5994 snmp_sess_async_send (25e618, 33eec0, 48080, 0, 0, 0) + 74
000c46d0 snmp_async_send (2671f8, 33eec0, 48080, 0, 0, 33f0c9) + 40
00048798 send_trap_to_sess (2671f8, 1b0aa0, 17e724, 9, ff07b5b0, 4) + 240
00047f60 netsnmp_send_traps (ffffffff, ffffffff, 17e810, a, 2b2d30, 0) + 878
00048054 send_enterprise_trap_vars (ffffffff, ffffffff, 17e810, a, 2b2d30, 0)
+ 44
00048940 send_trap_vars (ffffffff, ffffffff, 2b2d30, fffffff8, 0, 2bcdcd) +
98
00048a8c send_v2trap (2b2d30, ff07b760, d, 2, ff07bc0c, 4) + 14
0002ef90 test_check_trap_queue (2, 0, 1510f1d, ff1bf2e8, ff07be18,
ff07bd8c) + 1e0
000fe548 run_alarms (0, ff07be18, 0, 0, ff07be10, ff07bea1) + 1d0
00031a30 agent_check_and_process (1, 12d568, 2, 0, 0, 0) + 1e8
0002f058 test_snmp_handler (0, 0, 0, 0, 0, 0) + 58
# lsof -p 13375
...
test_tra 13375 root 7u unix 105,40 0t0 625
/devices/pseudo/t...@0:ticots->/var/snmp/master (0x300056cb1d8->0x30028056398)
...
# ps -eaf|grep snmpd
root 13370 13369 0 14:28:15 pts/15 0:01 snmpd -f -Dagentx
# pstack 13370
ff19cf54 sendto (b, 463ea8, 298, 0, 0, 0)
001aa23c _sess_async_send (437d50, 454f80, 0, 0, 78, 78) + f3c
001aa594 snmp_sess_async_send (437d50, 454f80, 0, 0, 78, 78) + 74
001a92d0 snmp_async_send (440fe8, 454f80, 0, 0, 165ae, 400fe4) + 40
001a9210 snmp_send (440fe8, 454f80, 0, a477b11, 0, 6) + 20
00089980 handle_master_agentx_packet (1, 440fe8, a477b11, 4fff68, 0, 290) +
7a8
001abb78 _sess_process_packet (437d50, 440fe8, 3eb818, 3eb750, 538ee0, 6e) +
ff8
001ad5e8 _sess_read (444ac8, ffbff468, 5, 7efefeff, 61, 73) + 1740
001adc90 snmp_sess_read2 (437d50, ffbff468, 21a0c0, ffbff288, 0, 0) + 30
001abe68 snmp_read2 (ffbff468, ffbff468, ffbff3e0, ffbff358, 4a5, 0) + 50
0004a3cc receive (0, 4, 2127c0, 2b13cc, 41d, ff00) + 6dc
00049934 main (3, ffbff69c, ffbff6ac, 2ad400, 0, 0) + 243c
00046b70 _start (0, 0, 0, 0, 0, 0) + 108
# lsof -p 13370
....
snmpd 13370 root 10u unix 105,39 0t0 625
/devices/pseudo/t...@0:ticots->/var/snmp/master (0x300056cb8f8)
(Vnode=0x30028056398)
snmpd 13370 root 11u unix 105,41 0t0 625
/devices/pseudo/t...@0:ticots->/var/snmp/master (0x300056cbc88)
snmpd 13370 root 12u unix 105,43 0t0 625
/devices/pseudo/t...@0:ticots->/var/snmp/master (0x300056cb730)
....
===============================================================================
Thanks.
--
Muhammad A Muquit . Alcatel-Lucent
------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
