Bugfix for bulk gets in SNMP.pm
Hi, We've been using SNMP.pm to pull out various tables from our switches into our home-grown network management system. However, there's one thing that's been causing issues for us for a long time, and that's getting the LLDP tables (lldpRemTable). The basic issues is that the values, at least when you query Cisco equipment, tends to be so big that the packets get truncated. There's logic to try to estimate the result size in the module and scale back accordingly on the number of repetitions, but it fails in this case because the average value is so big (it typically contains the entire “show ver” for the remote device). This causes “ack: gettable results not appropriate”, even the results _are_ fully appropriate. (We've been turning off bulk gets, but it's not an ideal solution, and in any case, it can still fail in the same way, it just needs bigger values to do so.) The attached patch (in context diff, because unified diff became unusually unreadable!) first rewrites _gettable_do_it to be a bit less $#{$ everywhere (so that I could understand it myself :-) ), and then fixes so that it properly deals with a truncated response: For each value returned, the right varbind is advanced, and the decision of which ones to keep querying is delayed until the very end. We've queried a variety of equipment we've got around (mostly Cisco and HP, though), and this seems to fix the issue for us. Would you consider this for inclusion in a future version? Thanks! /* Steinar */ -- Homepage: http://www.sesse.net/ *** /usr/lib/perl5/SNMP.pm 2009-10-29 17:05:23.0 +0100 --- include/SNMP.old.pm 2014-04-01 00:29:08.418112879 +0200 *** *** 790,811 $vbl = $_[$#_] if ($state->{'options'}{'callback'}); ! while ($#$vbl > -1 && !$this->{ErrorNum}) { ! if (($#$vbl + 1) % ($#{$state->{'stopconds'}} + 1) != 0) { ! if ($vbl->[$#$vbl][2] ne 'ENDOFMIBVIEW') { ! # unless it's an end of mib view we didn't get the ! # proper number of results back. ! print STDERR "ack: gettable results not appropriate\n"; ! } ! my @k = keys(%{$state->{'result_hash'}}); ! last if ($#k > -1); # bail with what we have ! return; ! } ! ! $state->{'varbinds'} = []; ! my $newstopconds; ! my $lastsetstart = ($state->{'repeatcount'}-1) * ($#{$state->{'stopconds'}}+1); for (my $i = 0; $i <= $#$vbl; $i++) { my $row_oid = SNMP::translateObj($vbl->[$i][0]); --- 790,800 $vbl = $_[$#_] if ($state->{'options'}{'callback'}); ! my $num_vbls = scalar @$vbl; ! my $num_stopconds = scalar @{$state->{'stopconds'}}; ! while ($num_vbls > 0 && !$this->{ErrorNum}) { ! my @found_eof = (0) x $num_stopconds; for (my $i = 0; $i <= $#$vbl; $i++) { my $row_oid = SNMP::translateObj($vbl->[$i][0]); *** *** 814,822 my $row_value = $vbl->[$i][2]; my $row_type = $vbl->[$i][3]; ! if ($row_oid =~ ! /^$state->{'stopconds'}[$i % ($#{$state->{'stopconds'}}+1)]/ && ! $row_value ne 'ENDOFMIBVIEW' ){ if ($row_type eq "OBJECTID") { --- 803,813 my $row_value = $vbl->[$i][2]; my $row_type = $vbl->[$i][3]; ! my $stopcond_num = $i % $num_stopconds; ! my $stopcond = $state->{'stopconds'}[$stopcond_num]; ! if ($row_oid !~ /^\Q$stopcond\E/ || $row_value eq 'ENDOFMIBVIEW') { ! $found_eof[$stopcond_num] = 1; ! } else { if ($row_type eq "OBJECTID") { *** *** 827,852 } # Place the results in a hash $state->{'result_hash'}{$row_index}{$row_text} = $row_value; ! # continue past this next time ! if ($i >= $lastsetstart) { ! push @$newstopconds, ! $state->{'stopconds'}->[$i%($#{$state->{'stopconds'}}+1)]; ! push @{$state->{'varbinds'}},[$vbl->[$i][0],$vbl->[$i][1]]; ! } } } ! if ($#$newstopconds == -1) { last; } ! if ($#{$state->{'varbinds'}} == -1) { ! print "gettable ack. shouldn't get here\n"; ! } $vbl = $state->{'varbinds'}; - $state->{'stopconds'} = $newstopconds; # # if we've been configured with a callback, then call the --- 818,847 } + # continue past this next time + + $state->{'varbinds'}[$stopcond_num] = [ $row_text, $row_index ]; + # Place the results in a hash $state->{'result_hash'}{$row_index}{$row_text} = $row_value; + } + } ! my @newstopconds = (); ! my @newvarbinds = (); ! for (my $i = 0; $i < $num_stopconds; ++$i) { ! unless ($found_eof[$i]) { ! push @newstopconds, $state->{'stopconds'}[$i]; ! push @newvarbinds, $state->{'varbinds'}[$i]; } } ! if ($#newstopconds == -1) { last; } ! $state->{'varbinds'} = \@newvarbinds; ! $state->{'stopconds'} = \@newstopconds; $vbl = $state->{'varbinds'}; # # if we've been configured with a callback, then call the -- ___
Re: Bugfix for bulk gets in SNMP.pm
On Tue, Apr 01, 2014 at 01:41:12PM +0200, Steinar H. Gunderson wrote: > The attached patch (in context diff, because unified diff became unusually > unreadable!) first rewrites _gettable_do_it to be a bit less $#{$ everywhere > (so that I could understand it myself :-) ), and then fixes so that it > properly deals with a truncated response: For each value returned, the right > varbind is advanced, and the decision of which ones to keep querying is > delayed until the very end. > > We've queried a variety of equipment we've got around (mostly Cisco and HP, > though), and this seems to fix the issue for us. Would you consider this for > inclusion in a future version? Hi, I don't think anyone picked up this. Bumping so it isn't forgotten. /* Steinar */ -- Homepage: http://www.sesse.net/ -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test & Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Net-snmp-coders mailing list Net-snmp-coders@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
Re: Bugfix for bulk gets in SNMP.pm
On Wed, Apr 09, 2014 at 09:34:47AM -0400, Bill Fenner wrote: > Thanks for the patch. In order to make sure that we don't lose track of > it, could you please upload it to http://www.net-snmp.org/patches/ ? Done; thanks. /* Steinar */ -- Homepage: http://www.sesse.net/ -- Put Bad Developers to Shame Dominate Development with Jenkins Continuous Integration Continuously Automate Build, Test & Deployment Start a new project now. Try Jenkins in the cloud. http://p.sf.net/sfu/13600_Cloudbees ___ Net-snmp-coders mailing list Net-snmp-coders@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
[PATCH] Fix doubled calls to callback for SNMPv3 reports
Hi, I've been bothered with my Perl scripts segfaulting due to use of freed data; it seems that under certain circumstances (e.g., when retries are needed), you can get callbacks for SNMP reports multiple times, and there is no way that I can see for the script to know which one is the last one or not. (Even if it did try to reproduce all the logic, it doesn't have access to the retry count member.) It seems to me that the callback should simply never be called for something like SNMPERR_NOT_IN_TIME_WINDOW; the library should just go about its business and only call the callback when it actually has a decision (even if that decision is just “it timed out”, of coures). I've attached a patch that does exactly this; it seems to fix my segfaults. I still think it's a bit odd that in the success branches, it suddenly assumes the magic value is of type struct synch_state* (for good measure, with a cast-as-lvalue, which I believe is undefined behavior wrt. aliasing); I haven't done anything about this. /* Steinar */ -- Homepage: http://www.sesse.net/ Index: net-snmp-5.7.2.1~dfsg/snmplib/snmp_api.c === --- net-snmp-5.7.2.1~dfsg.orig/snmplib/snmp_api.c 2014-10-19 21:22:42.851135962 +0200 +++ net-snmp-5.7.2.1~dfsg/snmplib/snmp_api.c 2014-10-19 21:25:05.988325201 +0200 @@ -5319,67 +5319,59 @@ * should be per session ! */ - if (callback == NULL - || callback(NETSNMP_CALLBACK_OP_RECEIVED_MESSAGE, sp, - pdu->reqid, pdu, magic) == 1) { - if (pdu->command == SNMP_MSG_REPORT) { - if (sp->s_snmp_errno == SNMPERR_NOT_IN_TIME_WINDOW || - snmpv3_get_report_type(pdu) == - SNMPERR_NOT_IN_TIME_WINDOW) { - /* - * trigger immediate retry on recoverable Reports - * * (notInTimeWindow), incr_retries == TRUE to prevent - * * inifinite resend - */ - if (rp->retries <= sp->retries) { - snmp_resend_request(slp, rp, TRUE); - break; - } else { - /* We're done with retries, so no longer waiting for a response */ - ((struct synch_state*)magic)->waiting = 0; - } - } else { - if (SNMPV3_IGNORE_UNAUTH_REPORTS) { - break; - } else { /* Set the state to no longer be waiting, since we're done with retries */ - ((struct synch_state*)magic)->waiting = 0; - } - } - + if (pdu->command == SNMP_MSG_REPORT) { + if (sp->s_snmp_errno == SNMPERR_NOT_IN_TIME_WINDOW || + snmpv3_get_report_type(pdu) == + SNMPERR_NOT_IN_TIME_WINDOW) { /* - * Handle engineID discovery. + * trigger immediate retry on recoverable Reports + * * (notInTimeWindow), incr_retries == TRUE to prevent + * * inifinite resend */ - if (!sp->securityEngineIDLen && pdu->securityEngineIDLen) { - sp->securityEngineID = - (u_char *) malloc(pdu->securityEngineIDLen); - if (sp->securityEngineID == NULL) { - /* - * TODO FIX: recover after message callback *? - */ + if (rp->retries <= sp->retries) { + snmp_resend_request(slp, rp, TRUE); + break; + } else { + /* We're done with retries, so no longer waiting for a response */ + ((struct synch_state*)magic)->waiting = 0; + } + } else { + if (SNMPV3_IGNORE_UNAUTH_REPORTS) { + break; + } else { /* Set the state to no longer be waiting, since we're done with retries */ + ((struct synch_state*)magic)->waiting = 0; + } + } + + /* + * Handle engineID discovery. + */ + if (!sp->securityEngineIDLen && pdu->securityEngineIDLen) { + sp->securityEngineID = + (u_char *) malloc(pdu->securityEngineIDLen); + if (sp->securityEngineID == NULL) { + return -1; + } + memcpy(sp->securityEngineID, pdu->securityEngineID, + pdu->securityEngineIDLen); + sp->securityEngineIDLen = pdu->securityEngineIDLen; + if (!sp->contextEngineIDLen) { + sp->contextEngineID = + (u_char *) malloc(pdu->securityEngineIDLen); + if (sp->contextEngineID == NULL) { return -1; } - memcpy(sp->securityEngineID, pdu->securityEngineID, + memcpy(sp->contextEngineID, + pdu->securityEngineID, pdu->securityEngineIDLen); - sp->securityEngineIDLen = pdu->securityEngineIDLen; - if (!sp->contextEngineIDLen) { - sp->contextEngineID = - (u_char *) malloc(pdu-> - securityEngineIDLen); - if (sp->contextEngineID == NULL) { - /* - * TODO FIX: recover after message callback *? - */ -return -1; - } - memcpy(sp->contextEngineID, - pdu->securityEngineID, - pdu->securityEngineIDLen); - sp->contextEngineIDLen = - pdu->securityEngineIDLen; - } + sp->contextEngineIDLen = pdu->securityEngineIDLen; } } + } + if (callback == NULL || + callback(NETSNMP_CALLBACK_OP_RECEIVED_MESSAGE, sp, + pdu->reqid, pdu, magic) == 1) { /* * Successful, so delete request. */ -
[PATCH] Let Perl access the security engine ID
Hi, When using SNMP::Session->new() without a SecEngineId, it is probed. However, there's no way to actually find the probed value from Perl (which would be useful for e.g. storing it for faster later reconnect; the probing is synchronous and can take a long time, especially if the host is done). I've attached a patch that adds two new functions for exactly this. /* Steinar */ -- Homepage: http://www.sesse.net/ Index: net-snmp-5.7.2.1~dfsg/perl/SNMP/SNMP.pm === --- net-snmp-5.7.2.1~dfsg.orig/perl/SNMP/SNMP.pm 2014-10-19 22:12:02.852839926 +0200 +++ net-snmp-5.7.2.1~dfsg/perl/SNMP/SNMP.pm 2014-10-19 22:20:12.0 +0200 @@ -1243,6 +1243,16 @@ return(wantarray() ? @res : $res[0]); } +sub get_sec_engine_id { + my $this = shift; + return SNMP::_get_sec_engine_id($this); +} + +sub get_context_engine_id { + my $this = shift; + return SNMP::_get_context_engine_id($this); +} + package SNMP::TrapSession; @SNMP::TrapSession::ISA = ('SNMP::Session'); @@ -2039,6 +2049,17 @@ =back +=item $sess->get_sec_engine_id + +Returns the security engine ID for the current session, whether probed +or provided by the client, in hex format suitable for the SecEngineId +parameter when creating a session in the future. Returns undef if we have not +had not had any contact with the remote agent yet. + +=item $sess->get_context_engine_id + +Like get_sec_engine_id, but for the context engine ID (ContextEngineId). + =back =head1 SNMP::TrapSession Index: net-snmp-5.7.2.1~dfsg/perl/SNMP/SNMP.xs === --- net-snmp-5.7.2.1~dfsg.orig/perl/SNMP/SNMP.xs 2014-10-19 22:20:12.0 +0200 +++ net-snmp-5.7.2.1~dfsg/perl/SNMP/SNMP.xs 2014-10-19 22:22:18.732965363 +0200 @@ -4740,6 +4740,50 @@ char * +snmp_get_sec_engine_id(sess_ref) +SV * sess_ref + CODE: + { + RETVAL = NULL; + if (SvROK(sess_ref)) { + SV **sess_ptr_sv = hv_fetch((HV*)SvRV(sess_ref), "SessPtr", 7, 1); + SnmpSession *ss = (SnmpSession *)SvIV((SV*)SvRV(*sess_ptr_sv)); + if (ss->securityEngineIDLen > 0) { + binary_to_hex(ss->securityEngineID, + ss->securityEngineIDLen, + &RETVAL); + } + } + } + OUTPUT: +RETVAL +CLEANUP: +netsnmp_free(RETVAL); + + +char * +snmp_get_context_engine_id(sess_ref) +SV * sess_ref + CODE: + { + RETVAL = NULL; + if (SvROK(sess_ref)) { + SV **sess_ptr_sv = hv_fetch((HV*)SvRV(sess_ref), "SessPtr", 7, 1); + SnmpSession *ss = (SnmpSession *)SvIV((SV*)SvRV(*sess_ptr_sv)); + if (ss->contextEngineIDLen > 0) { + binary_to_hex(ss->contextEngineID, + ss->contextEngineIDLen, + &RETVAL); + } + } + } + OUTPUT: +RETVAL +CLEANUP: +netsnmp_free(RETVAL); + + +char * snmp_get_type(tag, best_guess) char * tag int best_guess -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho___ Net-snmp-coders mailing list Net-snmp-coders@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
Re: [PATCH] Let Perl access the security engine ID
On Sun, Oct 19, 2014 at 09:05:05PM -0400, Bill Fenner wrote: > Thanks for the patches. Could you please submit them at > http://www.net-snmp.org/patches/ so that we don't lose track of them? Seemingly the exact same problem as one of my patches was found and patched by someone else: http://sourceforge.net/p/net-snmp/patches/1256/ Filed June 2013, no response from upstream. I have another patch stuck since April this year; also no response. Is there anyone actually looking at these patches? /* Steinar */ -- Homepage: http://www.sesse.net/ -- Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho ___ Net-snmp-coders mailing list Net-snmp-coders@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/net-snmp-coders