Re: ucd-snmp vs. net-snmp (foot print)
2009/7/24 Juliana Purjo : > I am working on ucd-snmp (version 4.2.6), extending a set of private company > MIBs. I have been told the foot-print of ucd-snmp is significantly lower > than that of newer 5.x.x series net-snmp - Is this true? Probably. There are a number of additional MIB modules with the 5.x line, as well as various other features, so it's quite probable that the code will slowly increase in size. That tends to be true of almost any piece of software. > what would it take to reduce the foot print. Please see the FAQ, which has suggestions regarding minimizing the size of the compiled and running software. > I notice the mib2c is completely rewritten which seems a benefit (I am using > customized configuration files and 5.x.x mib2c seems easier to work with). > Is there other notable reasons to consider moving to 5.x.x? It's not just mib2c - the v5 agent has a completely new handler mechanism for implementing MIB modules. Not only is it easier to write code for MIB tables, the processing should also be significantly more efficient, since related varbinds can be processed in parallel, rather than treating each varbind separately. Large chunks of the agent infrastructure have also been completely re-written, to support the new handler-based architecture. In fact, the old v4-style approach is now simply a wrapper sitting on top of the new handler mechanism, so there are additional overheads with this style of code. And of course, the v4 line is no longer being supported, so if you stick with this, then you are completely on your own. Unfortunately, we can't really provide you with any significant implementation experience of running the Net-SNMP agent on embedded systems. I keep putting out a plea for those who have done this to report back, but nobody ever seems to bother. There's a first-draft entry in the latest version of the FAQ, but it is in desperate need of some real feedback from those who have on-the-ground experience with doing this. Here's hoping Dave -- ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
RE: ucd-snmp vs. net-snmp (foot print)
Hello Dave, Kind thanks for your reply. And sorry for late reply due to a travel. > Date: Fri, 24 Jul 2009 21:58:50 +0100 > Subject: Re: ucd-snmp vs. net-snmp (foot print) > From: d.t.shi...@liverpool.ac.uk > To: jpu...@live.com > CC: net-snmp-users@lists.sourceforge.net > > 2009/7/24 Juliana Purjo : > > I am working on ucd-snmp (version 4.2.6), extending a set of private company > > MIBs. I have been told the foot-print of ucd-snmp is significantly lower > > than that of newer 5.x.x series net-snmp - Is this true? > > Probably. > There are a number of additional MIB modules with the 5.x line, > as well as various other features, so it's quite probable that the > code will slowly increase in size. That tends to be true of almost > any piece of software. Concerning MIB modules... Having looked at 4.2.x code base, I got an impression there is some code repetition in the agent implementations. Also there seems to be quite a lot of static buffers (defined inside functions) that are mostly sitting unused eating runtime memory? So I feel there might be possiblity to also decrease some code size contrary to the statistically provable tendency of software sizes growing. Easy way to save some code size - for the new code - might be to edit the mib2c configuration files to not generate so repetitive code constructs (maybe using more powerful table structures)??? But maybe it is more interesting to create new features and functionality than restructuring old code... :-) Is it quick to outline what kind of additional features are we talking here? Thinking of replacing v4 net-snmp, is it possible to disable most of this new weigth (with appropriate ./configure invocation)? Below you already mention new handler mechanism for MIB modules and wrapper code to support v4 style coding approach, which seems necessary weight. > > what would it take to reduce the foot print. > > Please see the FAQ, which has suggestions regarding minimizing > the size of the compiled and running software. > Thanks for the pointer. Some good hints. And running ./configure --help gives some additional ideas... > > > I notice the mib2c is completely rewritten which seems a benefit (I am using > > customized configuration files and 5.x.x mib2c seems easier to work with). > > Is there other notable reasons to consider moving to 5.x.x? > > It's not just mib2c - the v5 agent has a completely new handler mechanism > for implementing MIB modules. Not only is it easier to write code > for MIB tables, > the processing should also be significantly more efficient, since > related varbinds > can be processed in parallel, rather than treating each varbind separately. > Very nice. > Large chunks of the agent infrastructure have also been completely > re-written, to support the new handler-based architecture. In fact, the > old v4-style approach is now simply a wrapper sitting on top of the new > handler mechanism, so there are additional overheads with this style of code. I am happy to read from the FAQ that most agent code written for earlier releases should work okay with newest net-snmp (altough with some performance/foot print penalty). > And of course, the v4 line is no longer being supported, so if you stick with > this, then you are completely on your own. Yes, I can definitively understand this point. This is one major part of what triggered me to ask my question in the first place... :-) > > Unfortunately, we can't really provide you with any significant implementation > experience of running the Net-SNMP agent on embedded systems. I keep > putting out a plea for those who have done this to report back, but nobody > ever seems to bother. > There's a first-draft entry in the latest version of the FAQ, but > it is in desperate > need of some real feedback from those who have on-the-ground experience with > doing this. > > Here's hoping I am wishing to be able to experiment with newest version and do some quantitative and qualitative comparison - it certainly seems worthwhile to me - but I am afraid I am not able to with the time constraints I have to deal with at the moment... :-( > Dave _ With Windows Live, you can organize, edit, and share your photos. http://www.microsoft.com/middleeast/windows/windowslive/products/photo-gallery-edit.aspx-- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.n
Re: ucd-snmp vs. net-snmp (foot print)
2009/7/30 Juliana Purjo : > Concerning MIB modules... Having looked at 4.2.x code base, I got an > impression there is some code repetition in the agent implementations. Almost certainly. One of the things driving the 5.x handler mechanism was the recognition that there was a lot of processing that is common across many MIB implementations. By introducing a chain of handlers to process a request, this has allowed the use of shared "helper" routines, which can consolidate this common processing into one block of code. > Also there seems to be quite a lot of static buffers (defined inside > functions) that are mostly sitting unused eating runtime memory? Quite likely. Static buffers typically offer a simpler, safer and more time-efficient approach than dynamic allocation for internal scratch working space. In most environments, the size costs are not particularly significant. It would be fair to say that the Net-SNMP agent is not really geared towards embedded systems, where code size may well be of greater concern. > Is it quick to outline what kind of additional features are we talking here? Not really. The UCD line has been essentially dead for something like 6-7 years. (The 4.2.7 release was simply a housekeeping release, to pick up a number of bug fixes - 4.2.6 was the last "active" release on this line). I can't really recall everything (or even the more important stuff) we've done since 2002! > Thinking of replacing v4 net-snmp, is it possible to disable most of this > new weigth (with appropriate ./configure invocation)? Not easily. You'd probably need to use configure to select the MIB modules you want to include, then manually tweak the Makefiles (such as 'helpers/Makefile') to omit anything that these don't actually use. But this isn't something that the current build environment handles automatically. Dave -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
RE: ucd-snmp vs. net-snmp (foot print)
> Date: Sat, 1 Aug 2009 15:43:34 +0100 > Subject: Re: ucd-snmp vs. net-snmp (foot print) > From: d.t.shi...@liverpool.ac.uk > To: jpu...@live.com > CC: net-snmp-users@lists.sourceforge.net > > 2009/7/30 Juliana Purjo : > > Concerning MIB modules... Having looked at 4.2.x code base, I got an > > impression there is some code repetition in the agent implementations. > > Almost certainly. > One of the things driving the 5.x handler mechanism was the recognition > that there was a lot of processing that is common across many MIB > implementations. By introducing a chain of handlers to process a request, > this has allowed the use of shared "helper" routines, which can consolidate > this common processing into one block of code. > Sounds good. I really ought to have a better look into the v5.x net-snmp... > > Also there seems to be quite a lot of static buffers (defined inside > > functions) that are mostly sitting unused eating runtime memory? > > Quite likely. > Static buffers typically offer a simpler, safer and more time-efficient > approach than dynamic allocation for internal scratch working space. > In most environments, the size costs are not particularly significant. Static buffers are simple, efficient and very fine. But I fail to see the need to have separate buffers for each objects. I do not know if it makes any sense, but maybe we could have a function/API to get a pointer to a static buffer of desired size, which the agent code would utilize instead of using private static buffers. According to my understanding, as net-snmp is not multithreaded, (without knowing the architecture very well) I think it might work to reset the static buffer "unused area" pointer in the beginning of each request, and allow requesting multiple memory areas within one SNMP request; the size of the whole buffer could be a constant. [We could easily insert checks for overflow (sum of allocation sizes exceeding the total buffer size) and (development time) debug statements to map how big buffer we at most need in case somebody wants to tweak the buffer size; also buffer overflows could be checked easily by filling the buffer with magic numbers in the beginning of a request handling.] > It would be fair to say that the Net-SNMP agent is not really geared > towards embedded systems, where code size may well be of greater concern. Is there interest towards such tweaks? > > Is it quick to outline what kind of additional features are we talking here? > > Not really. > The UCD line has been essentially dead for something like 6-7 years. > (The 4.2.7 release was simply a housekeeping release, to pick up > a number of bug fixes - 4.2.6 was the last "active" release on this line). > > I can't really recall everything (or even the more important stuff) we've done > since 2002! > I understand. > > Thinking of replacing v4 net-snmp, is it possible to disable most of this > > new weigth (with appropriate ./configure invocation)? > > Not easily. > You'd probably need to use configure to select the MIB modules you want > to include, then manually tweak the Makefiles (such as 'helpers/Makefile') > to omit anything that these don't actually use. > > But this isn't something that the current build environment handles > automatically. > I understand. And I think in a perfect world, much of this tweaking work could be done by an intelligent linker...? > > Dave _ More than messages–check out the rest of the Windows Live™. http://www.microsoft.com/windows/windowslive/-- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
Re: ucd-snmp vs. net-snmp (foot print)
2009/8/3 Juliana Purjo : > But I fail to see the need > to have separate buffers for each objects. Remember that a single SNMP request may contain several different MIB objects - possibly from different areas of the overall MIB tree. If the agent attempted to re-use a single shared buffer for implementing multiple objects, then there is a very real danger that the processing for one object could over-write the processing for another. And introducing multiple such buffers would then need some mechanism for indicating which buffers were available. Extra complexity, additional overheads, and more to go wrong. Given that in most environments, run-time size is not a significant concern, using a separate buffer for each object feels to be a simpler, and more (time-)efficient approach. > According to my understanding, as net-snmp is not multithreaded, > (without knowing the architecture very well) I think it might work to reset > the static buffer "unused area" pointer in the beginning of each request, No - that wouldn't work. Although the agent itself is not multi-threaded, it is quite feasible for the processing of multiple requests to be interleaved - particularly if they involve one or more AgentX subagents, or similar "delegated" or "delayed" MIB modules. >> It would be fair to say that the Net-SNMP agent is not really geared >> towards embedded systems, where code size may well be of greater concern. > > Is there interest towards such tweaks? I have no idea. Given that we do get occasional queries about support for embedded systems, then possibly, yes. But given that nobody can be bothered to feed back any of their experiences in using this software on such systems, then perhaps no. Remember that this is an Open Source project. Nobody is paid to work on the code, or gets any recompense for their involvement with it. The bottom line is - if you need something doing, how badly? Do you need it enough to do the work yourself? >> You'd probably need to use configure to select the MIB modules you want >> to include, then manually tweak the Makefiles (such as 'helpers/Makefile') >> to omit anything that these don't actually use. > > And I think in a perfect world, much of this tweaking work > could be done by an intelligent linker...? The question is more - what object files would need to be linked. It's conceivable that an intellegent linker might be able to omit unnecessary modules, but I'm not entirely convinced. Particularly since the helper code ends up in a different library from the MIB implementation code. (And dynamic loading of modules muddies the water further, since such a module might require one of the hitherto unused helpers). I'm inclined to think that such selection is best done as part of constructing the build environment - perhaps by seeding the MIB module header files with suitable "config_require" directives, and stripping down the default list to the bare minimum. Dave -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
RE: ucd-snmp vs. net-snmp (foot print)
> Date: Mon, 3 Aug 2009 08:34:28 +0100 > Subject: Re: ucd-snmp vs. net-snmp (foot print) > From: d.t.shi...@liverpool.ac.uk > To: jpu...@live.com > CC: net-snmp-users@lists.sourceforge.net > > 2009/8/3 Juliana Purjo : > > But I fail to see the need > > to have separate buffers for each objects. > > Remember that a single SNMP request may contain several different > MIB objects - possibly from different areas of the overall MIB tree. > > If the agent attempted to re-use a single shared buffer for implementing > multiple objects, then there is a very real danger that the processing for > one object could over-write the processing for another. And introducing > multiple such buffers would then need some mechanism for indicating > which buffers were available. Extra complexity, additional overheads, > and more to go wrong. > > Given that in most environments, run-time size is not a significant concern, > using a separate buffer for each object feels to be a simpler, and more > (time-)efficient approach. Also in embedded Linux, the executable size is more important than runtime memory size. The flash sizes are still measured in single digit megabytes... And the linked application size seems to be the same whether you have 100kb of static buffer or 1kb of static buffer, probably the number of static buffers also does not make a difference. > > > According to my understanding, as net-snmp is not multithreaded, > > (without knowing the architecture very well) I think it might work to reset > > the static buffer "unused area" pointer in the beginning of each request, > > No - that wouldn't work. > Although the agent itself is not multi-threaded, it is quite feasible for the > processing of multiple requests to be interleaved - particularly if they > involve > one or more AgentX subagents, or similar "delegated" or "delayed" MIB > modules. > Okay, more complicated than I thought... I agree the implementation effort probably is not worth the trouble. > >> It would be fair to say that the Net-SNMP agent is not really geared > >> towards embedded systems, where code size may well be of greater concern. > > > > Is there interest towards such tweaks? My meaning was more to ask whether the project is open for such help. > I have no idea. Given that we do get occasional queries about support > for embedded systems, then possibly, yes. > But given that nobody can be bothered to feed back any of their experiences > in using this software on such systems, then perhaps no. Out of personal curiosity I am interested in doing some experimentation... If this curiosity ever channels into action, I will let you know (just doing simple test compilation with bare minimum configuration would give some hint about the executable size difference, even doing this with the x86/x86_64 tool chain). > Remember that this is an Open Source project. Nobody is paid to work > on the code, or gets any recompense for their involvement with it. > The bottom line is - if you need something doing, how badly? > Do you need it enough to do the work yourself? I remember very well this is an Open Source project, and I am very thankful for the work you and other people are doing to help this project, whether by answering uninformed questions or improving net-snmp... > >> You'd probably need to use configure to select the MIB modules you want > >> to include, then manually tweak the Makefiles (such as 'helpers/Makefile') > >> to omit anything that these don't actually use. > > > > And I think in a perfect world, much of this tweaking work > > could be done by an intelligent linker...? > > The question is more - what object files would need to be linked. > It's conceivable that an intellegent linker might be able to omit > unnecessary modules, but I'm not entirely convinced. > Particularly since the helper code ends up in a different library > from the MIB implementation code. (And dynamic loading of > modules muddies the water further, since such a module might > require one of the hitherto unused helpers). Yes, the linker could only optimize when statically linking the net-snmp... And this too is just hypothetical, meaningless babbling (unless done in a proper tool chain forum). > I'm inclined to think that such selection is best done as part of constructing > the build environment - perhaps by seeding the MIB module header files > with suitable "config_require" directives, and stripping down the default > list to the bare minimum. Okay. > > > Dave _ Share your memories online with anyone
Re: ucd-snmp vs. net-snmp (foot print)
2009/8/3 Juliana Purjo : >> From: d.t.shi...@liverpool.ac.uk >> Given that in most environments, run-time size is not a significant concern, >> using a separate buffer for each object feels to be a simpler, and more >> (time-)efficient approach. > > Also in embedded Linux, the executable size is more important than runtime > memory size. The flash sizes are still measured in single digit megabytes... > And the linked application size seems to be the same whether you have 100kb > of static buffer or 1kb of static buffer, probably the number of static > buffers also does not make a difference. Regarding runtime size: many embedded systems use SDRAM memory. The smallest SDRAM modules being sold today that I know of are 64 MB in size. And I know of embedded systems with 32 MB RAM in which UCD-SNMP has been deployed successfully. Regarding static buffers: the space occupied in an executable by a static buffer that has been stored by the compiler in a BSS section is independent of the buffer size. Regarding executable size: a filesystem like cramfs, cromfs or squashfs can be a great help to reduce the amount of non-volatile storage occupied by an executable. Bart. -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
Re: ucd-snmp vs. net-snmp (foot print)
On Fri, Jul 24, 2009 at 10:58 PM, Dave Shield wrote: > Unfortunately, we can't really provide you with any significant implementation > experience of running the Net-SNMP agent on embedded systems. I keep > putting out a plea for those who have done this to report back, but nobody > ever seems to bother. One aspect of Net-SNMP that should be documented with regard to embedded systems is whether or not Net-SNMP assumes that the system it is running on has a hostname. On some embedded systems the hostname (the string returned by gethostname()) has been left empty. Additionally, on many embedded systems DNS has not been set up (empty /etc/resolv.conf). People deploying Net-SNMP on such an embedded system need to know which components of Net-SNMP rely on access to a DNS server, if there are any such components. Bart. -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Net-snmp-users mailing list Net-snmp-users@lists.sourceforge.net Please see the following page to unsubscribe or change other options: https://lists.sourceforge.net/lists/listinfo/net-snmp-users
