Re: Opinion on some open bugs
On Mon, 11 Jul 2005, Serassio Guido wrote: Just installed now, and problem is still here ... :-( I like to know what happens on Solaris 10, where ipfilter is bundled into. Probably the same thing. the fault is imho at Squid, not Solaris/IP-Filter. The trouble is/was caused by the following: #define free + etc in squid.h to block misuse of standard malloc routines where the Squid versions should be used. This pollutes the C/C++ token namespace crashing any structures or classes having members of the same names. After fixing as proposed, ipfilter builds and installs fine, but Squid 3 broke on this patch that allow it to build without the headers fixing: http://www.squid-cache.org/cgi-bin/cvsweb.cgi/squid3/include/config.h.diff?r1=1.10&r2=1.11&f=h This is about something else than the original IP-Filter build problem.. Backing out this, Squid 3 build fine with both ARP and ipfilter support. Interesting. seems things have changed considerably since I looked at the original problem report some years ago. Regards Henrik
Re: Opinion on some open bugs
On Mon, 11 Jul 2005, Serassio Guido wrote: But now I'm a little confused: it seems that gcc on Solaris (x86 only ?) is very unstable, so I like to understand what Solaris x86 reference platform should we use for Squid development. Squid-2.5 is verified (build and trivial runtime tests only) on Solaris 9 Sparc using both GCC-3.2.3 and SunPro CC something. Have never tried Solaris x86. GCC is supposed to be quite stable on Solaris. At least on Sparc, and I have a hard time beleiving x86 would be much worse as very little is OS-dependent. But there may obviously be binutil related (ld, ar etc) issues. Probably better to use Solaris bundled binutils. But I am not a Solaris user so I don't know. Regards Henrik
RE: About SQUID and SNMP TRAPS , and snmp in general
On Mon, 11 Jul 2005, Lombao, Cesar wrote: Please, first, if I pollute too much the list let me know Not at all. The list needs people participating. Second, at this very earlier stage, is my opinion to use more the CMU library, it looks like nice, and most if the things are already done. For instance, if my theory is correct, there is an ASN parser build, that builds the MIB on execution time based in a MIB file (the mib.txt). snmplib in Squid is a heavily modified CMU SNMP 1.8. So, if you change something in the mib, you have not to modify much code. At the other side, the snmp_core creates the MIB in memory in a hard-coded way. I don't see any problem in terms of special select, or interrupt calls that can impact in squid. If you intend to use the SNMP agent or client from CMU SNMP you will run into some problems to fit this into the I/O loops of Squid. However, the CMU library copied into the /snmplib folder, seems not to create the sockets, it seems that is left to the implementation, and in that point the snmp_core has some functions to do that, I guess taking into account the problems of the select, and its blocking issues. The code dealing with networking I/O has been replaced. The library originally had all of that, but it wasn't useable from within Squid. So, this is my plan, and let's see if I'm able to get results (that remains to be seen): Take the current files in snmplib (The cmu library), create an snmp agent on my own, using the network functions in the snmp_core To test the TRAPS, I'll create some false TRAP definitions in the mib, and test it. If I'm success, then, later we can review if is posible to integrate into SQUID and how. Probably easier to stay within Squid from start. You can do a lot of testing from the debugger using suitable print statements simulating various events. About what TRAPS to add I think this point is independent of the implementation itself We have three"type" of traps: 1) Those who inform about a "database" change (some oid modified) 2) Those who report about an event, for instance, "Corruption in the DNS..." "Internal error..." 3) The "standard" ColStart, WarnStart, etc, etc. For type 1, I think many other things come first Indeed. For type 2, here is the point where I'm strngly interested. To have a proper monitoring, it would be nice when some critical issue happens a trap be sent. Which ones? My suggestions in no special order: - Fatal errors - Startup/shutdown events - peers detected dead/alive. - Log write failures - Cache oversized - Close to running out of filedescriptors - Threshold on select/poll loop latency, to warn when there is overload conditions. and probably a few more. Regards Henrik
Re: Opinion on some open bugs
Hi Henrik, At 23.52 10/07/2005, Henrik Nordstrom wrote: On Sun, 10 Jul 2005, Serassio Guido wrote: - #199: for me could be closed: current version of Squid 2.5 and 3.0 build fine on Solaris x86 with ARP acls enabled. Do you have IP-Filter installed? --enable-ipf-transparent is required for the problem to arise. No. Just installed now, and problem is still here ... :-( I like to know what happens on Solaris 10, where ipfilter is bundled into. The correct fix is in theory not very hard, but requires a bit of juggling around with how we manage include files to never include system headers after squid.h. However, there is some complications due to the FD_SETSIZE overriding which is also done in squid.h and needs to be done before the include of system headers.. Because of this I didn't want to attemt fixing this in 2.5. Things on Solaris seems to be more complex: I had this problem building ipfilter on my Solaris 9 x86: http://www.phildev.net/ipf/IPFsolaris.html#solaris19 After fixing as proposed, ipfilter builds and installs fine, but Squid 3 broke on this patch that allow it to build without the headers fixing: http://www.squid-cache.org/cgi-bin/cvsweb.cgi/squid3/include/config.h.diff?r1=1.10&r2=1.11&f=h Backing out this, Squid 3 build fine with both ARP and ipfilter support. But now I'm a little confused: it seems that gcc on Solaris (x86 only ?) is very unstable, so I like to understand what Solaris x86 reference platform should we use for Squid development. Regards Guido - Guido Serassio Acme Consulting S.r.l. - Microsoft Certified Partner Via Lucia Savarino, 1 10098 - Rivoli (TO) - ITALY Tel. : +39.011.9530135 Fax. : +39.011.9781115 Email: [EMAIL PROTECTED] WWW: http://www.acmeconsulting.it/
RE: About SQUID and SNMP TRAPS , and snmp in general
Please, first, if I pollute too much the list let me know, there are a lot of things I don't understand, and perhaps simply there is a reason for that. Second, at this very earlier stage, is my opinion to use more the CMU library, it looks like nice, and most if the things are already done. For instance, if my theory is correct, there is an ASN parser build, that builds the MIB on execution time based in a MIB file (the mib.txt). So, if you change something in the mib, you have not to modify much code. At the other side, the snmp_core creates the MIB in memory in a hard-coded way. I don't see any problem in terms of special select, or interrupt calls that can impact in squid. However, the CMU library copied into the /snmplib folder, seems not to create the sockets, it seems that is left to the implementation, and in that point the snmp_core has some functions to do that, I guess taking into account the problems of the select, and its blocking issues. So, this is my plan, and let's see if I'm able to get results (that remains to be seen): Take the current files in snmplib (The cmu library), create an snmp agent on my own, using the network functions in the snmp_core To test the TRAPS, I'll create some false TRAP definitions in the mib, and test it. If I'm success, then, later we can review if is posible to integrate into SQUID and how. -- About what TRAPS to add I think this point is independent of the implementation itself We have three"type" of traps: 1) Those who inform about a "database" change (some oid modified) 2) Those who report about an event, for instance, "Corruption in the DNS..." "Internal error..." 3) The "standard" ColStart, WarnStart, etc, etc. For type 1, I think many other things come first For type 2, here is the point where I'm strngly interested. To have a proper monitoring, it would be nice when some critical issue happens a trap be sent. Which ones ? Those who develop the core of SQUID should let know, I mean, those events who are wrriten down in the log. For type 3, I would not take care at this moment. - -Original Message- From: Henrik Nordstrom [mailto:[EMAIL PROTECTED] Sent: lunes, 11 de julio de 2005 16:38 To: Lombao, Cesar Cc: Henrik Nordstrom; Squid Developers Subject: RE: About SQUID and SNMP TRAPS , and snmp in general On Mon, 11 Jul 2005, Lombao, Cesar wrote: > I was taking a look on the source code, and, besides the functions > provided by the UCM snmp library, I don't see any other code to send > traps. There was quite a bit of code deleted from the library some years back.. have a quite strong memory of a fair bit of this being related to traps and set PDUs. > In fact, I'm still trying to understand how this library and the > snmp_core.cc and snmp_agent.cc modules work together. the library is used for encoding/decoding. snmp_agent.cc defines the general MIB extractors and snmp_core.c is the agent implementation (and also defines the MIB for some reason.. but this part is better moved to snmp_agent.cc) The original agent in the CMU SNMP library is not used for mainly the same reason the agent in net-snmp doesn't fit well within Squid I suppose. > To send a trap is something, in theory, trivial. Open a dgram socket, > and send the info to the trapsink. Yes, and is why I think currently is easier to just resurrect/reimplement the SNMP PDU processing components for this rather than to look into how to fit a more modern SNMP agent into Squid. > The point is not that, the point is all the details surronding that: > MIB: > Needs to be defined in the MIB the traps sent. > > TRAPS: > Needs to be defined wich traps be sent. (When special errors happen? > when MIB updated?) Yes. The two go pretty much together. When to send traps and what to include. > CONFIGURATION: > If SQUID uses its own snmp agent, then is needed to add a confguation > line to set the trapsink destianation for the traps sent. Yes. > And of course, all the time I was talking about SNMPv1 only ;-) Only desire to support SNMPv1/2c at this time. For SNMPv3 we better look at using a modern agent. Another point to consider is if ther is events requiring an SNMP Inform message for somewhat reliable delivery, or if best effort traps is sufficient. Regards Henrik
Introducing myself
Hi all: First of all, I'd like to learn Squid's architecture and after that work on squid performance issues, such as replacement policy, disk cache I/O and OS tunning. I've already downloaded the tech paper about LFUDA and GDSF. My first questions should be simple questions relied over squid coding and how things works. About easy things. For example, where can i get a paper/information about LRU replacement policy? -- []'s Lucas Brasilino [EMAIL PROTECTED] Procuradoria da República no Estado de Roraima +55-95-6239642
RE: About SQUID and SNMP TRAPS , and snmp in general
On Mon, 11 Jul 2005, Lombao, Cesar wrote: I was taking a look on the source code, and, besides the functions provided by the UCM snmp library, I don't see any other code to send traps. There was quite a bit of code deleted from the library some years back.. have a quite strong memory of a fair bit of this being related to traps and set PDUs. In fact, I'm still trying to understand how this library and the snmp_core.cc and snmp_agent.cc modules work together. the library is used for encoding/decoding. snmp_agent.cc defines the general MIB extractors and snmp_core.c is the agent implementation (and also defines the MIB for some reason.. but this part is better moved to snmp_agent.cc) The original agent in the CMU SNMP library is not used for mainly the same reason the agent in net-snmp doesn't fit well within Squid I suppose. To send a trap is something, in theory, trivial. Open a dgram socket, and send the info to the trapsink. Yes, and is why I think currently is easier to just resurrect/reimplement the SNMP PDU processing components for this rather than to look into how to fit a more modern SNMP agent into Squid. The point is not that, the point is all the details surronding that: MIB: Needs to be defined in the MIB the traps sent. TRAPS: Needs to be defined wich traps be sent. (When special errors happen? when MIB updated?) Yes. The two go pretty much together. When to send traps and what to include. CONFIGURATION: If SQUID uses its own snmp agent, then is needed to add a confguation line to set the trapsink destianation for the traps sent. Yes. And of course, all the time I was talking about SNMPv1 only ;-) Only desire to support SNMPv1/2c at this time. For SNMPv3 we better look at using a modern agent. Another point to consider is if ther is events requiring an SNMP Inform message for somewhat reliable delivery, or if best effort traps is sufficient. Regards Henrik
RE: About SQUID and SNMP TRAPS , and snmp in general
I was taking a look on the source code, and, besides the functions provided by the UCM snmp library, I don't see any other code to send traps. In fact, I'm still trying to understand how this library and the snmp_core.cc and snmp_agent.cc modules work together. To send a trap is something, in theory, trivial. Open a dgram socket, and send the info to the trapsink. The point is not that, the point is all the details surronding that: MIB: Needs to be defined in the MIB the traps sent. TRAPS: Needs to be defined wich traps be sent. (When special errors happen? when MIB updated? ) CONFIGURATION: If SQUID uses its own snmp agent, then is needed to add a confguation line to set the trapsink destianation for the traps sent. And of course, all the time I was talking about SNMPv1 only ;-) -Original Message- From: Henrik Nordstrom [mailto:[EMAIL PROTECTED] Sent: domingo, 10 de julio de 2005 6:35 To: Henrik Nordstrom Cc: Lombao, Cesar; squid-dev@squid-cache.org Subject: Re: About SQUID and SNMP TRAPS , and snmp in general On Sat, 9 Jul 2005, Henrik Nordstrom wrote: > There is two approaches, either resurrect the now deleted trap snmp > code in the old simplminded snmp library used (and distributed) by > Squid, or do it more proper and replace the SNMP agent glue in Squid > with a new one using the more up to date net-snmp library API > http://net-snmp.sourceforge.net/>, > making Squid a SMUX or AgentX sub-agent to an existing system SNMP > agent supporting either method. A quick inspection of the net-snmp agent API indicates it is not immediately suitable to be called from inside Squid. This due to a design heavily centered around select() semantics and fd_set processing. It may be possible to build glue around, but I have doubts in how the library uses the fd_set:s. Any library having static/local fd_set instances is not useable from Squid (filedescriptor limitation problem). So for now I think it would be better to simply resurrect/reimplement the TRAP PDU sending code. Since we are only looking at sending traps there is not much danger or maintenance headache. Regards Henrik
