RE: [Syslog] timeline
Just to clarify, Kiwi Syslog V8 currently sends CRLF as the TCP delimiter, but will accept LF, CR, CRLF, LFCR and NULL as valid delimiters on the incoming stream. We will be changing our sending delimiter to LF in the near future to make it more compatible with syslog-ng etc. Cheers Andrew Rainer, Stunnel is a secure wrapper for TCP stream. Actually delimiting Syslog is done in the TCP part rather than TLS (or stunnel) part in Syslog-ng with stunnel. One can use stunnel to secure any Syslog TCP transport, such as rsyslog and kiwisyslog, and kiwisyslog does use CRLF for delimiting (http://www.kiwisyslog.com/whats_new_syslog.htm). Stunnel implementation is different from Syslog TLS transport, and I don' t think it is the exact implementation of Syslog TLS transport. I have not been aware of a Syslog implementation in TLS-transport style till now. So, most of the implementation may be modified, slightly or heavily, to existing code to get it comply to the specification. Miao > -Original Message- > From: Rainer Gerhards [mailto:[EMAIL PROTECTED] > Sent: Tuesday, August 15, 2006 12:41 PM > To: Miao Fuyou > Cc: [EMAIL PROTECTED] > Subject: RE: [Syslog] timeline > > Miao, > > I am actually concerned about backward compatibility with > existing code > *without* the need to upgrade any of that code. As you know, > deployed software tends to stick. > > If we use just LF, existing, deployed technology (e.g. syslog-ng with > stunnel) would be able to understand a message sent from a "new style" > syslogd. Having the octet count in front of the message > removes that ability, as the old syslogd will no longer see > the at the start of the message. > > I agree that it is trivial to modify code to take care for > the octet counter. But this is not my concern. My concern is > that I would like to achive as good as possible compatibility > with existing deployed (aka > "unmodified") technology. I should have been more specific on that. > Sorry for the omission... > > I am also unaware of any implementation that mandates CR LF > over just LF. Could you let me know which ones are these? > > Rainer > > > -Original Message- > > From: Miao Fuyou [mailto:[EMAIL PROTECTED] > > Sent: Monday, August 14, 2006 7:07 PM > > To: Rainer Gerhards > > Cc: [EMAIL PROTECTED] > > Subject: RE: [Syslog] timeline > > > > > > Hi, Rainer, > > > > A new implementation could rely on byte-counting only and > then delete > > LF from the frame(appplication knows exactly where the LF > is), it may > > not force us to use escapes. For LF, I think it is difficult to get > > 100% compatibility for a legacy implementation to comply > TLS-transport > > without any change to the code. At least, some > imlementation may need > > to change CR LF to LF because some implementations use CR LF rather > > than LF. So, it may be ok to add several LOC to delete FRAME-LEN SP > > from the frame. > > > > I still prefer byte-counting only to byte-counting+LF even > if it is a > > feasible tradeoff. > > > > Miao > > > > > -Original Message- > > > From: Rainer Gerhards [mailto:[EMAIL PROTECTED] > > > Sent: Monday, August 14, 2006 10:18 PM > > > To: Miao Fuyou > > > Subject: RE: [Syslog] timeline > > > > > > We should not go byte-counting + LF. This is the worst choice: it > > > > > > A) breaks compatibility > > > B) Forces us to use escapes > > > > > > So we get the bad of both worlds, without any benefits. > > > > > > Rainer > > > > > > > -Original Message- > > > > From: Miao Fuyou [mailto:[EMAIL PROTECTED] > > > > Sent: Monday, August 14, 2006 12:58 AM > > > > To: 'Anton Okmianski (aokmians)'; 'David Harrington'; > > > [EMAIL PROTECTED] > > > > Subject: RE: [Syslog] timeline > > > > > > > > > > > > My vote: byte-counting only > byte-counting + LF > LF > > > > > > > > > > > > > ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] delineated datagrams
Rainer, I'm in favour of using the LF delimiter as a starting point. This way we can get something that works with Cisco PIX, Netscreen, Monitorware, Kiwi, Syslog-ng etc. Then it becomes an easy task to just wrap the session with TLS. How do you suggest we escape the LF? Regards Andrew Kiwi Enterprises -Original Message- From: Rainer Gerhards [mailto:[EMAIL PROTECTED] Sent: Friday, 21 July 2006 2:02 a.m. To: Rainer Gerhards; Tom Petch; [EMAIL PROTECTED] Subject: RE: [Syslog] delineated datagrams WG, now with the consensus on moving forward with -transport-tls, I would like to re-state my thoughts on delineated datagrams: I think a compromise to get -transport-tls going is that we might actually define LF to be the end of record marker and require the message inside the "frame" to escape LF. That would require two characters to be escaped (of for the LF and one for the escape character itself). Such a compromise would also be consistent with what is currently done in practice. And, indeed, we could avoid the byte-counting. That would fully bring us in sync to what is done today (syslog-ng, cisco Pix, rsyslog, Kiwi Syslog, MonitorWare to name a few). I still consider this to be a non-perfect framing, but I think it would be acceptable. In the medium term, we should look into a more sophisticated framing, maybe borrowed from NETCONF. But that should come after we have delivered something. A byte-count could be prefixed to each record, but that would break compatibility with existing implementations (this is not absolutely necessary, but I consider it a very-nice-to-have, especially if the price for it is low). Comments appreciated, Rainer > -Original Message- > From: Rainer Gerhards [mailto:[EMAIL PROTECTED] > Sent: Thursday, July 06, 2006 4:17 PM > To: Tom Petch; [EMAIL PROTECTED] > Subject: RE: [Syslog] delineated datagrams > > Tom, > > I think your and Anton's commetn below so far is the most important > comment on -transport-tls technical issues (assuming that the > certificate issue can relatively easy be fixed by specifying a cipher > suite). > > IMHO the comment applies to any shim layer for stream protocols, not > just TLS. I think a compromise to get something like -transport-tls > going is that we might actually define LF to be the end of > record marker > and require the message inside the "frame" to escape LF. That would > require two characters to be escaped (of for the LF and one for the > escape character itself). Such a compromise would also be consistent > with what is currently done in practice. And, indeed, we > could avoid the > byte-counting. That would fully bring us in sync to what is done today > (syslog-ng, cisco Pix, rsyslog, Kiwi Syslog, MonitorWare to > name a few). > I still consider this to be a non-perfect framing, but I > think it would > be acceptable. In the medium term, we should look into a more > sophisticated framing, maybe borrowed from NETCONF. But that > should come > after we have delivered something. If that might be a compromise, I > could quickly draft an I-D that *documents* the way TCP based stream > transport is used today. -transport-tls would then just need to add > description of TLS handshaking. Or the I-D could be informational > describing what can be found in practice. I do not know if that would > have any effect on the patent office's decision (but I can claim > publically-available previous work, around Jan 2003 - see > http://adiscon.org/specs/selp-2003-01-15.txt). > > For the legal folks: I have running implementations and > public documents > describing the method outlined above. It is fraudulent if somebody > claims he has invented the method I have described here, at > least if he > hasn't invented it roughly 10 years or so ago. > > Rainer > > > -Original Message- > > From: Tom Petch [mailto:[EMAIL PROTECTED] > > Sent: Thursday, June 22, 2006 11:47 AM > > To: Anton Okmianski (aokmians); [EMAIL PROTECTED] > > Subject: Re: [Syslog] delineated datagrams > > > > > > - Original Message - > > From: "Anton Okmianski (aokmians)" <[EMAIL PROTECTED]> > > To: "Tom Petch" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> > > Sent: Tuesday, June 20, 2006 8:18 PM > > Subject: RE: [Syslog] delineated datagrams > > > > > > Tom: > > > > I think these are valid concerns. They span different layers: > > > > 1. If we only care about network-layer reliability (as in > > byte insert/remove > > examples): client/server can be recommended to reset > > connection every so often. > > Decent corruption detection is already part of TCP/IP and > > layer 2 protocols. > > > > 2. If we care about app-layer reliability (as in > > encode/decode error examples): > > app-level ACK. As in RFC 3195, for example. This certainly > > expands the scope > > quite a bit beyond just secure transport. > > > > Anton > > > > My concern was in between, the shim between TCP and syslog > > that is TLS. > > > > With UDP t
[Syslog] Syslog TLS and the Huawei IPR claim
Chris, After reading the IPR document, my vote is for A. A. The WG SHOULD proceed with draft-ietf-syslog-transport-tls as a Working Group document. Regards Andrew Kiwi Enterprises Chris Lonvick wrote: > Hi Folks, > > Please continue to send in your opinion on this. I'll determine > consensus next Thursday and outline our steps to go forward. > > Thanks, > Chris > > On Wed, 5 Jul 2006, Chris Lonvick wrote: > >> Hi Folks, >> >> Everyone has now had a week to think about the IETF process on IPR >> claims. The first decision that we need to make is about the terms of >> the claim. >> I'd like to hear what people think about the terms that Huawei has >> presented. >> https://datatracker.ietf.org/public/ipr_detail_show.cgi?&ipr_id=724 >> >> Please keep in mind that we can (and should) proceed with >> syslog-transport-tls if the terms appear reasonable and you (as >> implementors) are willing to accept them. Let's keep this discussion >> focused. >> - We do not need a discussion of the terms. >> - We do not need any legal opinions. >> - We do not need a discussion of technical alternatives on this thread. >> >>> From that, I'd like to hear either: >> >> A) The WG SHOULD proceed with draft-ietf-syslog-transport-tls as a >> Working Group document. >> >> or >> >> B) The WG SHOULD NOT proceed with draft-ietf-syslog-transport-tls as >> a Working Group document. >> >> I'll leave this open until the 19th (so people going to the IETF can >> catch their collective breaths and give a good opinion. >> >> If the consensus appears to be "A" then we can get straight back to >> work. >> >> If the consensus appears to be "B" then I will very briefly ask if >> there are changes to the terms that would make them acceptable. I'll >> only ask that if the consensus appears to be "B" so don't insert your >> opinions on that at this time. I'm (just barely) willing to ask that >> (once) of the Huawei lawyers but I feel that negotiating terms is not >> going to move us forward; it's likely to be a rat-hole discussion and >> I won't let us go down there. >> >> If the consensus remains "B" then we will likely move away from >> syslog-transport-tls. Where we move to will be a different >> discussion so please don't insert your opinion about that on this >> discussion thread. David has opened that discussion on a separate >> thread so you may discuss it on the list, but I'm not focused on that >> at this time. >> >> Thanks, >> Chris >> >> ___ >> Syslog mailing list >> Syslog@lists.ietf.org >> https://www1.ietf.org/mailman/listinfo/syslog >> >> > > ___ > Syslog mailing list > Syslog@lists.ietf.org > https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] TIMESTAMP and leap seconds
Hi Rainer, Happy new year! Your idea of ignoring the leap seconds sounds very sensible to me. Cheers Andrew -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Rainer Gerhards Sent: Monday, 2 January 2006 11:42 p.m. To: [EMAIL PROTECTED] Subject: [Syslog] TIMESTAMP and leap seconds Hi all, first of all, I would like to whish a happy new year to each of you! I am now back in the office and at final edits to syslog-protocol. I discovered one more thing: The current draft supports leap seconds. There already is a lot of discussion whether or not leap seconds should be introduced in the future. However, the way leap seconds are handled will be largely invisible to a syslog sender (except where it is sitting on a time-tracking device, which is highly unlikely). On the other hand, leap second processing can be pretty complicated at the receiver side. I expect that most implementations will not abide strict handling in any case. As such, I suggest that leap second support be removed from the TIMESTAMP. Similarily, a sender with unknown time should then not use the special TIMESTAMP but "-", which also keeps it consistent with the rest of the header NIL values. If nobody objects, I'll change this during the edit. Rainer ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting
Either solution would work, I have no preference either way. :) Cheers Andrew -Original Message- From: Anton Okmianski (aokmians) [mailto:[EMAIL PROTECTED] Sent: Thursday, 1 December 2005 7:38 a.m. To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: RE: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting I think for TCP mapping a transport header with message size would be more appropriate framing than termination character. Thanks, Anton. > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Ross > Sent: Wednesday, November 30, 2005 7:19 AM > To: [EMAIL PROTECTED] > Subject: RE: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting > > > Rainer, > > That sounds good to me at this stage, and it keeps the door > open. I would prefer to see all binary data encoded in some > "safe" format like base64. It just makes logging the data to > file much easier. For instance, if the binary data contained > a LF character, when it was logged to disk, it would end up > as two separate messages when opened in notepad etc. > > Also, if we are to transport syslog over TCP at some stage, > we need to keep a delimiter character free from use in the > message. Again, a LF would be a good choice for this delimiter. > > Cheers > > Andrew > > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] > On Behalf Of Rainer Gerhards > Sent: Wednesday, 30 November 2005 9:26 p.m. > To: [EMAIL PROTECTED] > Subject: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting > > > Hi WG, > > I have received notes via private mail telling me there seem > to be some existing (and eventually soon upcoming) valid use > cases for binary data in syslog. I think there is no point in > arguing whether that's fortunate or not. It simply looks like > that's the way it is. I do not like the idea of breaking > existing use cases for syslog (because that will only lead to > implementors ignoring the spec and the story of syslog > inconsistencies continues...). As such, I think we need to > provide at least some minimal support for it (aka "not outlaw it"). > > At first, this implies that NUL octets may be present in the message. > > I propose that we write text that discourages the use of NUL, > but allows it if needed. That text should also allow, but > discourage, a receiver to modify messages containing NUL. > With that, we allow the use case, but do not make it a "show > stopper" for implementing compliant software. This would also > be pretty much in sync with what we currently find in > practice, so it is already expected behaviour. Finally, such > text would caution implementors that when NUL octets are > present, chancs are high that eventually present digitial > signatures will be broken. In my point of view, that's fair > and efficient. > > Chris proposal for #5 (character encoding) also provides an > elegant solution for binary data. We can use something like: > > [enc="binary"] > > or > > [enc="base-64"] > > I do NOT intend to specify this - I think it should be in the > scope of a separate document specifying the use of binary > data. Then would also be the right time to discuss all issues > that arise out of it. For now, I just would like to keep the > door open. > > Finally, I propose to extend Chris format so that the message > size can be conveyed. This has been brought up several times > and I think a clean solution is now obvious: > > [enc="utf-8" lang="en" size="MSG-size-in-octets"] > > MSG-size-in-octets would be the size of the MSG part (just > that!) in octets. Counting just the MSG part is sufficient, > as the rest of the message consists of fields properly > delimited. The size is probably most useful for binary data. > > Please comment. > > Rainer > > ___ > Syslog mailing list > Syslog@lists.ietf.org > https://www1.ietf.org/mailman/listinfo/syslog > > > ___ > Syslog mailing list > Syslog@lists.ietf.org > https://www1.ietf.org/mailman/listinfo/syslog > ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting
>We are still ok with always having UTF-8 in SD values, right? >We need this for foreign usernames. We have discussed this before. Yes, this would work for me. We need to ensure that the SD-IDs are always going to be encoded in a known format. UTF-8 is a good choice. Cheers Andrew ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] #2, max message size - Need to resolve this
My vote is for the way Rainer has worded it now. Specify the minimum msg size in syslog-protocol and define max message size in the transport documents. Cheers Andrew Hi Folks, We need to resolve this one. I've heard from Rainer and a very few others. I'd like to hear from more people on this. Choose one: __ The maximum message length needs to be defined in syslog-protocol. __ The maximum message length should be defined in the transport documents. __ I have a different idea Please VOTE NOW! Thanks, Chris ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting
Rainer, That sounds good to me at this stage, and it keeps the door open. I would prefer to see all binary data encoded in some "safe" format like base64. It just makes logging the data to file much easier. For instance, if the binary data contained a LF character, when it was logged to disk, it would end up as two separate messages when opened in notepad etc. Also, if we are to transport syslog over TCP at some stage, we need to keep a delimiter character free from use in the message. Again, a LF would be a good choice for this delimiter. Cheers Andrew -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Rainer Gerhards Sent: Wednesday, 30 November 2005 9:26 p.m. To: [EMAIL PROTECTED] Subject: [Syslog] #3 NUL octets, #4 binary data, #8 octet-counting Hi WG, I have received notes via private mail telling me there seem to be some existing (and eventually soon upcoming) valid use cases for binary data in syslog. I think there is no point in arguing whether that's fortunate or not. It simply looks like that's the way it is. I do not like the idea of breaking existing use cases for syslog (because that will only lead to implementors ignoring the spec and the story of syslog inconsistencies continues...). As such, I think we need to provide at least some minimal support for it (aka "not outlaw it"). At first, this implies that NUL octets may be present in the message. I propose that we write text that discourages the use of NUL, but allows it if needed. That text should also allow, but discourage, a receiver to modify messages containing NUL. With that, we allow the use case, but do not make it a "show stopper" for implementing compliant software. This would also be pretty much in sync with what we currently find in practice, so it is already expected behaviour. Finally, such text would caution implementors that when NUL octets are present, chancs are high that eventually present digitial signatures will be broken. In my point of view, that's fair and efficient. Chris proposal for #5 (character encoding) also provides an elegant solution for binary data. We can use something like: [enc="binary"] or [enc="base-64"] I do NOT intend to specify this - I think it should be in the scope of a separate document specifying the use of binary data. Then would also be the right time to discuss all issues that arise out of it. For now, I just would like to keep the door open. Finally, I propose to extend Chris format so that the message size can be conveyed. This has been brought up several times and I think a clean solution is now obvious: [enc="utf-8" lang="en" size="MSG-size-in-octets"] MSG-size-in-octets would be the size of the MSG part (just that!) in octets. Counting just the MSG part is sufficient, as the rest of the message consists of fields properly delimited. The size is probably most useful for binary data. Please comment. Rainer ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] #5 - character encoding (was: Consensus?)
>Hi Rainer, > >Why don't we look at it from the other direction? We could state that any >encoding is acceptable - for ease-of-use/migration with existing syslog >implementations. It is RECOMMENDED that UTF-8 be used. When it is >used, an SD-ID element will be REQUIRED. e.g. - [enc="utf-8" lang="en"] I like that idea too. So, if no SD-ID encoding element is specified, then we must assume US-ASCII and deal with it accordingly?? Cheers Andrew ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] RE: Message format (human readable)
Hi John, I guess the whole thing depends on how one would define "human readable". In my view, XML tags and content are still human readable. Binary data could also be counted as human readable if it was preceded with a tag and encoded in base64 etc. Something like [Bin_Data_Base64 = "kahskdjashd=="] would be fine. That way it would be obvious to someone reading the log what part is what. The idea is to avoid someone trying to send chunks of unencoded binary or hex data over syslog. Syslog was originally intended as a way of alerting operators to system malfunctions or things that required their attention. By tailing the Alert/Warning/Error logs on the console, it was easy to keep an eye on the condition of the systems. Some apps would also send verbose debug information to the logs. This was usually never seen by the operator and simply written to disk for analysis later. Sending XML medical or encoded binary data over syslog would fall into the debug category since no human would be reviewing the logs in real time on the screen. As long as you choose the level and facility () correctly, I feel it would still work fine. That is just my opinion. I'd be interested to hear what others on the list have to say. Cheers Andrew ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] New direction and proposed charter
Rainer, I agree that 3164 is only really valid with respect to the . When we implemented it in Kiwi Syslog we found no device actually used the 3164 format exactly. Sometimes the hostname was there, sometimes not. Having to write parsing code to work out if a hostname was actually a TAG or not was a huge processing overhead. We now disable 3164 parsing by default. Cheers Andrew ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
[Syslog] Null character
Rainer, FWIIW, I've seen Netscreen, NetGear and some LinkSys devices send a Null character at the end of each message. Not all versions of the firmware would include the Null at the end which leads me to believe it may be a bug in some releases of the firmware. When sending syslog over TCP, Netscreen firewalls will delimit each message with LF NULL. This probably won't have any impact on the 'CLR' since it is the very last character in the message, but I thought I would raise the issue with you as a real-life case of a Null character being used. Cheers Andrew ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] New direction and proposed charter
>The only tricky issue that remains is the NUL octet. The more I think >about it, the more I think the CLR to disallow it is less evil than to >make it stay... I agree that having the CLR for NUL octet exclusion is OK. Quick question, if someone is sending international data in UTF-8 format, can there ever be a valid UTF-8 sequence that includes a NUL octet value? If so, we need to allow NUL values in the MSG. Does anyone on the list know UTF-8 well enough to answer this? Cheers Andrew ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] RE: Message format
Hi Rainer, I concur on the message size issue. Let's leave it as it stands until we get to a TCP mapping. Can you mention in your document that the real life max UDP size was found to be 4K bytes? I think this is a valuable finding and would save a lot of hair pulling on the part of implementers. Cheers Andrew -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Rainer Gerhards Sent: Wednesday, 23 November 2005 10:00 p.m. To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: RE: [Syslog] RE: Message format Andrew, on the size: Though I have some concerns, I can agree with your point of view. In fact, one of the syslog-protocol revisions had a mechanism called multi-part messages. This could be utilized. Maybe we should do a separate spec on "large size messages". That wouldn't be too much effort and be a truely optional feature (I even think we could simply carry over the text from that draft version). What I am currently concerned about is putting this size issue to a rest. I think the compromise is good enough, especially as we do NOT yet specify a plain tcp mapping (we even don't know if we will ever get consensus to do that). That means the currently proposed text keeps the options open to do anything we might later decide. And it acutally puts a hard limit to roughly 64K by the fact that only UDP is supported. As I have outlined in another mail, that practical limit seems to be more 4K than 64K. Given that situation, I strongly suggest not to get another round of max size discussion. I think we urgently need now a consensus on the lowest denominator and get that consensus published. Else we will be discussing and discussing but never achive any milestone. I like the approach of baby-steps, which will give us something usable after each step. The core thing we need to do is have a format specification including layered architecture) that allows us to build on. Then, I think, we can focus on specific issues. Rainer > -Original Message- > From: Andrew Ross [mailto:[EMAIL PROTECTED] > Sent: Wednesday, November 23, 2005 9:51 AM > To: Rainer Gerhards > Subject: RE: [Syslog] RE: Message format > > > >> Mapping over UDP should be limited to a single message per packet. > >I agree on that. If we need an ultra-compact UDP delivery, we could > >later add it in a separate transport mapping. > > Yes, good idea. I doubt anyone will ever want to do this, or > at least go to > the effort of trying to get it drafted into an RFC ;-) > > >> When mapping over plain TCP I believe we should limit the > >> total message size > >> to 65507 bytes (to keep it compatible with UDP) and delimit > >> each message > >> stream with an LF, or CRLF. Either delimiter would work for me. > > >I would prefer not to restart the size discussion at this > point. I think > >the current compromise (everyone must support 2K, anyone > might support > >as much as he likes) is sufficient for most, if not all, > cases. I would > >not like to see an application to become non-compliant just > because it > >needs to transmit 65508 bytes inside a message. > > > I realise this should have been brought up earlier in the > draft process, > however, I would really like to see a limit on the message > size so that it > is directly compatible with UDP. If we allow an opened ended > message size, > people *will* use it for non syslog related things. I feel > that any message > longer than will fit into a UDP packet should be broken into > two or more > separate messages by the sender, even if sent over TCP. This > allows me to > allocate a maximum known buffer size for incoming TCP > messages. There is a > potential for huge messages filling the memory and memory > buffer overflows > happening if the messages are not limited in size. "Syslog" > is meant to be a > human readable system log message. Anything longer (including > binary crash > dumps or other things people misuse syslog for) should be broken into > separate messages by the sender, or sent over a different protocol. > > > I think we should keep syslog simple and flexible, but not at > the expense of > making it handle things it was never meant for. If a message > needs to be > broken into many chunks, the SD-ID tags could be used to tie all the > messages together again by the parser. The syslog receiver or > relay will > just handle them as separate messages and not even know they > were split. > This makes things so much simpler. > > Cheers > > Andrew > > ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
[Syslog] RE: Message format
>> Mapping over UDP should be limited to a single message per packet. >I agree on that. If we need an ultra-compact UDP delivery, we could >later add it in a separate transport mapping. Yes, good idea. I doubt anyone will ever want to do this, or at least go to the effort of trying to get it drafted into an RFC ;-) >> When mapping over plain TCP I believe we should limit the >> total message size >> to 65507 bytes (to keep it compatible with UDP) and delimit >> each message >> stream with an LF, or CRLF. Either delimiter would work for me. >I would prefer not to restart the size discussion at this point. I think >the current compromise (everyone must support 2K, anyone might support >as much as he likes) is sufficient for most, if not all, cases. I would >not like to see an application to become non-compliant just because it >needs to transmit 65508 bytes inside a message. I realise this should have been brought up earlier in the draft process, however, I would really like to see a limit on the message size so that it is directly compatible with UDP. If we allow an opened ended message size, people *will* use it for non syslog related things. I feel that any message longer than will fit into a UDP packet should be broken into two or more separate messages by the sender, even if sent over TCP. This allows me to allocate a maximum known buffer size for incoming TCP messages. There is a potential for huge messages filling the memory and memory buffer overflows happening if the messages are not limited in size. "Syslog" is meant to be a human readable system log message. Anything longer (including binary crash dumps or other things people misuse syslog for) should be broken into separate messages by the sender, or sent over a different protocol. I think we should keep syslog simple and flexible, but not at the expense of making it handle things it was never meant for. If a message needs to be broken into many chunks, the SD-ID tags could be used to tie all the messages together again by the parser. The syslog receiver or relay will just handle them as separate messages and not even know they were split. This makes things so much simpler. Cheers Andrew ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
[Syslog] Message format
WG, Sorry for joining in the discussion late. I've only just found some time to reply. My thoughts below... The new format looks great. VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID [SD-ID]s MSG Replace all received null characters with either <00> or /0. My preference is <00>. Keep MSGID in the header as a required field SD-IDs should come before the MSG. Otherwise encoding issues and MSG delimiter will become a problem. Store all messages written to disk in UTF-8 format. This allows any received encoding to be stored safely without loss or corruption. My preference is to enforce UTF-8 for data encoding on the wire. This allows US-ASCII to be used for the first 127 characters and Unicode mappings into UTF-8 for all other international characters. Trying to switch encodings for each message based on the SD-ID language or local setting will be a parsing nightmare. As far as I know, all modern systems are now capable of sending in US-ASCII or mapping their own language into UTF-8. Can anyone think of a good reason not to enforce UTF-8? I believe the above format would be easy to implement in both a sender and receiver. Mandating that the disk storage format is UTF-8 would also help reporting and parsing of all languages and character sets. Mapping over UDP should be limited to a single message per packet. When mapping over plain TCP I believe we should limit the total message size to 65507 bytes (to keep it compatible with UDP) and delimit each message stream with an LF, or CRLF. Either delimiter would work for me. Rainer, keep up your good work and persistence on the drafts. I believe the new format will solve a lot of problems. Cheers Andrew ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] Secure substrate - need your input
Good points Anton, My preference is also for using SSL/TLS. From an implementers point of view, there is a good supply of SSL/TLS components and source code available (both commercial and open source). This would make it easy for customers to add secured syslog support to their apps. We currently use SSH in our secure syslog tunnel app, but could replace it with SSL very easily. I heard that Rainer is going to add SSL support soon as well. Please, no one suggest we try and run the secure protocol over UDP :-) Let's keep to TCP as the transport for the secure/reliable protocol and keep using UDP for the simple send and forget system. The idea is to make it as simple as possible for a programmer to make his application/appliance send syslog messages. Grab an SSL library, choose a socket stack and Bob's your mother's brother. I believe that one of the reasons that 3195 is so slow to take off is that it is not easy to create an implementation from scratch. It is a lot easier to just grab a UDP socket library and put at the start of each message. Simple is as simple does. Cheers Andrew -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Anton Okmianski (aokmians) Sent: Wednesday, 26 October 2005 4:11 a.m. To: Chris Lonvick (clonvick); [EMAIL PROTECTED] Subject: RE: [Syslog] Secure substrate - need your input > Hi Folks, > > I'll be asking this in Vancouver but would like to get some > input from the mailing list. > > Our charter says that we will develop a secure method to > transport syslog messages. We have BEEP (RFC 3195) but it > has a low implementation record. > Other groups have specified BEEP as well but are also moving > along towards using SSH or SSL. > > > 1) What secure substrate should the WG look towards: > > __ ssl > > __ ssh > > __ dtls > http://www.ietf.org/internet-drafts/draft-rescorla-dtls-05.txt > > __ other I believe it should be SSL 3.0 / TLS 1.0. At least in web-server farm environments, you got SSL everywhere with a bunch of SSL accelerator hardware already in place. I also think several mappings can be defined. I am not a fan of options when it comes to standards, but allowing syslog over any kind of secure transport is a good thing. This was the whole idea of separating syslog-protocol from syslog-transport. Frankly, I don't think it will be a lot of work to define those transport mappings. > 2) Why? SSL/TLS is the most widely deployed network security protocol today. All e-commerce happens over it. Dozens of companies provide all shapes and forms of SSL accelerators. Many routers now have SSL accelartor modules. If I need to manage a large environment of servers where distributed logging really makes sense, being able to re-use existing infrastructure for scaling really helps. A search for "SSH accelerator" on google does not reveal anything interesting, but "SSL accelarator" shows up with a bunch of listings, several of which claim thousands of new SSL sessions per second. This confirms my experience. BTW, with SSL session caching, accelerators (ie. load balancers) can do 100,000 connections per second and more. So, to scale syslog over SSH would I have to wait for SSH accelerators to come to market? I see that there is a lot of work around SSH connection protocol and its potential use in new protocols. I have not followed these developments. There must have been a good reason for it. I would like to understand why people object to SSL, which is a well established technology. Any pointers? Thanks, Anton. > > > > Thanks, > Chris > > ___ > Syslog mailing list > Syslog@lists.ietf.org > https://www1.ietf.org/mailman/listinfo/syslog > ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] plain tcp syslog
Hi Rainer, A document describing syslog over TCP would be very handy. Two things that need to be identified are: the delimiting character - usually a LF (ASCII &H0A) and how long the maximum message length should be (4096 chrs seems to cover most implementations I've seen in the wild). NetScreen firewalls can also send syslog via TCP. Regards Andrew Kiwi Enterprises -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Rainer Gerhards Sent: Tuesday, 18 October 2005 1:49 a.m. To: [EMAIL PROTECTED] Subject: [Syslog] plain tcp syslog Hi WG, Chris has put some questions about RFC 3195 usage on the agenda for the next IETF. In preparation for this, I am going to ask a question that I know is very unpopular in the WG. We have discussed the issue of a very-simple, non-BEEP based plain tcp syslog several times on this list. The idea always has violently been rejected. However, the current status is that RFC 3195 is nicely standardized but seldomly implemented and even less often deployed. Plain tcp syslog, on the other hand, is not standardized but widely deployed. It is implemented at least in: - syslog-ng - rsyslog - Kiwi syslog daemon - WinSyslog/MonitorWare Agent/EventReporter - Cisco PIX As of my experience, many syslog-ng installations use plain tcp syslog. All of the implementations listed are interoperable. The list is most probably not complete, these were just the products that came immediately to my mind. The end user-base is also continously asking about such a simple transport - this is probably why it is implemented so often. Given the obvious importance of this protocol, wouldn't it make sense to at least document its observed behaviour, much as RFC 3164 documents UDP based syslog observed behaviour? Such a document could also be useful to document the security and (un)reliability issues coming along with the "plain tcp" syslog. Eventually, this could even increase demand for more reliable solutions like RFC 3195. Feedback is appreciated. Rainer Gerhards ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
RE: [Syslog] RE: [Syslog-sec] AD Review fordraft-ietf-syslog-protocol-14
The idea of using "[EMAIL PROTECTED]" sounds like a good solution to me. Regards Andrew Kiwi Enterprises -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Rainer Gerhards Sent: Tuesday, 11 October 2005 4:17 a.m. To: [EMAIL PROTECTED] Subject: [Syslog] RE: [Syslog-sec] AD Review fordraft-ietf-syslog-protocol-14 Hi list, I have talked offline to some collagues. Most of them share Alex (and my) concerns on the name space designator size. However, all share the concerns about just using x- as a prefix, thus providing no solution for namespace collisions. Given the problems we often face with namespace collisions, I think that we should actually require strict rules. So while space is an issue, it is worth to sacrify some space (octets) to solve the namespace issue. So we are in for namespace identifiers. On the issue of what exactly to use, we talked about something like ssh, but with shorter identifiers. Definitely, that would introduce a syntax not yet used in other protocols, be it looks more intuitive that the - prefix. The suggestion is to use @. So instead of #1 [EMAIL PROTECTED] or #2 19406-mySDID we could use [EMAIL PROTECTED] (19406 is the Adiscon-assigend enterprise ID - is there an example ID like the example.com domain?) This is brief, close to SNMP and looks familiar. Feedback is appreciated. If there are no objections to this approach, I will create some text. Rainer > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > Alexander Clemm (alex) > Sent: Tuesday, October 04, 2005 9:10 PM > To: Chris Lonvick (clonvick); David B Harrington > Cc: syslog-sec@employees.org > Subject: RE: [Syslog-sec] AD Review for draft-ietf-syslog-protocol-14 > > In general, the "@example.company.com" name space is a nice idea. > However, I am concerned about the length that this > introduces. I would > much prefer to have a more compact encoding, resembling what > parameters > would look like in SDP more than what they would look like > XML (in terms > of compactness). This is one reason why I actually like the > proposal to > use the company identifier (typically 3 digits) as prefix (followed by > some delimiter) as was suggested to denote a private name space. > > Just my 2 cents. > --- Alex > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Chris > Lonvick (clonvick) > Sent: Monday, October 03, 2005 6:25 PM > To: David B Harrington > Cc: syslog-sec@employees.org > Subject: RE: [Syslog-sec] AD Review for draft-ietf-syslog-protocol-14 > > Hi David, > > On Fri, 30 Sep 2005, David B Harrington wrote: > > > Hi, > > > > Because I believe we should be working to integrate our network > > management standards, at least to the point they can secure and > > correlate data easily across NM interfaces, I would like to see the > > approach adopted by syslog to be similar to the approaches used by > > other IETF protocols, especially network management protocols. > > I'd like that as well. > > > > > SNMP uses the vendor ID approach, managed by IANA. > > Netconf has no data model, so we don't know what they will use for > > vendor extensions. > > I'm not sure what ipfix is using. > > > > Who will manage the @cisco.com registrations? IANA or > another external > > > agency? Will the assignments be as stable as IANA assignments? > > The "[EMAIL PROTECTED]" namespace for SSH is for private use > and will not > be registered with anyone. It's been working well enough for the SSH > community with the warning of, "It is up to each domain how it manages > its local namespace." I will say that this practice in SSH is not as > widespread as SNMP but it has been done and it seems to be working. > > It would be good to have discussion of this on the mailing list and we > can hopefully finalize what we want in Vancouver. Your input would be > appreciated. > > Thanks, > Chris > ___ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > ___ > Syslog-sec mailing list > Syslog-sec@www.employees.org > http://www.employees.org/mailman/listinfo/syslog-sec > ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog ___ Syslog mailing list Syslog@lists.ietf.org https://www1.ietf.org/mailman/listinfo/syslog
