Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
On Aug 3, 2007, at 11:24 AM, Dave Crocker wrote: My point was about the failure to make sure there was large-scale, multi-vendor, in-the-wild *service*. Anything that constraint [in] what can go wrong will limit the ability to make the technology robust and usable. There are currently millions of unconstrained large-scale, in-the- wild services being manipulated and controlled by criminals. Constraints that must be taken seriously are related to the economies limiting the staging of DDoS attacks. Criminals often utilize tens of thousands of 0wned systems. These systems often send large email campaigns. Any scheme that expects receipt of a message to invoke a process that initiates additional traffic must be carefully considered. Expecting recipients to employ the local-part of a purported originator's email-address to then construct dozens or even hundreds of DNS transactions wholly unrelated to the actual source of the message is a prime example of how economies related to DDoS attacks are being gravely shifted in the wrong direction. Spammer are already spamming, either directly or through an ISP's outbound server. Lacking a reasonable constraint on the recipient process, criminals can attack without expending their resources and not expose the location of their systems. Using SPF/Sender-ID as an example, just one DNS resource record is able to source an attack comprised of millions of recipient generated DNS transactions. The lack of receipt process constraint in this case is an example of either negligence or incompetence. Here an attack may employ several levels of indirection and yet nothing germane to the attack will be found within email logs. Not imposing reasonable constraints does not make the Internet either more robust or usable. -Doug ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
Tom.Petch wrote: Certainly there were early prototypes of OSI modules, and even running products. ... OSI got well beyond the prototype stage. Major manufacturers produced products and I was involved with their implementation. So did minor manufacturers. We (Wollongong) developed and sold a full OSI stack, from clnp up through x.400. That's why I said "running products". At base, however, such products were purchased more as a checklist item than for real use. My point was about the failure to make sure there was large-scale, multi-vendor, in-the-wild *service*. Anything that constraint what can go wrong will limit the ability to make the technology robust and usable. It is the focus on pragmatic steps that make the technology usable that I believe Clark was referring to, by saying "running code". d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
Tom.Petch wrote: Certainly there were early prototypes of OSI modules, and even running products. ... OSI got well beyond the prototype stage. Major manufacturers produced products and I was involved with their implementation. So did minor manufacturers. We (Wollongong) developed and sold a full OSI stack, from clnp up through x.400. that's why I said "running products". At base, however, such products were purchased more as a checklist item than for real use. My point was about the failure to make sure there was large-scale, multi-vendor, in-the-wild *service*. Anything that constraint what can go wrong will limit the ability to make the technology robust and usable. It is the focus on pragmatic steps that make the technology usable that I believe Clark was referring to, by saying "running code". d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
- Original Message - From: "Dave Crocker" <[EMAIL PROTECTED]> To: "David Conrad" <[EMAIL PROTECTED]> Cc: Sent: Thursday, August 02, 2007 8:22 PM Subject: Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?) > > David Conrad wrote: > > I'd offer that the OSI protocol stack was probably significantly more > > reviewed than the TCP/IP stack. > > Depends what you mean by "more reviewed". > > More eyes looking at the specs? Probably yes. More critical analysis by > senior technical architects? Probably not. > > > At the very least, running code is an empirical proof that an > > architecture _can_ work. > > Again, depends upon what one means by running code. > > Certainly there were early prototypes of OSI modules, and even running > products. Clearly, that was not enough. In contrast, the Internet code was > deployed and used in a running service, with increasing scale. So the > distinction between prototype and production is probably of fundamental > importance. (I think that Dave Clark really meant "running service" when he > said "running code".) > OSI got well beyond the prototype stage. Major manufacturers produced products and I was involved with their implementation. From c.1990 to c.1995 we all knew that, with such a weight of political pressure behind it, OSI was bound to sweep all before it. By 1995, the tide had turned, but it was not the lack of interoperable, production software that did the turning. Tom Petch > d/ > > -- > >Dave Crocker >Brandenburg InternetWorking >bbiw.net > > ___ > Ietf mailing list > Ietf@ietf.org > https://www1.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
> yes! > I tried to resist the 47th rehash of this thread, but... too late... > > Within a commercial environment, the organization has to be > fairly convinced that their better mousetrap is going to work, > in order to fund it, productize it, document it, sell it, and support it. > > This process will always find more bugs in the mousetrap than > simply documenting it and skipping all the other steps. > > If a vendor bothers to do all this, and multiple IETFers can say in a BoF > that they have used the mousetrap and it really does work, > that is worth a whole lot more than "I read the draft and > it looks pretty good". yes. but then again, vendors are insensitive to certain kinds of bugs. the myriad bugs produced by introduction of NAT are good examples. a little bit of analysis should have convinced any responsible vendor to either not sell NAT products, or to be honest in marketing them and to accompany them with rather strong disclaimers. (not to attack NATs specifically, they're just the most obvious of many examples and the easiest ones to cite) > There is a certain amount of healthy risk that the IESG > can take when chartering new standards-track work. > Prior implementations should not be a gating factor, but > it makes their decision much easier when there is objective > evidence the mousetrap actually works and it is already being > used by the industry. again, being used by the industry is no indicator of soundness. and being used by the industry in the absence of public protocol review is highly correlated with poor design. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
David Conrad wrote: I'd offer that the OSI protocol stack was probably significantly more reviewed than the TCP/IP stack. Depends what you mean by "more reviewed". More eyes looking at the specs? Probably yes. More critical analysis by senior technical architects? Probably not. > At the very least, running code is an empirical proof that an > architecture _can_ work. Again, depends upon what one means by running code. Certainly there were early prototypes of OSI modules, and even running products. Clearly, that was not enough. In contrast, the Internet code was deployed and used in a running service, with increasing scale. So the distinction between prototype and production is probably of fundamental importance. (I think that Dave Clark really meant "running service" when he said "running code".) d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
Lixia Zhang wrote: .. I think we've seen several examples of where the IETF has spent significant amount of energy, ranging from heated discussions to specification work, on solutions that simply won't fly. It would be useful if that energy waste could be reduced. Having 'running code' as a barrier for serious consideration within the IETF may be one approach. I agree that running code should be given extra weight, but I am not sure that running code should be a requirement for something which is not well understood yet (some Lemonade WG documents come to mind). forgive me for jumping into the middle of a discussion (and I did not know which of the lemonade doc's the above referred to), but my past experience seems suggesting that an attempt to implement a "not well understood" idea is a good way towards a better understanding of how to make the idea work, or what can be potential issues. Yes, but this is only useful once one understands what is actually needed in a spec to begin with ;-). I found running code most useful when the spec is nearly ready for publication. IMHO, "running code" gets more credit than is warranted. While it is certainly useful as both proof of concept and proof of implementability, mere existence of running code says nothing about the quality of the design, its security, scalability, breadth of applicability, and so forth. "running code" was perhaps sufficient in ARPAnet days when there were only a few hundred hosts and a few thousand users of the network. It's not sufficient for global mission critical infrastructure. it seems to me the above argues that running code is necessary, but not sufficient as evidence of a sound design. Agreed. Running code is useful to identify things that are difficult to implement or unclear. (well, that is the interpretation; I have not seen anywhere a claim that running code is sufficient, but rather simply to filter out the weed) ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
Lixia Zhang wrote: .. I think we've seen several examples of where the IETF has spent significant amount of energy, ranging from heated discussions to specification work, on solutions that simply won't fly. It would be useful if that energy waste could be reduced. Having 'running code' as a barrier for serious consideration within the IETF may be one approach. I agree that running code should be given extra weight, but I am not sure that running code should be a requirement for something which is not well understood yet (some Lemonade WG documents come to mind). forgive me for jumping into the middle of a discussion (and I did not know which of the lemonade doc's the above referred to), but my past experience seems suggesting that an attempt to implement a "not well understood" idea is a good way towards a better understanding of how to make the idea work, or what can be potential issues. yes! I tried to resist the 47th rehash of this thread, but... too late... Within a commercial environment, the organization has to be fairly convinced that their better mousetrap is going to work, in order to fund it, productize it, document it, sell it, and support it. This process will always find more bugs in the mousetrap than simply documenting it and skipping all the other steps. If a vendor bothers to do all this, and multiple IETFers can say in a BoF that they have used the mousetrap and it really does work, that is worth a whole lot more than "I read the draft and it looks pretty good". There is a certain amount of healthy risk that the IESG can take when chartering new standards-track work. Prior implementations should not be a gating factor, but it makes their decision much easier when there is objective evidence the mousetrap actually works and it is already being used by the industry. But implementation and deployment are not enough alone. There also needs to be some pre-existing consensus that a standard version could be written and approved by the IETF, and people are willing to work on it. The slogan says "rough consensus and running code". It doesn't say "rough consensus, then running code". Without running code, there is no deployment. Without deployment, there is no point to this exercise. Andy IMHO, "running code" gets more credit than is warranted. While it is certainly useful as both proof of concept and proof of implementability, mere existence of running code says nothing about the quality of the design, its security, scalability, breadth of applicability, and so forth. "running code" was perhaps sufficient in ARPAnet days when there were only a few hundred hosts and a few thousand users of the network. It's not sufficient for global mission critical infrastructure. it seems to me the above argues that running code is necessary, but not sufficient as evidence of a sound design. (well, that is the interpretation; I have not seen anywhere a claim that running code is sufficient, but rather simply to filter out the weed) Lixia ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
.. I think we've seen several examples of where the IETF has spent significant amount of energy, ranging from heated discussions to specification work, on solutions that simply won't fly. It would be useful if that energy waste could be reduced. Having 'running code' as a barrier for serious consideration within the IETF may be one approach. I agree that running code should be given extra weight, but I am not sure that running code should be a requirement for something which is not well understood yet (some Lemonade WG documents come to mind). forgive me for jumping into the middle of a discussion (and I did not know which of the lemonade doc's the above referred to), but my past experience seems suggesting that an attempt to implement a "not well understood" idea is a good way towards a better understanding of how to make the idea work, or what can be potential issues. IMHO, "running code" gets more credit than is warranted. While it is certainly useful as both proof of concept and proof of implementability, mere existence of running code says nothing about the quality of the design, its security, scalability, breadth of applicability, and so forth. "running code" was perhaps sufficient in ARPAnet days when there were only a few hundred hosts and a few thousand users of the network. It's not sufficient for global mission critical infrastructure. it seems to me the above argues that running code is necessary, but not sufficient as evidence of a sound design. (well, that is the interpretation; I have not seen anywhere a claim that running code is sufficient, but rather simply to filter out the weed) Lixia ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
I'd offer that the OSI protocol stack was probably significantly more reviewed than the TCP/IP stack. At the very least, running code is an empirical proof that an architecture _can_ work. Rgds, -drc On Aug 1, 2007, at 8:35 AM, Eric Burger wrote: My faulty recollection is that in our game of rock-paper-scissors, Running Code beats Untested Idea, but Well Reviewed Architecture and Protocol beats Running Code. On 7/31/07 11:34 PM, "Keith Moore" <[EMAIL PROTECTED]> wrote: Jun-ichiro itojun Hagino wrote: IMHO, "running code" gets more credit than is warranted. While it is certainly useful as both proof of concept and proof of implementability, mere existence of running code says nothing about the quality of the design, its security, scalability, breadth of applicability, and so forth. "running code" was perhaps sufficient in ARPAnet days when there were only a few hundred hosts and a few thousand users of the network. It's not sufficient for global mission critical infrastructure. tend to agree. how about "multiple interoperable implementations"? that's certainly better than one implementation, especially if implemented on multiple platforms. though still, I think, this is not sufficient in general. again, I'm biased because I've heard too many arguments of the form "we have running code for , and it's already (somewhat) deployed so we have to approve it as a standard without changing it". Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
My faulty recollection is that in our game of rock-paper-scissors, Running Code beats Untested Idea, but Well Reviewed Architecture and Protocol beats Running Code. On 7/31/07 11:34 PM, "Keith Moore" <[EMAIL PROTECTED]> wrote: > Jun-ichiro itojun Hagino wrote: >>> IMHO, "running code" gets more credit than is warranted. While it is >>> certainly useful as both proof of concept and proof of implementability, >>> mere existence of running code says nothing about the quality of the >>> design, its security, scalability, breadth of applicability, and so >>> forth. "running code" was perhaps sufficient in ARPAnet days when there >>> were only a few hundred hosts and a few thousand users of the network. >>> It's not sufficient for global mission critical infrastructure. >>> >> >> tend to agree. how about "multiple interoperable implementations"? >> > that's certainly better than one implementation, especially if > implemented on multiple platforms. though still, I think, this is not > sufficient in general. > > again, I'm biased because I've heard too many arguments of the form "we > have running code for , and it's already (somewhat) > deployed so we have to approve it as a standard without changing it". > > Keith > > > ___ > Ietf mailing list > Ietf@ietf.org > https://www1.ietf.org/mailman/listinfo/ietf Notice: This email message, together with any attachments, may contain information of BEA Systems, Inc., its subsidiaries and affiliated entities, that may be confidential, proprietary, copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
Keith Moore wrote: The danger here is that when people bring work to IETF, they might refuse to change protocols which are already deployed. This already happens to far too great a degree. People keep arguing that because they have running/deployed code, IETF has to standardize exactly what they have already produced. In many cases things that are deployed before they get widespread design review are very poorly designed. Indeed. And I have to admit that I have been in situations like this myself. There were several cases when I was reluctant to upgrade my code to the latest draft when I've implemented a previous version. I think we've seen several examples of where the IETF has spent significant amount of energy, ranging from heated discussions to specification work, on solutions that simply won't fly. It would be useful if that energy waste could be reduced. Having 'running code' as a barrier for serious consideration within the IETF may be one approach. I agree that running code should be given extra weight, but I am not sure that running code should be a requirement for something which is not well understood yet (some Lemonade WG documents come to mind). IMHO, "running code" gets more credit than is warranted. While it is certainly useful as both proof of concept and proof of implementability, Yes. I found that implementing a spec before WG/IETF LC pretty much always improves the spec. So although I see many benefits in running code, I don't think it should be given more weight that it deserves. mere existence of running code says nothing about the quality of the design, its security, scalability, breadth of applicability, and so forth. Agree. "running code" was perhaps sufficient in ARPAnet days when there were only a few hundred hosts and a few thousand users of the network. It's not sufficient for global mission critical infrastructure. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
On Tue, 2007-07-31 at 17:24 -0400, Keith Moore wrote: > IMHO, "running code" gets more credit than is warranted. While it is > certainly useful as both proof of concept and proof of > implementability, mere existence of running code says nothing about > the quality of the design, its security, scalability, breadth of > applicability, and so forth. "running code" was perhaps sufficient in > ARPAnet days when there were only a few hundred hosts and a few > thousand users of the network. It's not sufficient for global mission > critical infrastructure. A simple axiom "Do not execute scripts from strangers" is often violated. The Noscript plugin for Firefox represents an excellent (and highly recommended) example of this principle in action. Unfortunately, a mouse-click is often not required for a computer to become 0wned. : ( When coping with spam, security issues related to DNS are often ignored. Concerns raised in the draft-otis-spf-dos-exploit were dismissed by suggesting list of bogus NS records are not limited to the same extent anyway. Many libraries implementing SPF do not impose limits on the MX record, or the number of NXDOMAIN, suggested as fixes in the OpenSPF group's rebuttal. http://www.openspf.org/draft-otis-spf-dos-exploit_Analysis Ironically, the rebuttal points out a bogus NS record method that worsens a DDoS barrage that can be caused by SPF. SPF remains a significant risk, even when limited to just 10 SPF record transactions per email-address evaluated. With local-part macro expansion, these DNS transactions represent a gift of a recipient's resources given at no cost to the spammer. DDoS attacks made absolutely free and unstoppable! Offering a method to macro expand elements of the email-address local-part, when used in a spam campaign, allows a _single_ cached SPF record to cause an _unlimited_ number of DNS transactions from any remote DNS resolver servicing SPF libraries. Uncached targeted DDoS attacks are not tracked by email logs and can not be mitigated. The gain of this attack can be highly destructive, while remaining virtually free to spammers wishing to also stage the attack. In addition to offering a means for staging a DDoS attack on authoritative servers, unfettered access afforded to remote recursive DNS servers by SPF scripts permits brute force DNS poisoning. Even knowing whether SPF related exploits are ongoing is not easily discerned. With the current state of affairs related to web browsers, the axiom "Do not execute scripts from strangers" is a concept that should be seriously taken to heart. -Doug ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
Jun-ichiro itojun Hagino wrote: >> IMHO, "running code" gets more credit than is warranted. While it is >> certainly useful as both proof of concept and proof of implementability, >> mere existence of running code says nothing about the quality of the >> design, its security, scalability, breadth of applicability, and so >> forth. "running code" was perhaps sufficient in ARPAnet days when there >> were only a few hundred hosts and a few thousand users of the network. >> It's not sufficient for global mission critical infrastructure. >> > > tend to agree. how about "multiple interoperable implementations"? > that's certainly better than one implementation, especially if implemented on multiple platforms. though still, I think, this is not sufficient in general. again, I'm biased because I've heard too many arguments of the form "we have running code for , and it's already (somewhat) deployed so we have to approve it as a standard without changing it". Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
> IMHO, "running code" gets more credit than is warranted. While it is > certainly useful as both proof of concept and proof of implementability, > mere existence of running code says nothing about the quality of the > design, its security, scalability, breadth of applicability, and so > forth. "running code" was perhaps sufficient in ARPAnet days when there > were only a few hundred hosts and a few thousand users of the network. > It's not sufficient for global mission critical infrastructure. tend to agree. how about "multiple interoperable implementations"? itojun ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
on the value of "running code" (was Re: Do you want to have more meetings outside US ?)
> The danger here is that when people bring work to IETF, they might > refuse to change protocols which are already deployed. This already happens to far too great a degree. People keep arguing that because they have running/deployed code, IETF has to standardize exactly what they have already produced. In many cases things that are deployed before they get widespread design review are very poorly designed. >> I think we've seen several examples of where the IETF has spent >> significant amount of energy, ranging from heated discussions to >> specification work, on solutions that simply won't fly. It would be >> useful if that energy waste could be reduced. Having 'running code' as >> a barrier for serious consideration within the IETF may be one approach. > I agree that running code should be given extra weight, but I am not > sure that running code should be a requirement for something which is > not well understood yet (some Lemonade WG documents come to mind). IMHO, "running code" gets more credit than is warranted. While it is certainly useful as both proof of concept and proof of implementability, mere existence of running code says nothing about the quality of the design, its security, scalability, breadth of applicability, and so forth. "running code" was perhaps sufficient in ARPAnet days when there were only a few hundred hosts and a few thousand users of the network. It's not sufficient for global mission critical infrastructure. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
