Re: Scenario: OpenSSL in D language, pros/cons
On 5/6/14, 2:37 AM, JR wrote: Apologies for the negativity. It's not that much of a deal, but your code will have to be very unreliant upon phobos to be completely @safe. It's a huge deal. Most of phobos should be @safe or @trusted! Please submit everything you find as bugs. Thanks! -- Andrei
Re: Scenario: OpenSSL in D language, pros/cons
On Monday, 5 May 2014 at 14:59:13 UTC, Etienne wrote: On 2014-05-04 4:34 AM, Daniele M. wrote: I have read this excellent article by David A. Wheeler: http://www.dwheeler.com/essays/heartbleed.html And since D language was not there, I mentioned it to him as a possible good candidate due to its static typing and related features. However, now I am asking the community here: would a D implementation (with GC disabled) of OpenSSL have been free from Heartbleed-type vulnerabilities? Specifically http://cwe.mitre.org/data/definitions/126.html and http://cwe.mitre.org/data/definitions/20.html as David mentions. I find this perspective very interesting, please advise :) I'm currently working on a TLS library using only D. I've shared the ASN.1 parser here: https://github.com/globecsys/asn1.d The ASN.1 format allows me to compile the data structures to D from the tls.asn1 in the repo I linked to. It uses the equivalent of D template structures extensively with what's called an Information Object Class. Obviously, when it's done I need a DER serializer/deserializer which I intend on editing MsgPackD, and then I can do a handshake (read a ASN.1 certificate) and encrypt/decrypt AES/RSA using the certificate information and this cryptography library: https://github.com/apartridge/crypto . I've never expected any help so I'm not sure what the licensing will be. I'm currently working on the generation step for the ASN.1 to D compiler, it's very fun to make a compiler in D. I've been working on an ASN.1 parser and D code generator [0]. Didn't get the BER encoder working but.. I thought it may be a good idea mention it here. [0] https://github.com/rikkimax/asn1
Re: Scenario: OpenSSL in D language, pros/cons
On Monday, 5 May 2014 at 15:01:05 UTC, Andrei Alexandrescu wrote: On 5/5/14, 2:32 AM, JR wrote: On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote: And then comes my next question: except for that malloc-hack, would it have been possible to write it in @safe D? I guess that if not, module(s) could have been made un-@safe. Not saying that a similar separation of concerns was not possible in OpenSSL itself, but that D could have made it less development-expensive in my opinion. TDPL SafeD visions notwithstanding, @safe is very very limiting. I/O is forbidden so simple Hello Worlds are right out, let alone advanced socket libraries. Sounds like a library bug. Has it been submitted? -- Andrei When mentioned in #d it was met with replies of "well *obviously*", so I chalked it up to an irreconcilability of @safe's. Perhaps I'm expecting too much of the subset. My code certainly does no pointer arithmetic, but adding @safe: in select locations quickly shows that string operations like indexOf are unsafe, as is everything in concurrency (including thisTid), getopt, std.conv.to (to avoid casts), and all socket operations. All of those can throw, but I don't see how they can corrupt memory. Apologies for the negativity. It's not that much of a deal, but your code will have to be very unreliant upon phobos to be completely @safe. I appreciate these threads gauging community reactions and I hope the mood will be lighter post-dconf, but at present there's still a sour taste left in my mouth after the final-by-default pivot. (Manu and Thaut, please don't leave~ :< )
Re: Scenario: OpenSSL in D language, pros/cons
On Tue, 06 May 2014 09:56:11 +0200 Timon Gehr via Digitalmars-d wrote: > On 05/05/2014 12:41 PM, Jonathan M Davis via Digitalmars-d wrote: > > Regardless, there's > > nothing fundamentally limited about @safe except for operations > > which are actually unsafe with regards to memory > > What does 'actually unsafe' mean? @safe will happily ban statements > that will never 'actually' result in memory corruption. @safe is supposed to ban any code where it can't prove that it can't access memory that it shouldn't - be it to read from it to write to it - as well as code that it can't prove will not result in other code accessing memory that it shouldn't have access to. It's quite possible that some code which shouldn't be being banned is currently being banned, and it may be the case that we'll be stuck with some code being banned simply because we can't make the compiler smart enough to know that it doesn't need to ban it (though hopefully that's kept to a minimum). But @trusted is there precisely for the situations where an operation is unsafe in the general case but is perfectly safe in a specific case based on information that the compiler does not have access to but the programmer does (such as knowing that certain values will never be passed to the given function). However, I don't think that there's much question that at this point @safe isn't quite correct. There are definitely cases right now that are considered @safe when they shouldn't be (e.g. https://issues.dlang.org/show_bug.cgi?id=8838 ), and I expect that there are also cases where code is considered @system when the compiler should consider it @safe (though I don't know of any of those off the top of my head). - Jonathan M Davis
Re: Scenario: OpenSSL in D language, pros/cons
On 05/05/2014 12:41 PM, Jonathan M Davis via Digitalmars-d wrote: Regardless, there's nothing fundamentally limited about @safe except for operations which are actually unsafe with regards to memory What does 'actually unsafe' mean? @safe will happily ban statements that will never 'actually' result in memory corruption.
Re: Scenario: OpenSSL in D language, pros/cons
On 2014-05-05 2:54 PM, Daniele M. wrote: Have you thought about creating an SSL/TLS implementations tester instead? You mean testing existing TLS libraries using this information? The advantages of using all-D is having zero-copy buffers that inline with the other layers of streams when built inside another D project. I can also add processor-specific assembler-code algorithms of AES and RSA from openSSL (optimizing the critical parts can put it on par or better in speed). To answer the question about safety, the code is very modular and so when you decide to zero out memory of keys before/after serialization/deserialization or even for the buffers, it happens for everything regardless of the complexity of the application. It's definitely easier to make it safer!
Re: Scenario: OpenSSL in D language, pros/cons
On Monday, 5 May 2014 at 10:41:41 UTC, Jonathan M Davis via Digitalmars-d wrote: On Mon, 05 May 2014 10:24:27 + via Digitalmars-d wrote: On Monday, 5 May 2014 at 09:32:40 UTC, JR wrote: > On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote: >> And then comes my next question: except for that >> malloc-hack, >> would it have been possible to write it in @safe D? I guess >> that if not, module(s) could have been made un-@safe. Not >> saying that a similar separation of concerns was not >> possible >> in OpenSSL itself, but that D could have made it less >> development-expensive in my opinion. > > TDPL SafeD visions notwithstanding, @safe is very very > limiting. > > I/O is forbidden so simple Hello Worlds are right out, let > alone advanced socket libraries. I/O is not forbidden, it's just that writeln and friends currently can't be made safe, but that is being worked on AFAIK. While I/O usually goes through the OS, the system calls can be manually verified and made @trusted. As the underlying OS calls are all C functions, there will always be @system code involved in I/O, but in most cases, we should be able to wrap those functions in D functions which are @trusted. Regarldess, I would think that SSL could be implemented without sockets - that is, all of its operations should be able to operate on arbitrary data regardless of whether that data is sent over a socket or not. And if that's the case, then even if the socket operations themselves had to be @system, then everything else should still be able to be @safe. Most of the problems with @safe stem either from library functions that don't use it like they should, or because the compiler does not yet do a good enough job with attribute inference on templated functions. Both problems are being addressed, so the situation will improve over time. Regardless, there's nothing fundamentally limited about @safe except for operations which are actually unsafe with regards to memory, and any case where something isn't @safe when it's actually memory safe should be and will be fixed (as well as any situation which isn't memory safe but is considered @safe anyway - we do unfortunately still have a few of those). - Jonathan M Davis You nailed it. If we wanted to translate the theoretical exercise into something real, it would be nice to have an implementation of PolarSSL that works on ring buffers only, then leave network layer integration to clients. Much cleaner separation of concerns.
Re: Scenario: OpenSSL in D language, pros/cons
On Monday, 5 May 2014 at 14:59:13 UTC, Etienne wrote: On 2014-05-04 4:34 AM, Daniele M. wrote: I have read this excellent article by David A. Wheeler: http://www.dwheeler.com/essays/heartbleed.html And since D language was not there, I mentioned it to him as a possible good candidate due to its static typing and related features. However, now I am asking the community here: would a D implementation (with GC disabled) of OpenSSL have been free from Heartbleed-type vulnerabilities? Specifically http://cwe.mitre.org/data/definitions/126.html and http://cwe.mitre.org/data/definitions/20.html as David mentions. I find this perspective very interesting, please advise :) I'm currently working on a TLS library using only D. I've shared the ASN.1 parser here: https://github.com/globecsys/asn1.d The ASN.1 format allows me to compile the data structures to D from the tls.asn1 in the repo I linked to. It uses the equivalent of D template structures extensively with what's called an Information Object Class. Obviously, when it's done I need a DER serializer/deserializer which I intend on editing MsgPackD, and then I can do a handshake (read a ASN.1 certificate) and encrypt/decrypt AES/RSA using the certificate information and this cryptography library: https://github.com/apartridge/crypto . I've never expected any help so I'm not sure what the licensing will be. I'm currently working on the generation step for the ASN.1 to D compiler, it's very fun to make a compiler in D. This is a quite radical approach, I am very interested to see its development! Have you thought about creating an SSL/TLS implementations tester instead? With the compiled information I see this goal quite well in range.
Re: Scenario: OpenSSL in D language, pros/cons
05-May-2014 18:59, Etienne пишет: On 2014-05-04 4:34 AM, Daniele M. wrote: I have read this excellent article by David A. Wheeler: http://www.dwheeler.com/essays/heartbleed.html And since D language was not there, I mentioned it to him as a possible good candidate due to its static typing and related features. However, now I am asking the community here: would a D implementation (with GC disabled) of OpenSSL have been free from Heartbleed-type vulnerabilities? Specifically http://cwe.mitre.org/data/definitions/126.html and http://cwe.mitre.org/data/definitions/20.html as David mentions. I find this perspective very interesting, please advise :) I'm currently working on a TLS library using only D. I've shared the ASN.1 parser here: https://github.com/globecsys/asn1.d Cool, keep us posted. The ASN.1 format allows me to compile the data structures to D from the tls.asn1 in the repo I linked to. It uses the equivalent of D template structures extensively with what's called an Information Object Class. Obviously, when it's done I need a DER serializer/deserializer which I intend on editing MsgPackD, and then I can do a handshake (read a ASN.1 certificate) and encrypt/decrypt AES/RSA using the certificate information and this cryptography library: https://github.com/apartridge/crypto . I've never expected any help so I'm not sure what the licensing will be. I'm currently working on the generation step for the ASN.1 to D compiler, it's very fun to make a compiler in D. Aye, D seems to be a nice choice for writing compilers. -- Dmitry Olshansky
Re: Scenario: OpenSSL in D language, pros/cons
On 5/5/14, 2:32 AM, JR wrote: On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote: And then comes my next question: except for that malloc-hack, would it have been possible to write it in @safe D? I guess that if not, module(s) could have been made un-@safe. Not saying that a similar separation of concerns was not possible in OpenSSL itself, but that D could have made it less development-expensive in my opinion. TDPL SafeD visions notwithstanding, @safe is very very limiting. I/O is forbidden so simple Hello Worlds are right out, let alone advanced socket libraries. Sounds like a library bug. Has it been submitted? -- Andrei
Re: Scenario: OpenSSL in D language, pros/cons
On 2014-05-04 4:34 AM, Daniele M. wrote: I have read this excellent article by David A. Wheeler: http://www.dwheeler.com/essays/heartbleed.html And since D language was not there, I mentioned it to him as a possible good candidate due to its static typing and related features. However, now I am asking the community here: would a D implementation (with GC disabled) of OpenSSL have been free from Heartbleed-type vulnerabilities? Specifically http://cwe.mitre.org/data/definitions/126.html and http://cwe.mitre.org/data/definitions/20.html as David mentions. I find this perspective very interesting, please advise :) I'm currently working on a TLS library using only D. I've shared the ASN.1 parser here: https://github.com/globecsys/asn1.d The ASN.1 format allows me to compile the data structures to D from the tls.asn1 in the repo I linked to. It uses the equivalent of D template structures extensively with what's called an Information Object Class. Obviously, when it's done I need a DER serializer/deserializer which I intend on editing MsgPackD, and then I can do a handshake (read a ASN.1 certificate) and encrypt/decrypt AES/RSA using the certificate information and this cryptography library: https://github.com/apartridge/crypto . I've never expected any help so I'm not sure what the licensing will be. I'm currently working on the generation step for the ASN.1 to D compiler, it's very fun to make a compiler in D.
Re: Scenario: OpenSSL in D language, pros/cons
On Mon, 05 May 2014 10:24:27 + via Digitalmars-d wrote: > On Monday, 5 May 2014 at 09:32:40 UTC, JR wrote: > > On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote: > >> And then comes my next question: except for that malloc-hack, > >> would it have been possible to write it in @safe D? I guess > >> that if not, module(s) could have been made un-@safe. Not > >> saying that a similar separation of concerns was not possible > >> in OpenSSL itself, but that D could have made it less > >> development-expensive in my opinion. > > > > TDPL SafeD visions notwithstanding, @safe is very very limiting. > > > > I/O is forbidden so simple Hello Worlds are right out, let > > alone advanced socket libraries. > > I/O is not forbidden, it's just that writeln and friends > currently can't be made safe, but that is being worked on AFAIK. > While I/O usually goes through the OS, the system calls can be > manually verified and made @trusted. As the underlying OS calls are all C functions, there will always be @system code involved in I/O, but in most cases, we should be able to wrap those functions in D functions which are @trusted. Regarldess, I would think that SSL could be implemented without sockets - that is, all of its operations should be able to operate on arbitrary data regardless of whether that data is sent over a socket or not. And if that's the case, then even if the socket operations themselves had to be @system, then everything else should still be able to be @safe. Most of the problems with @safe stem either from library functions that don't use it like they should, or because the compiler does not yet do a good enough job with attribute inference on templated functions. Both problems are being addressed, so the situation will improve over time. Regardless, there's nothing fundamentally limited about @safe except for operations which are actually unsafe with regards to memory, and any case where something isn't @safe when it's actually memory safe should be and will be fixed (as well as any situation which isn't memory safe but is considered @safe anyway - we do unfortunately still have a few of those). - Jonathan M Davis
Re: Scenario: OpenSSL in D language, pros/cons
On Monday, 5 May 2014 at 09:32:40 UTC, JR wrote: On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote: And then comes my next question: except for that malloc-hack, would it have been possible to write it in @safe D? I guess that if not, module(s) could have been made un-@safe. Not saying that a similar separation of concerns was not possible in OpenSSL itself, but that D could have made it less development-expensive in my opinion. TDPL SafeD visions notwithstanding, @safe is very very limiting. I/O is forbidden so simple Hello Worlds are right out, let alone advanced socket libraries. I/O is not forbidden, it's just that writeln and friends currently can't be made safe, but that is being worked on AFAIK. While I/O usually goes through the OS, the system calls can be manually verified and made @trusted.
Re: Scenario: OpenSSL in D language, pros/cons
On Sunday, 4 May 2014 at 21:18:24 UTC, Daniele M. wrote: And then comes my next question: except for that malloc-hack, would it have been possible to write it in @safe D? I guess that if not, module(s) could have been made un-@safe. Not saying that a similar separation of concerns was not possible in OpenSSL itself, but that D could have made it less development-expensive in my opinion. TDPL SafeD visions notwithstanding, @safe is very very limiting. I/O is forbidden so simple Hello Worlds are right out, let alone advanced socket libraries.
Re: Scenario: OpenSSL in D language, pros/cons
On Monday, 5 May 2014 at 08:04:24 UTC, Jonathan M Davis via Digitalmars-d wrote: On Mon, 05 May 2014 07:39:13 + Paulo Pinto via Digitalmars-d wrote: Sometimes I wonder how much money have C design decisions cost the industry in terms of anti-virus, static and dynamic analyzers tools, operating systems security enforcements, security research and so on. All avoidable with bound checking by default and no implicit conversions between arrays and pointers. Well, a number of years ago, the folks who started the codebase of the larger products at the company I work at insisted on using COM everywhere, because we _might_ have to interact with 3rd parties, and they _might_ not want to use C++. So, foolishly, they mandated that _nowhere_ in the codebase should any C++ objects be passed around except by pointer. They then had manual reference counting on top of that to deal with memory management. That decision has cost us man _years_ in time working on reference counting-related bugs. Simply using smart pointers instead would probably have saved the company millions. COM may have its place, but forcing a whole C++ codebase to function that way was just stupid, especially when pretty much none of it ever had to interact directly with 3rd party code (and even if it had, it should have been done through strictly defined wrapper libraries; it doesn't make sense that 3rd parties would hook into the middle of your codebase). So the decision was made and CComPtr<> and _com_ptr_t<> weren't used?! I feel your pain. Well, now the codebase is WinRT ready. :) Seemingly simple decisions can have _huge_ consequences - especially when that decision affects millions of lines of code, and that's definitely the case with some of the decisions made for C. Some of them may have be unavoidable give the hardware situation and programming climate at the time that C was created, but we've been paying for them ever since. And unfortunately, the way things are going at this point, nothing will ever really overthrow C. We'll have to deal with it on some level for a long, long time to come. - Jonathan M Davis I doubt those decisions really made sense, given the other system programming languages at the time. Most of them did bounds checking, had no implicit conversions and operating systems were being written with them. Algol 60 reference compiler did not allow disabling bounds checking at all, for example. Quote from Tony Hoare's ACM award article[1]: "A consequence of this principle is that every occurrence of every subscript of every subscripted variable was on every occasion checked at run time against both the upper and the lower declared bounds of the array. Many years later we asked our customers whether they wished us to provide an option to switch off these checks in the interests of efficiency on production runs. Unanimously, they urged us not to--they already knew how frequently subscript errors occur on production runs where failure to detect them could be disastrous. I note with fear and horror that even in 1980 language designers and users have not learned this lesson. In any respectable branch of engineering, failure to observe such elementary precautions would have long been against the law." [1]http://www.labouseur.com/projects/codeReckon/papers/The-Emperors-Old-Clothes.pdf
Re: Scenario: OpenSSL in D language, pros/cons
On Mon, 05 May 2014 07:39:13 + Paulo Pinto via Digitalmars-d wrote: > Sometimes I wonder how much money have C design decisions cost > the industry in terms of anti-virus, static and dynamic analyzers > tools, operating systems security enforcements, security research > and so on. > > All avoidable with bound checking by default and no implicit > conversions between arrays and pointers. Well, a number of years ago, the folks who started the codebase of the larger products at the company I work at insisted on using COM everywhere, because we _might_ have to interact with 3rd parties, and they _might_ not want to use C++. So, foolishly, they mandated that _nowhere_ in the codebase should any C++ objects be passed around except by pointer. They then had manual reference counting on top of that to deal with memory management. That decision has cost us man _years_ in time working on reference counting-related bugs. Simply using smart pointers instead would probably have saved the company millions. COM may have its place, but forcing a whole C++ codebase to function that way was just stupid, especially when pretty much none of it ever had to interact directly with 3rd party code (and even if it had, it should have been done through strictly defined wrapper libraries; it doesn't make sense that 3rd parties would hook into the middle of your codebase). Seemingly simple decisions can have _huge_ consequences - especially when that decision affects millions of lines of code, and that's definitely the case with some of the decisions made for C. Some of them may have be unavoidable give the hardware situation and programming climate at the time that C was created, but we've been paying for them ever since. And unfortunately, the way things are going at this point, nothing will ever really overthrow C. We'll have to deal with it on some level for a long, long time to come. - Jonathan M Davis
Re: Scenario: OpenSSL in D language, pros/cons
On Monday, 5 May 2014 at 06:35:07 UTC, Jonathan M Davis via Digitalmars-d wrote: On Sun, 04 May 2014 13:29:33 + Meta via Digitalmars-d wrote: The only language I would really trust is one in which it is impossible to write unsafe code, because you can then know that the developers can't use such unsafe hacks, even if they wanted to. Realistically, I think that you ultimately have to rely on the developers doing a good job. Good tools help a great deal (including a programming language that's safe by default while still generally being efficient), but if you try and restrict the programmer such that they can only do things that are guaranteed to be safe, I think that you're bound to make it impossible to do a number of things, which tends to not only be very frustrating to the programmers, but it can also make it impossible to get the performance that you need in some circumstances. So, while you might be able to better trust a library written in a language that's designed to make certain types of problems impossible, I don't think that it's realistic for that language to get used much in anything performance critical like an SSL implementation. Ultimately, I think that the trick is to make things as safe as they can be without actually making it so that the programmer can't do what they need to be able to do. And while, I don't think that D hit the perfect balance on that one (e.g. we should have made @safe the default if we wanted that), I think that we've done a good job of it overall - certainly far better than C or C++. - Jonathan M Davis Sometimes I wonder how much money have C design decisions cost the industry in terms of anti-virus, static and dynamic analyzers tools, operating systems security enforcements, security research and so on. All avoidable with bound checking by default and no implicit conversions between arrays and pointers. -- Paulo
Re: Scenario: OpenSSL in D language, pros/cons
On Sun, 04 May 2014 13:29:33 + Meta via Digitalmars-d wrote: > The only language I would > really trust is one in which it is impossible to write unsafe > code, because you can then know that the developers can't use > such unsafe hacks, even if they wanted to. Realistically, I think that you ultimately have to rely on the developers doing a good job. Good tools help a great deal (including a programming language that's safe by default while still generally being efficient), but if you try and restrict the programmer such that they can only do things that are guaranteed to be safe, I think that you're bound to make it impossible to do a number of things, which tends to not only be very frustrating to the programmers, but it can also make it impossible to get the performance that you need in some circumstances. So, while you might be able to better trust a library written in a language that's designed to make certain types of problems impossible, I don't think that it's realistic for that language to get used much in anything performance critical like an SSL implementation. Ultimately, I think that the trick is to make things as safe as they can be without actually making it so that the programmer can't do what they need to be able to do. And while, I don't think that D hit the perfect balance on that one (e.g. we should have made @safe the default if we wanted that), I think that we've done a good job of it overall - certainly far better than C or C++. - Jonathan M Davis
Re: Scenario: OpenSSL in D language, pros/cons
On 04/05/14 23:20, Daniele M. wrote: You are right, devs would eventually abuse everything possible, although it would make it for sure more visible: you cannot advertize an un-@safe library as @safe, although I agree that a lot depends from devs/users culture. In D, you can at least statically determine which part of the code is @safe and un-@safe. -- /Jacob Carlborg
Re: Scenario: OpenSSL in D language, pros/cons
On Sun, 04 May 2014 21:18:22 + "Daniele M. via Digitalmars-d" wrote: > On Sunday, 4 May 2014 at 10:23:38 UTC, Jonathan M Davis via > Digitalmars-d wrote: > And then comes my next question: except for that malloc-hack, > would it have been possible to write it in @safe D? I guess that > if not, module(s) could have been made un-@safe. Not saying that > a similar separation of concerns was not possible in OpenSSL > itself, but that D could have made it less development-expensive > in my opinion. I don't know what all OpenSSL is/was doing, and I haven't looked into it in great detail. I'm familiar with what caused heartbleed, and I'm somewhat familiar with OpenSSL's API from having dealt with it at work, but most of what I know about OpenSSL, I know from co-workers who have had to deal with it and other stuff that I've read about it, and in general, from what I understand, it's just plain badly designed and badly written, and it's a miracle that it works as well as it does. Most of the problems seem to stem from how the project is managed (including having horrible coding style and generally not liking to merge patches), but it's also certain that a number of the choices that they've made make it easier for security problems to creep in (e.g. using their own malloc in an attempt to gain some speed on some OSes). >From what I know of SSL itself (and I've read some of the spec, but not all of it), very little of it (and probably none of it save for the actual operations on the sockets) actually requires anything that's @system. The problem is when you go to great lengths to optimize the code, which the OpenSSL guys seem to have done. When you do that, you do things like turn off array bounds checking and generally try and avoid many of the safety features that a language like D provides, since many of them do incur at least some overhead. Actually implementing SSL itself wouldn't take all that long from what I understand. The main problem is in maintenance - probably in particular with regards to the fact that you'd have to keep adding support for more encryption methods as they come out (which technically aren't part of SSL itself), but I'm not familiar enough with the details to know all of the nooks and crannies that would cause maintenance nightmares. The base spec is less than 70 pages long though. The fellow who answered the question here seems to think that implementing SSL itself is actually fairly easy and that he's done it several times for companies already: http://security.stackexchange.com/questions/55465 I fully expect that if someone were to implement it in D, it would be safer out of the box than a C implementation would be. But if you had to start playing tricks to get it faster, that would increase the security risk, and in order for folks to trust it, you'd have to get the code audited, which is a whole other order of pain (and one that potentially costs money, depending on who does the auditing). > > If I had more time, I'd actually be tempted to write an SSL > > implementation in > > D, but even if I were to do an excellent job of it, it would > > still need to be > > vetted by security experts to make sure that it didn't have > > horrible security > > bugs in it (much as it would be likely that there would be > > fewer thanks to > > the fact that it would be writen in D), and I suspect that it's > > the kind of > > thing that many people aren't likely to trust because of how > > critical it is. > Nobody would expect/trust a single person to do this job :P > Working in an open source project would be best. If someone around here implemented SSL in D, I fully expect that it would be open source, and I fully expect that one person could do it. It's just a question of how long it would take them - though obviously sharing the work among multiple people could make it faster. Where it's definitely required that more people get involved is when you want the code audited to ensure that it's actually safe and secure. And that's where implementing an SSL library is fundamentally different from implementing most libraries - it's so integral to security that it doesn't really cut it to just throw an implementation together and toss it out there for folks to use it if they're interested. > > Unfortunately, even if > > something better _were_ written in D, it's probably only the D > > folks who > > would benefit, since it's not terribly likely at this point > > that very many > > folks are going to wrap a D library in order to use it another > > language. > > > Here I don't completely agree: if we can have a binary-compatible > implementation done in D, then we would be able to modify > software to eventually use it as a dependency. I don't see the > necessary D dependencies as prohibitive here. If D were to be part of a typical Linux distro, a D compiler would have to be part of a typical Linux distro. We're getting there with gdc as it's getting merged into gcc, but I don't think that
Re: Scenario: OpenSSL in D language, pros/cons
On Sunday, 4 May 2014 at 13:29:34 UTC, Meta wrote: On Sunday, 4 May 2014 at 08:34:20 UTC, Daniele M. wrote: I have read this excellent article by David A. Wheeler: http://www.dwheeler.com/essays/heartbleed.html And since D language was not there, I mentioned it to him as a possible good candidate due to its static typing and related features. However, now I am asking the community here: would a D implementation (with GC disabled) of OpenSSL have been free from Heartbleed-type vulnerabilities? Specifically http://cwe.mitre.org/data/definitions/126.html and http://cwe.mitre.org/data/definitions/20.html as David mentions. I find this perspective very interesting, please advise :) While D is a somewhat safer language by *default*, it makes it fairly easy to escape from the safe part of the language and write unsafe code (array bounds checking can be turned off even for @safe code). Seeing as the OpenSSL devs went as far as to write an a buggy, custom implementation of malloc for a speed gain, turning off array bounds checking and ignoring @safe seems like the first thing they would do. The only language I would really trust is one in which it is impossible to write unsafe code, because you can then know that the developers can't use such unsafe hacks, even if they wanted to. You are right, devs would eventually abuse everything possible, although it would make it for sure more visible: you cannot advertize an un-@safe library as @safe, although I agree that a lot depends from devs/users culture.
Re: Scenario: OpenSSL in D language, pros/cons
On Sunday, 4 May 2014 at 10:23:38 UTC, Jonathan M Davis via Digitalmars-d wrote: On Sun, 04 May 2014 08:34:19 + "Daniele M. via Digitalmars-d" wrote: I have read this excellent article by David A. Wheeler: http://www.dwheeler.com/essays/heartbleed.html And since D language was not there, I mentioned it to him as a possible good candidate due to its static typing and related features. However, now I am asking the community here: would a D implementation (with GC disabled) of OpenSSL have been free from Heartbleed-type vulnerabilities? Specifically http://cwe.mitre.org/data/definitions/126.html and http://cwe.mitre.org/data/definitions/20.html as David mentions. I find this perspective very interesting, please advise :) @safe code protects against indexing an array out-of-bounds. So, if OpenSSL had been implemented in D, and its heartbeat feature used @safe code, then heartbleed would not have been possible. As soon as an attempt was made to index passed the end of the array, it would have thrown a RangeError and killed the program. This is what I thought too. Now, even if OpenSSL had been implemented in D, if it had used @system or @trusted code for its heartbeat feature, then it could have had the bug just as easily in D as it did in C. And given all of the horrible practices in OpenSSL, I very much doubt that having it written in D would have prevented much, because anyone making the choices that the OpenSSL folks have been making would likely have ended up with horrible D code which was mostly @system and probably doing all kinds of things that are inherently risky, forgoeing many of the benefits that D provides. I think that it's safe to say that D makes it easier to catch problems like this, but it doesn't entirely prevent them, and bad programming practices can pretty much always get around protections that the language provides unless the language provides no ways around those protections (which isn't the case in D, because it's a systems language and needs to provide low-level access and features for those programs that need them - it just doesn't use those by default). I should have said: a D implementation of OpenSSL *without* the malloc sorcerery, sorry for leaving it out of OP. Reason I'd like to ignore that for this discussion is that I consider it a bug. And we are all in the post-heartbleed scenario now so it looks huge, but Theo De Raadt already said it all, nothing to add on it. And then comes my next question: except for that malloc-hack, would it have been possible to write it in @safe D? I guess that if not, module(s) could have been made un-@safe. Not saying that a similar separation of concerns was not possible in OpenSSL itself, but that D could have made it less development-expensive in my opinion. If I had more time, I'd actually be tempted to write an SSL implementation in D, but even if I were to do an excellent job of it, it would still need to be vetted by security experts to make sure that it didn't have horrible security bugs in it (much as it would be likely that there would be fewer thanks to the fact that it would be writen in D), and I suspect that it's the kind of thing that many people aren't likely to trust because of how critical it is. Nobody would expect/trust a single person to do this job :P Working in an open source project would be best. So, I don't know how good an idea it is at this point for someone to write an implementation of SSL or TLS in D. Certainly, it's the type of thing where we've generally tried to wrap existing C libraries in order to avoid having to spend the time, effort, and expertise on in order to fully implement it ourselves. The Go guys did it, but if I understand correctly, the fellow that did it was one of the OpenSSL developers, so presumably he's already very familiar with all of the ins and outs of SSL, and I don't know if any of us here are (I'm certainly not - if I were doing it, I'd pretty much just have to go off of its associated RFCs, for better or worse). I wasn't aware of this effort, interesting experiment. At this point though, if I were looking at using an existing implementation of SSL, I'd probably be looking at GnuTLS rather than OpenSSL given how horrible OpenSSL's codebase is. I don't know that GnuTLS is any better, but it wouldn't be hard for it to be. OpenSSL is horrible both with regards to its implementation and its API, and we'd all be better off if something better replaced it (be it GnuTLS or something else). I would pick LibreSSL: http://www.libressl.org/ And merge patches as their progress goes on. Unfortunately, even if something better _were_ written in D, it's probably only the D folks who would benefit, since it's not terribly likely at this point that very many folks are going to wrap a D library in order to use it another language. Here I don't completely agree: if we can have a binary-compatible implementation d
Re: Scenario: OpenSSL in D language, pros/cons
On 5/4/2014 9:29 AM, Meta wrote: While D is a somewhat safer language by *default*, it makes it fairly easy to escape from the safe part of the language and write unsafe code Yea, I'm finding that in some ways, D accidentally encourages @system/@trusted code. For example, if you need some sensitive data zeroed out when done, and for whatever reason you aren't able to just have it live on the stack (to use RAII), then you need RefCounted which AFAICS throws any chance of being @safe out the window. Maybe RefCounted could somehow take advantage of Unique to provide @safe-ty?
Re: Scenario: OpenSSL in D language, pros/cons
On Sunday, 4 May 2014 at 08:34:20 UTC, Daniele M. wrote: I have read this excellent article by David A. Wheeler: http://www.dwheeler.com/essays/heartbleed.html And since D language was not there, I mentioned it to him as a possible good candidate due to its static typing and related features. However, now I am asking the community here: would a D implementation (with GC disabled) of OpenSSL have been free from Heartbleed-type vulnerabilities? Specifically http://cwe.mitre.org/data/definitions/126.html and http://cwe.mitre.org/data/definitions/20.html as David mentions. I find this perspective very interesting, please advise :) While D is a somewhat safer language by *default*, it makes it fairly easy to escape from the safe part of the language and write unsafe code (array bounds checking can be turned off even for @safe code). Seeing as the OpenSSL devs went as far as to write an a buggy, custom implementation of malloc for a speed gain, turning off array bounds checking and ignoring @safe seems like the first thing they would do. The only language I would really trust is one in which it is impossible to write unsafe code, because you can then know that the developers can't use such unsafe hacks, even if they wanted to.
Re: Scenario: OpenSSL in D language, pros/cons
On Sun, 04 May 2014 08:34:19 + "Daniele M. via Digitalmars-d" wrote: > I have read this excellent article by David A. Wheeler: > > http://www.dwheeler.com/essays/heartbleed.html > > And since D language was not there, I mentioned it to him as a > possible good candidate due to its static typing and related > features. > > However, now I am asking the community here: would a D > implementation (with GC disabled) of OpenSSL have been free from > Heartbleed-type vulnerabilities? Specifically > http://cwe.mitre.org/data/definitions/126.html and > http://cwe.mitre.org/data/definitions/20.html as David mentions. > > I find this perspective very interesting, please advise :) @safe code protects against indexing an array out-of-bounds. So, if OpenSSL had been implemented in D, and its heartbeat feature used @safe code, then heartbleed would not have been possible. As soon as an attempt was made to index passed the end of the array, it would have thrown a RangeError and killed the program. Now, even if OpenSSL had been implemented in D, if it had used @system or @trusted code for its heartbeat feature, then it could have had the bug just as easily in D as it did in C. And given all of the horrible practices in OpenSSL, I very much doubt that having it written in D would have prevented much, because anyone making the choices that the OpenSSL folks have been making would likely have ended up with horrible D code which was mostly @system and probably doing all kinds of things that are inherently risky, forgoeing many of the benefits that D provides. I think that it's safe to say that D makes it easier to catch problems like this, but it doesn't entirely prevent them, and bad programming practices can pretty much always get around protections that the language provides unless the language provides no ways around those protections (which isn't the case in D, because it's a systems language and needs to provide low-level access and features for those programs that need them - it just doesn't use those by default). If I had more time, I'd actually be tempted to write an SSL implementation in D, but even if I were to do an excellent job of it, it would still need to be vetted by security experts to make sure that it didn't have horrible security bugs in it (much as it would be likely that there would be fewer thanks to the fact that it would be writen in D), and I suspect that it's the kind of thing that many people aren't likely to trust because of how critical it is. So, I don't know how good an idea it is at this point for someone to write an implementation of SSL or TLS in D. Certainly, it's the type of thing where we've generally tried to wrap existing C libraries in order to avoid having to spend the time, effort, and expertise on in order to fully implement it ourselves. The Go guys did it, but if I understand correctly, the fellow that did it was one of the OpenSSL developers, so presumably he's already very familiar with all of the ins and outs of SSL, and I don't know if any of us here are (I'm certainly not - if I were doing it, I'd pretty much just have to go off of its associated RFCs, for better or worse). At this point though, if I were looking at using an existing implementation of SSL, I'd probably be looking at GnuTLS rather than OpenSSL given how horrible OpenSSL's codebase is. I don't know that GnuTLS is any better, but it wouldn't be hard for it to be. OpenSSL is horrible both with regards to its implementation and its API, and we'd all be better off if something better replaced it (be it GnuTLS or something else). Unfortunately, even if something better _were_ written in D, it's probably only the D folks who would benefit, since it's not terribly likely at this point that very many folks are going to wrap a D library in order to use it another language. However, regardless of whether we ever end up with an SSL implementation in D, I think that in the long run, D will show itself to be much, much better than C or C++ at writing code that has a low number of security bugs. - Jonathan M Davis
Scenario: OpenSSL in D language, pros/cons
I have read this excellent article by David A. Wheeler: http://www.dwheeler.com/essays/heartbleed.html And since D language was not there, I mentioned it to him as a possible good candidate due to its static typing and related features. However, now I am asking the community here: would a D implementation (with GC disabled) of OpenSSL have been free from Heartbleed-type vulnerabilities? Specifically http://cwe.mitre.org/data/definitions/126.html and http://cwe.mitre.org/data/definitions/20.html as David mentions. I find this perspective very interesting, please advise :)
