Re: Carmack about static analysis
I find that dynamic typing is useful sometimes and static typing other
times. Certainly dynamic (duck) typing is useful for scripts and
similar, and static typing is certainly better for writing code that
is especially picky about its values. I liken it to the way I write
code in PHP, the front-end stuff is written to cover 90% of the use
cases, and I hope that it doesn't fail (it shouldn't unless they do
try to break the site), but if it does, I hear about it because people
are going to submit tickets about it. For any backend code, crons for
example, I always write super-tight double and triple-checked code
because there's not going to be a pissed customer immediately on my
ass if something fails.
I find that given the choice, I'd prefer static typing, since it
matches the way I reason about code, This function takes two integers
and a string and returns a string, but being weighed down by types
for small programs is annoying.
On 11 February 2012 13:59, bearophile bearophileh...@lycos.com wrote:
Jonathan M Davis:
I just can't stand the idea that whether an if statement is true or not could
change the type of a variable (e.g. it's set to a string in one branch and an
int in the other).
You have found something that sometimes I like to do in Python, that I can't
do in D, a reduced example:
import std.stdio;
int findSolution() {
return -2;
}
void main() {
int x = findSolution();
writeln(x = 0 ? x : Solution not found.);
}
There are of course ways to perform to do the same in D too, but for me this
is the most natural way to express that intent.
I consider dynamic typing to be a _huge_ negative. It may
be fine for short scripts and the like, but I'll never write anything serious
using a dynamically typed language if I can avoid it.
If you try to add static typing to something like the Mathematica Language
(that is a Lisp variant) you produce something that's probably unusable for
those exploratory programming / mathematical purposes.
Racket has recently added a Typed Scheme, it uses gradual typing. You are
able to add types to some functions of your program, and to use very specific
types or more general ones as type constraints. They have even added a way to
perform dynamic typing in C# (and it's way more than just having a variant
type, they have had to change many things to implement it). I think the
programming world is moving a bit toward a mix of static and dynamic typing
(and a mix of functional and imperative/OOP programming).
Bye,
bearophile
Re: Carmack about static analysis
Le 10/02/2012 05:37, Walter Bright a écrit : On 2/9/2012 12:09 PM, Bruno Medeiros wrote: Nice article! I particularly liked this comment: The classic hacker disdain for “bondage and discipline languages” is short sighted – the needs of large, long-lived, multi-programmer projects are just different than the quick work you do for yourself. I implicitly agree with you. But people have written large programs in dynamic languages, and claim it works out equivalently for them. I don't have enough experience in that direction to decide if that's baloney or not. Well I did that. The language doesn't replace programmer discipline and skills. Or, to say thing another way : the programmer is more important than the language. But it is clear that successful code in non typed languages ends up beeign typed anyway, just based on discipline, in most of the code. Typeless is great when sketching some piece of code, but you'll way more problem at the end.
Re: Carmack about static analysis
On 2/10/2012 3:10 AM, deadalnix wrote: Typeless is great when sketching some piece of code, but you'll way more problem at the end. I've heard people say that typeless is just as good, because you load them up with unit tests that verify the types. To me, this doesn't seem like any advantage. I'd rather have the language automatically check things for me, rather than worrying about having complete unit test coverage, let alone the bother of writing them.
Re: Carmack about static analysis
On Fri, Feb 10, 2012 at 10:19:11AM -0800, Walter Bright wrote: On 2/10/2012 3:10 AM, deadalnix wrote: Typeless is great when sketching some piece of code, but you'll way more problem at the end. I've heard people say that typeless is just as good, because you load them up with unit tests that verify the types. To me, this doesn't seem like any advantage. I'd rather have the language automatically check things for me, rather than worrying about having complete unit test coverage, let alone the bother of writing them. Though if it were D, it'd be so easy to write unittests that this wouldn't be a problem. But then D is strongly-typed anyway. :) T -- Some days you win; most days you lose.
Re: Carmack about static analysis
On Fri, Feb 10, 2012 at 11:19 AM, Walter Bright newshou...@digitalmars.comwrote: On 2/10/2012 3:10 AM, deadalnix wrote: Typeless is great when sketching some piece of code, but you'll way more problem at the end. I've heard people say that typeless is just as good, because you load them up with unit tests that verify the types. To me, this doesn't seem like any advantage. I'd rather have the language automatically check things for me, rather than worrying about having complete unit test coverage, let alone the bother of writing them. I actually read an article recently from someone who had written large applications in dynamic languages and had come to the conclusion that the productivity gains you have with the dynamic typing are pretty much lost to the additional unit testing you must do to ensure everything works. I've always had an uneasy feeling when working in dynamic languages but chalked it up to my own inexperience. Regards, Brad Anderson
Re: Carmack about static analysis
On 09/02/2012 22:34, bearophile wrote: Comparing languages with small code snippets doesn't tell you all you want to know about how a language scales for very large programs, of course, so they aren't enough. But such small snippets are very useful any way because large programs are mostly made of small parts; and it's still true that being able to remove one line from a group of 4 lines sometimes means reducing the size of a large program by 10% or more. So little syntax niceties matter even for huge programs. This is also why (as example) Python list comps are very useful for programs one million lines of code long too. Yeah, I'm not saying comparing small snippets is not useful... Clear it is, I mean, it's not practical to write a full application just to compare languages, obviously. Rather it's the mindset when examining such snippets that is important: one should be thinking of the impact the language has not just on code like the snippet, but on a medium-large sized code as well (and code written by multiple people). And I've seen several sites and discussions where such a mindset is completely ignored... (like people dissing the fact that in Java you can't write a function outside a class, you have to use a static method, and thus have a class wrapping it around...) -- Bruno Medeiros - Software Engineer
Re: Carmack about static analysis
On Friday, February 10, 2012 12:08:19 Brad Anderson wrote: I actually read an article recently from someone who had written large applications in dynamic languages and had come to the conclusion that the productivity gains you have with the dynamic typing are pretty much lost to the additional unit testing you must do to ensure everything works. I've always had an uneasy feeling when working in dynamic languages but chalked it up to my own inexperience. I just can't stand the idea that whether an if statement is true or not could change the type of a variable (e.g. it's set to a string in one branch and an int in the other). I consider dynamic typing to be a _huge_ negative. It may be fine for short scripts and the like, but I'll never write anything serious using a dynamically typed language if I can avoid it. - Jonathan M Davis
Re: Carmack about static analysis
On 2012-02-10 20:08, Brad Anderson wrote: On Fri, Feb 10, 2012 at 11:19 AM, Walter Bright newshou...@digitalmars.com mailto:newshou...@digitalmars.com wrote: On 2/10/2012 3:10 AM, deadalnix wrote: Typeless is great when sketching some piece of code, but you'll way more problem at the end. I've heard people say that typeless is just as good, because you load them up with unit tests that verify the types. To me, this doesn't seem like any advantage. I'd rather have the language automatically check things for me, rather than worrying about having complete unit test coverage, let alone the bother of writing them. I actually read an article recently from someone who had written large applications in dynamic languages and had come to the conclusion that the productivity gains you have with the dynamic typing are pretty much lost to the additional unit testing you must do to ensure everything works. I've always had an uneasy feeling when working in dynamic languages but chalked it up to my own inexperience. Regards, Brad Anderson I completely agree. I've many times wanted to have static typing in Ruby and JavaScript. -- /Jacob Carlborg
Re: Carmack about static analysis
Jonathan M Davis:
I just can't stand the idea that whether an if statement is true or not could
change the type of a variable (e.g. it's set to a string in one branch and an
int in the other).
You have found something that sometimes I like to do in Python, that I can't do
in D, a reduced example:
import std.stdio;
int findSolution() {
return -2;
}
void main() {
int x = findSolution();
writeln(x = 0 ? x : Solution not found.);
}
There are of course ways to perform to do the same in D too, but for me this is
the most natural way to express that intent.
I consider dynamic typing to be a _huge_ negative. It may
be fine for short scripts and the like, but I'll never write anything serious
using a dynamically typed language if I can avoid it.
If you try to add static typing to something like the Mathematica Language
(that is a Lisp variant) you produce something that's probably unusable for
those exploratory programming / mathematical purposes.
Racket has recently added a Typed Scheme, it uses gradual typing. You are able
to add types to some functions of your program, and to use very specific types
or more general ones as type constraints. They have even added a way to perform
dynamic typing in C# (and it's way more than just having a variant type, they
have had to change many things to implement it). I think the programming world
is moving a bit toward a mix of static and dynamic typing (and a mix of
functional and imperative/OOP programming).
Bye,
bearophile
Re: Carmack about static analysis
On 24/12/2011 12:42, bearophile wrote: A new blog post by the very good John Carmack, I like how well readable this post is: http://altdevblogaday.com/2011/12/24/static-code-analysis/ Nice article! I particularly liked this comment: The classic hacker disdain for “bondage and discipline languages” is short sighted – the needs of large, long-lived, multi-programmer projects are just different than the quick work you do for yourself. that throws a jab at a lot of the obsession with dynamic languages that goes on out there. It's something I've echoed in the past, especially when people start comparing languages to one another using small code snippets, and picking on issues/advantages that actually are only significant when writing small sized code, but not at all for medium/large sized apps. (the Hello-World Snippet Fallacy?) -- Bruno Medeiros - Software Engineer
Re: Carmack about static analysis
On 24/12/2011 23:27, Adam D. Ruppe wrote:
On Saturday, 24 December 2011 at 23:12:27 UTC, bearophile wrote:
I was talking about the abundance of (({}){()}) and not about
identifiers length.
It's the same fallacy. I can't read Carmack's mind, but
I'm sure he's talking about shortening code the same way
I would mean it if I said it - simpler concepts, fewer cases,
less repetition.
It's about how much you have to think about, now how much you
have to read/write.
Exactly.
Reminds me of the issues that some people have with Java
closures/lambdas. You do closures/lambdas in Java by means of creating
an anonymous class and then implementing a method, and this is much more
verbose than a plain lambda syntax, like D has (or many other functional
languages). But although it is much more verbose, it is actually not
that much more complex, it doesn't add much more to think about.
There is however another issue with Java's closures/lambdas which is not
complained or mentioned as much as the above, but is actually much more
annoying because it adds more semantic complexity: the closure can't
modify (the immediate/head value of) the outer variables. So when
modifying is necessary, you often see code where people instantiate a
one element array, the closure accesses the array (the array reference
is not modified), it then modifies the element inside, in index 0, and
then that is retrieved by the outer code when the closure finishes. This
on the other is a much more serious issue, and adds more complexity to
the code beyond just verbosity.
--
Bruno Medeiros - Software Engineer
Re: Carmack about static analysis
Bruno Medeiros: the needs of large, long-lived, multi-programmer projects are just different than the quick work you do for yourself. that throws a jab at a lot of the obsession with dynamic languages that goes on out there. It's something I've echoed in the past, especially when people start comparing languages to one another using small code snippets, and picking on issues/advantages that actually are only significant when writing small sized code, but not at all for medium/large sized apps. (the Hello-World Snippet Fallacy?) Many programs are small or very small, and they will keep being small. I write many of those. So there are many situations where paying for a lot of infrastructure and bondage in your code isn't good. (Haskell programmers sometimes don't agree with this idea, but the world of programming is large enough to allow two persons with very different opinions to be both acceptably right, both are able to find a way to write code in a good enough way). From my experience dynamic languages as Python are very good (often better than D, unless the problem being explored requires a large amount of computations) for exploratory programming. Usual such explorations are done on short programs, so this is also a special case of the precedent point. Comparing languages with small code snippets doesn't tell you all you want to know about how a language scales for very large programs, of course, so they aren't enough. But such small snippets are very useful any way because large programs are mostly made of small parts; and it's still true that being able to remove one line from a group of 4 lines sometimes means reducing the size of a large program by 10% or more. So little syntax niceties matter even for huge programs. This is also why (as example) Python list comps are very useful for programs one million lines of code long too. Bye, bearophile
Re: Carmack about static analysis
On 2/9/2012 12:09 PM, Bruno Medeiros wrote: Nice article! I particularly liked this comment: The classic hacker disdain for “bondage and discipline languages” is short sighted – the needs of large, long-lived, multi-programmer projects are just different than the quick work you do for yourself. I implicitly agree with you. But people have written large programs in dynamic languages, and claim it works out equivalently for them. I don't have enough experience in that direction to decide if that's baloney or not.
Re: Carmack about static analysis
Le 27/12/2011 02:07, Andrei Alexandrescu a écrit : On 12/26/11 6:36 PM, Walter Bright wrote: On 12/26/2011 9:03 AM, Andrei Alexandrescu wrote: On 12/26/11 10:29 AM, deadalnix wrote: Do you know what is the impact of 64bits on false positive proportion when the GC isn't precise ? No. I would think it would dramatically reduce it. At the same time, false positives are not the only reason for which precision is good. Precision reduces the amount of work done by the scanner, allows e.g. using distinct regions for objects with/without embedded pointers, and allows moving objects. Andrei Using different region for object that or may not contains pointers is something that can be achieved with the current state of D. Agreed for everythign else.
Re: Carmack about static analysis
On 12/25/2011 10:23 PM, Andrei Alexandrescu wrote: As a first step, we must make all allocations except stack type-aware, and leave only the stack to be imprecise. Couldn't the GC'ing the stack be handled in a similar style to how the Windows x64 ABI functions with respect to exception handling? The unwinding code lives outside the normal program flow to reduce runtime overhead as much as posisble, at least until an exception is thrown. At which point the exception handlers traverse this data structure to unwind the stack from wherever you are in your function (and its caller and so on). I would imagine a GC system could do something very similar. There are a few MSDN blogs with a bunch of useful links out to more detailed information: http://blogs.msdn.com/b/freik/archive/2006/01/04/509372.aspx http://blogs.msdn.com/b/freik/archive/2005/03/17/398200.aspx http://msdn.microsoft.com/en-us/library/1eyas8tf.aspx
Re: Carmack about static analysis
On 2011-12-26 21:55, Walter Bright wrote: On 12/26/2011 3:47 AM, Jacob Carlborg wrote: It's like when you see code like this (Ruby): unless !foo end Double negation. Code like the snippet above can be really annoying. In general, variables and conditions should never be labeled with negation: bool notFeature; I agree. -- /Jacob Carlborg
Re: Carmack about static analysis
On 2011-12-27 00:23, Robert Jacques wrote: On Mon, 26 Dec 2011 07:43:46 -0800, Jacob Carlborg d...@me.com wrote: On 2011-12-26 15:52, Andrei Alexandrescu wrote: On 12/26/11 6:53 AM, Jacob Carlborg wrote: What about the parallel/concurrent collector that has been linked a couple of times in this newsgroup. Would that be possible and a good idea to implement? A parallel collector would be, of course, a valuable addition. Andrei This is the collector I was referring to: http://www.artima.com/lejava/articles/azul_pauseless_gc.html The C4 pause-less GC, while very cool, is its own operating system. It doesn't run on-top of Windows, Linux, Mac, etc. CDGC, on the other hand, is a concurrent collector that only runs on *nixes. Thread-local GCs are a form of mostly parallel/concurrent collectors which are a good match for D, but requires our memory model to support/include regions. There has also been some work recently in making fully precise collectors for C, which would be directly applicable to D. That's why I said if possible to implement. -- /Jacob Carlborg
Re: Carmack about static analysis
On Tue, 27 Dec 2011 10:35:02 -0800, Sean Cavanaugh worksonmymach...@gmail.com wrote: On 12/25/2011 10:23 PM, Andrei Alexandrescu wrote: As a first step, we must make all allocations except stack type-aware, and leave only the stack to be imprecise. Couldn't the GC'ing the stack be handled in a similar style to how the Windows x64 ABI functions with respect to exception handling? The unwinding code lives outside the normal program flow to reduce runtime overhead as much as posisble, at least until an exception is thrown. At which point the exception handlers traverse this data structure to unwind the stack from wherever you are in your function (and its caller and so on). I would imagine a GC system could do something very similar. There are a few MSDN blogs with a bunch of useful links out to more detailed information: http://blogs.msdn.com/b/freik/archive/2006/01/04/509372.aspx http://blogs.msdn.com/b/freik/archive/2005/03/17/398200.aspx http://msdn.microsoft.com/en-us/library/1eyas8tf.aspx Fully precise collectors for C have been done; e.g. MAGPIE: PRECISE GARBAGE COLLECTION FOR C
Re: Carmack about static analysis
On 24.12.2011 16:33, Andrei Alexandrescu wrote:
On 12/24/2011 09:11 AM, Derek wrote:
On Sat, 24 Dec 2011 23:42:41 +1100, bearophile
bearophileh...@lycos.com wrote:
In theory functional-style is a good to shrink the code, but in D
functional-style code is a jungle of (({}){()}) so it's hard to write
and hard to read.
As I mentioned once, the proposition of writing functional code in D is
quite a bit different from other languages. In languages dedicated to
pure or almost pure functional semantics, there may literally be no
other way to achieve anything except in functional style; if there is,
the style is almost invariably stilted.
In D, functional style code must compete with a well honed imperative
paradigm. And as unpleasant as that may be, sometimes code that uses
mutation may be better than functional code by some metrics.
I once, long ago, suggested to Walter/Andrei that D is approaching the
ASCII equivalent of APL. The actual text of D source code can be a
hindrance to reading the code, let alone understanding what has been
written. I know it is hyperbole, but some D source modules are close to
write-only code, meaning that it takes specialist/expert D knowledge to
understand some source code. I'm not so certain that this is a desirable
feature of any programming language.
We recently discussed this a bit in connection with the standard
library. It definitely could be written in a manner that is easier to
understand for the more casual reader, but that would mean reducing the
capabilities (e.g. no reduce with multiple parameters) and making it
slower.
That being said, I personally find contemporary application D code very
readable. It's a pleasure to get to it after using C++ at work.
Anyhow, is there anything you have in mind that we have the chance of
improving at this point?
Improved syntax for compile-time reflection. is() expressions,
especially is(typeof()), and __traits, are the least readable parts of
the language. They inevitably lead to a mass of braces and parentheses.
Re: Carmack about static analysis
On 2011-12-26 03:31, Adam D. Ruppe wrote: On Monday, 26 December 2011 at 00:40:47 UTC, Andrei Alexandrescu wrote: I understand. Personally, I am unable to bring myself to fathom why writing this has any advantage: if(condition) return false just avoids a negation in the condition. It's easier to miss a ! entirely than to miss a return false, and personally, I've been finding it takes an extra brain cycle to invert a condition anyway. I like an individual line of code to be dumb as a rock anyway. It's like when you see code like this (Ruby): unless !foo end Double negation. Code like the snippet above can be really annoying. -- /Jacob Carlborg
Re: Carmack about static analysis
Le 26/12/2011 05:23, Andrei Alexandrescu a écrit : On 12/25/11 8:56 PM, deadalnix wrote: Le 25/12/2011 05:27, Andrei Alexandrescu a écrit : Got the GC book this morning, already read 2.5 chapters :o). Andrei You'll find that you can do a lot when GC is aware of immutability (in functionnal languages, most things tends to be immutable). In the actual state of druntime, the GC is bad, but worse, cannot be imporoved to take advantage of this, and never will unless a major changement is done in GC interface. What changes do you have in mind? I think GC should be aware of immutable/shared on type qualifier. Thread lcoal can be collected in a given thread without caring about others. immutable can be used as great advantage to collect concurrently. I've gone through a larger portion of the book and one thing has become very clear to me: we must improve the precision of the collector. This would be a gating feature towards taking the GC pretty much anywhere. As a first step, we must make all allocations except stack type-aware, and leave only the stack to be imprecise. I dpo agree. However, this will generate a lot of code bloat. This seems to be an issue for many people (see topic on static this and executable size for more on that). This tell us that we need a way to choose between GC. (just like JVM does). For swap friendlyness, we can separate memory blocks that may contains pointer and the ones that cannot. If something is swapped, the GC soesnt need to go throuw it and generate a huge amount of (slow as hell) IO. For the same reason, we may want to use different memory blocks for TLS of each thread. If a thread is sleeping and swapped you don't want an working thread to prevent this memory to go on swap. The exemple or Chrome browser on a notebook is very explicit on that point : it has a huge memory footprint and will swap almost inevitably on thoses machines. However, it keep being fast because each tab has its own memory pool and a tab that hasn't the focus can be swapped almost completely. Firefox has a smaller memory fooprint, however, users complains more about its memory consuption. Because they are not aware of want is really different. Firefox consume less memory, but is way less cache friendly and slow down dramatically when it has to swap. This is a big topic, and we may want to start a topic on that point specifically instead of hijacking this thread.
Re: Carmack about static analysis
On 2011-12-26 05:23, Andrei Alexandrescu wrote: On 12/25/11 8:56 PM, deadalnix wrote: Le 25/12/2011 05:27, Andrei Alexandrescu a écrit : Got the GC book this morning, already read 2.5 chapters :o). Andrei You'll find that you can do a lot when GC is aware of immutability (in functionnal languages, most things tends to be immutable). In the actual state of druntime, the GC is bad, but worse, cannot be imporoved to take advantage of this, and never will unless a major changement is done in GC interface. What changes do you have in mind? I've gone through a larger portion of the book and one thing has become very clear to me: we must improve the precision of the collector. This would be a gating feature towards taking the GC pretty much anywhere. As a first step, we must make all allocations except stack type-aware, and leave only the stack to be imprecise. A tactical detail of the above has become equally clear to me - we MUST use D's reflection and introspection capabilities in increasing the precision of the collector. It's fairly easy to write a generic function mark() (as in the mark stage of the mark/sweep collector) that accepts any object type with indirections (class, array, associative array, pointer to struct). The function marks the pointer and then iterates through all fields of the object and recurses to itself to mark other reference/pointer/etc fields of the object. This one generic function obviates a lot of the code necessary today for conservatively marking pointers, and also reduces and simplifies enormously the new code that would otherwise be needed in order to define precise collection. Pointers to instances of that generic functions would be accessible from each allocated block; to transitively mark the block, the function is called via pointer. Essentially instances of that function (which I estimate to a 50-liner) would take care of mark() for the entire universe of D types - a humongous success of compile-time reflection. Andrei What about the parallel/concurrent collector that has been linked a couple of times in this newsgroup. Would that be possible and a good idea to implement? -- /Jacob Carlborg
Re: Carmack about static analysis
On 12/26/11 6:53 AM, Jacob Carlborg wrote: What about the parallel/concurrent collector that has been linked a couple of times in this newsgroup. Would that be possible and a good idea to implement? A parallel collector would be, of course, a valuable addition. Andrei
Re: Carmack about static analysis
On 2011-12-26 15:52, Andrei Alexandrescu wrote: On 12/26/11 6:53 AM, Jacob Carlborg wrote: What about the parallel/concurrent collector that has been linked a couple of times in this newsgroup. Would that be possible and a good idea to implement? A parallel collector would be, of course, a valuable addition. Andrei This is the collector I was referring to: http://www.artima.com/lejava/articles/azul_pauseless_gc.html -- /Jacob Carlborg
Re: Carmack about static analysis
Le 26/12/2011 05:23, Andrei Alexandrescu a écrit : I've gone through a larger portion of the book and one thing has become very clear to me: we must improve the precision of the collector. This would be a gating feature towards taking the GC pretty much anywhere. Do you know what is the impact of 64bits on false positive proportion when the GC isn't precise ?
Re: Carmack about static analysis
On Monday, 26 December 2011 at 00:40:47 UTC, Andrei Alexandrescu wrote: I might be biased, but I ask for it to be corrected if I see it during a code review at work, and if someones writes such during an interview, it doesn't gain them any points. To conclude our chat here too, I do think you have a legitimate point.
Re: Carmack about static analysis
On 12/26/11 10:29 AM, deadalnix wrote: Le 26/12/2011 05:23, Andrei Alexandrescu a écrit : I've gone through a larger portion of the book and one thing has become very clear to me: we must improve the precision of the collector. This would be a gating feature towards taking the GC pretty much anywhere. Do you know what is the impact of 64bits on false positive proportion when the GC isn't precise ? No. Andrei
Re: Carmack about static analysis
On 12/26/2011 3:47 AM, Jacob Carlborg wrote: It's like when you see code like this (Ruby): unless !foo end Double negation. Code like the snippet above can be really annoying. In general, variables and conditions should never be labeled with negation: bool notFeature;
Re: Carmack about static analysis
On Mon, 26 Dec 2011 07:43:46 -0800, Jacob Carlborg d...@me.com wrote: On 2011-12-26 15:52, Andrei Alexandrescu wrote: On 12/26/11 6:53 AM, Jacob Carlborg wrote: What about the parallel/concurrent collector that has been linked a couple of times in this newsgroup. Would that be possible and a good idea to implement? A parallel collector would be, of course, a valuable addition. Andrei This is the collector I was referring to: http://www.artima.com/lejava/articles/azul_pauseless_gc.html The C4 pause-less GC, while very cool, is its own operating system. It doesn't run on-top of Windows, Linux, Mac, etc. CDGC, on the other hand, is a concurrent collector that only runs on *nixes. Thread-local GCs are a form of mostly parallel/concurrent collectors which are a good match for D, but requires our memory model to support/include regions. There has also been some work recently in making fully precise collectors for C, which would be directly applicable to D.
Re: Carmack about static analysis
Robert Jacques: The C4 pause-less GC, while very cool, is its own operating system. It doesn't run on-top of Windows, Linux, Mac, etc. I think that eventually Linux will have to include some of those ideas, to allow language designers or virtual machine designers, to implement better garbage collectors. At the moment the interaction between GCs and virtual memory managers of the OS is often naive or bad. In the meantime a serious D programmer that has to process lot of data efficiently is possibly willing to use a patched Linux to use a C4-like GC from D code. But I think such garbage collectors are a lot of work to implement, so I think for now it's better to aim to something simpler for D2, that requires a smaller number of human-years of programming to be built :-) Bye, bearophile
Re: Carmack about static analysis
On 12/26/2011 9:03 AM, Andrei Alexandrescu wrote: On 12/26/11 10:29 AM, deadalnix wrote: Do you know what is the impact of 64bits on false positive proportion when the GC isn't precise ? No. I would think it would dramatically reduce it.
Re: Carmack about static analysis
On 12/26/11 6:36 PM, Walter Bright wrote: On 12/26/2011 9:03 AM, Andrei Alexandrescu wrote: On 12/26/11 10:29 AM, deadalnix wrote: Do you know what is the impact of 64bits on false positive proportion when the GC isn't precise ? No. I would think it would dramatically reduce it. At the same time, false positives are not the only reason for which precision is good. Precision reduces the amount of work done by the scanner, allows e.g. using distinct regions for objects with/without embedded pointers, and allows moving objects. Andrei
Re: Carmack about static analysis
Walter Bright wrote: bool notFeature; Counterexample: | bool sense; I do not believe, that a majority of developers evaluates `nonsense' as being equal to `!sense'. -manfred
Re: Carmack about static analysis
Universal Function Call Syntax. C# calls them extension methods. Basically, the ability to call a method directly on a class (in the form of A.MyMethod) without it being declared in the class. Usually just a static/global method that gets rewritten. Instead of MyMethod(A), you can use A.MyMethod. On 25/12/2011 1:51 AM, Chad J wrote: On 12/24/2011 10:37 PM, Kapps wrote: The two biggest things that I believe D would benefit from, in terms of readability, are UFCS and C#-style lambdas (which were already discussed). Something like 'array(find(test, select!(a.name)(filter!(a.num 3)(Elements' Could be rewritten much cleaner as Elements.filter!a.num 3 .select!a.name .find(test) .array(); The first one is very confusing, hard to split across multiple lines without losing meaning, and difficult to see what it even operates on (assume Elements is not an array and thus can not be used with the current implementation of UFCS). The second one, you can clearly see operates on Elements, you can clearly split it across multiple lines, and it's very obvious where things stop and end. Plus, it eliminates at least one set of parenthesis per function call, not to mention being much easier to write as it's in order of steps instead of opposite order. Pardon, what does UFCS stand for?
Re: Carmack about static analysis
On 12/25/2011 02:57 AM, Kapps wrote: Universal Function Call Syntax. C# calls them extension methods. Basically, the ability to call a method directly on a class (in the form of A.MyMethod) without it being declared in the class. Usually just a static/global method that gets rewritten. Instead of MyMethod(A), you can use A.MyMethod. On 25/12/2011 1:51 AM, Chad J wrote: On 12/24/2011 10:37 PM, Kapps wrote: The two biggest things that I believe D would benefit from, in terms of readability, are UFCS and C#-style lambdas (which were already discussed). Something like 'array(find(test, select!(a.name)(filter!(a.num 3)(Elements' Could be rewritten much cleaner as Elements.filter!a.num 3 .select!a.name .find(test) .array(); The first one is very confusing, hard to split across multiple lines without losing meaning, and difficult to see what it even operates on (assume Elements is not an array and thus can not be used with the current implementation of UFCS). The second one, you can clearly see operates on Elements, you can clearly split it across multiple lines, and it's very obvious where things stop and end. Plus, it eliminates at least one set of parenthesis per function call, not to mention being much easier to write as it's in order of steps instead of opposite order. Pardon, what does UFCS stand for? Ah good. I always thought it a bit odd that we didn't have that. Better yet if it worked with operator overloading... then the operator overloads could check for null.
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 20:19:48 UTC, Andrei Alexandrescu wrote: Analogies of programming language with human language are flawed and take nowhere interesting. The only text that gets close to the level of precision and non-ambiguity needed is legal text; reading any amount of legal text blunts one's desire to be more like it. Andrei I know of one attempt for natural language programming, Hypertalk, the language of the Hypercard multimedia tool Apple developped in the 1990's. The tool was very interesting in itself, but the programming language is very limited and quite verbose due to the choice of being a natural language. I suspect it is the case of all natural languages used as programming languages. Apart maybe in artificial intelligence, I believe natural language has no useful use.
Re: Carmack about static analysis
On 25.12.2011 00:44, Timon Gehr wrote:
Not really. Functional style code tends to be conceptually simpler.
Having code that is more readable can help. Getting rid of (({return
{return}}){return()}) makes the code more readable, whereas excessively
shortening identifiers does the opposite.
See here for an example of what bearophile is talking about:
http://pastebin.com/2rEdx0RD
r=cons(st(1UL),cons(st(1UL),lz({return zipWith((Lazy!ulong a,Lazy!ulong
b){return lz({return a()+b();});},r,r().tail)();})));
Re: Carmack about static analysis
On 2011-12-25 02:20, Adam D. Ruppe wrote:
On Saturday, 24 December 2011 at 23:51:57 UTC, bearophile wrote:
Using certain abstractions sometimes helps to write one idea only once
in a program. Etc.
This is the biggest one, and applies to all kinds of code.
I like to write little functions with meaningful names.
bool isOdd(int i) {
if((i % 2) == 0)
return false;
else
return true;
}
filter!isOdd(myCollection);
I find that nicer than
filter!i % 2 != 0(myCollection);
despite it being longer.
With the former, it says very simply that it wants odd
numbers at the usage location.
With the latter, you have to think for a second about
what the modulus operator actually does and what the
definition of an odd number is before you can get to
why the filter is there at all.
It gets even worse if the other function has non-trivial
code. Best to just give it a name so you don't have to think
about the implementation at all at the usage site.
I completely agree with this. It can be used in if-statements and
similar as well.
--
/Jacob Carlborg
Re: Carmack about static analysis
On 2011-12-25 05:44, Andrei Alexandrescu wrote:
On 12/24/2011 07:20 PM, Adam D. Ruppe wrote:
On Saturday, 24 December 2011 at 23:51:57 UTC, bearophile wrote:
Using certain abstractions sometimes helps to write one idea only once
in a program. Etc.
This is the biggest one, and applies to all kinds of code.
I like to write little functions with meaningful names.
bool isOdd(int i) {
if((i % 2) == 0)
return false;
else
return true;
}
filter!isOdd(myCollection);
I find that nicer than
filter!i % 2 != 0(myCollection);
despite it being longer.
Different strokes for different folks. IMHO it would be difficult to
justify the verboseness and the unnecessary (twice) flow of control. I
see every letter beyond this as a liability:
bool isOdd(int i)
{
return (i 1) != 0;
}
The point was not how to implement isOdd, it was to use a symbol instead.
With the former, it says very simply that it wants odd
numbers at the usage location.
Giving an operation, however trivial, a symbol, is often a good thing.
There are of course limits, and each engineer has their own ideas where
to draw the line.
Andrei
--
/Jacob Carlborg
Re: Carmack about static analysis
On 12/25/2011 02:44 PM, simendsjo wrote:
On 25.12.2011 00:44, Timon Gehr wrote:
Not really. Functional style code tends to be conceptually simpler.
Having code that is more readable can help. Getting rid of (({return
{return}}){return()}) makes the code more readable, whereas excessively
shortening identifiers does the opposite.
See here for an example of what bearophile is talking about:
http://pastebin.com/2rEdx0RD
r=cons(st(1UL),cons(st(1UL),lz({return zipWith((Lazy!ulong a,Lazy!ulong
b){return lz({return a()+b();});},r,r().tail)();})));
http://pastebin.com/C6vf9DQQ
r=cons(st(cast(T)1),lz({return merge(merge(map((Lazy!T a){return
lz({return 2*a();});},r),map((Lazy!T a){return lz({return
3*a();});},r)),map((Lazy!T a){return lz({return 5*a();});},r))();}));
D'oh!
Re: Carmack about static analysis
On Sunday, 25 December 2011 at 04:44:57 UTC, Andrei Alexandrescu
wrote:
I see every letter beyond this as a liability:
Yeah, though like Jacob said, the implementation doesn't
matter as much as the idea of giving it a name.
But, if you're curious about why I wrote it long form, it's
just a quirk I sometimes do; if I read something out loud
differently than how I wrote it, I'll sometimes rewrite it.
Here, I dictated it to myself as if it's divisible by two,
return false (not odd), otherwise, return true (is odd), and
decided to write it almost exactly like that.
Usually, these little things are meaningless once compiled,
but check this out:
===
bool test(int a) {
if(a%2 == 0)
return false;
else
return true;
}
bool test2(int a) {
return a1;
}
===
$ dmd -O -c test.d # 2.057 btw
$ objdump -d test.o
Disassembly of section .text._D5test24testFiZb:
_D5test24testFiZb:
0: 55 push %ebp
1: 8b ec mov%esp,%ebp
3: 50 push %eax
4: 99 cltd
5: 33 c2 xor%edx,%eax
7: 25 01 00 00 00 and$0x1,%eax
c: 03 d0 add%eax,%edx
e: 83 fa 01cmp$0x1,%edx
11: 19 c0 sbb%eax,%eax
13: 8b e5 mov%ebp,%esp
15: 40 inc%eax
16: 5d pop%ebp
17: c3 ret
Disassembly of section .text._D5test25test2FiZb:
_D5test25test2FiZb:
0: 55 push %ebp
1: 8b ec mov%esp,%ebp
3: 50 push %eax
4: 25 01 00 00 00 and$0x1,%eax
9: f7 d8 neg%eax
b: 19 c0 sbb%eax,%eax
d: 8b e5 mov%ebp,%esp
f: f7 d8 neg%eax
11: 5d pop%ebp
12: c3 ret
Almost the same, but not quite I think the two
functions should have compiled identically. With gcc
(compiling the code as C), it comes out:
test:
0: 55 push %ebp
1: 89 e5 mov%esp,%ebp
3: 8b 45 08mov0x8(%ebp),%eax
6: 83 e0 01and$0x1,%eax
9: 5d pop%ebp
a: c3 ret
for both functions.
(btw Akismet thinks your post looks like spam. LOL)
Re: Carmack about static analysis
On 12/25/2011 11:53 PM, Timon Gehr wrote:
On 12/25/2011 02:44 PM, simendsjo wrote:
On 25.12.2011 00:44, Timon Gehr wrote:
Not really. Functional style code tends to be conceptually simpler.
Having code that is more readable can help. Getting rid of (({return
{return}}){return()}) makes the code more readable, whereas excessively
shortening identifiers does the opposite.
See here for an example of what bearophile is talking about:
http://pastebin.com/2rEdx0RD
r=cons(st(1UL),cons(st(1UL),lz({return zipWith((Lazy!ulong a,Lazy!ulong
b){return lz({return a()+b();});},r,r().tail)();})));
http://pastebin.com/C6vf9DQQ
r=cons(st(cast(T)1),lz({return merge(merge(map((Lazy!T a){return
lz({return 2*a();});},r),map((Lazy!T a){return lz({return
3*a();});},r)),map((Lazy!T a){return lz({return 5*a();});},r))();}));
D'oh!
With UFCS and alternate delegate syntax:
r=st(cast(T)1).cons(lz(= r.map((Lazy!T a)= lz(= 2*a()))
.merge(r.map((Lazy!T a)= lz(= 3*a(
.merge(r.map((Lazy!T a)= lz(= 5*a(()
));
With delegate parameter type inference for non-template delegate type
parameters (as proposed by Andrei in a recent bug report):
r=st(cast(T)1).cons(lz(= r.map((a)= lz(= 2*a()))
.merge(r.map((a)= lz(= 3*a(
.merge(r.map((a)= lz(= 5*a(()
));
With language support for non-strict evaluation:
@nonstrict:
r=cast(T)1
.cons(
r.map((a)= 2*a)
.merge(r.map((a)= 3*a))
.merge(r.map((a)= 5*a))
);
Re: Carmack about static analysis
Timon Gehr:
http://pastebin.com/C6vf9DQQ
If possible I suggest to remove all the BigInt from the module, and put the
bigint import into the main():
void main() {
import std.bigint;
auto h = hamming!BigInt();
writeln(take(20, h));
writeln(h()[1_691]);
writeln(h()[1_000_000]);
}
@nonstrict:
r=cast(T)1
.cons(
r.map((a)= 2*a)
.merge(r.map((a)= 3*a))
.merge(r.map((a)= 5*a))
);
This version looks acceptable :-)
Bye,
bearophile
Re: Carmack about static analysis
On 12/26/2011 12:30 AM, bearophile wrote:
Timon Gehr:
http://pastebin.com/C6vf9DQQ
If possible I suggest to remove all the BigInt from the module, and put the
bigint import into the main():
void main() {
import std.bigint;
auto h = hamming!BigInt();
writeln(take(20, h));
writeln(h()[1_691]);
writeln(h()[1_000_000]);
}
That could be done, but I don't think it is terribly important.
@nonstrict:
r=cast(T)1
.cons(
r.map((a)= 2*a)
.merge(r.map((a)= 3*a))
.merge(r.map((a)= 5*a))
);
This version looks acceptable :-)
Bye,
bearophile
It would be even better with methods-as-operators:
r=cast(T)1 cons (( r map a=2*a ) merge ( r map a=3*a ) merge ( r map
a=5*a ))
Re: Carmack about static analysis
On 12/25/11 5:13 PM, Adam D. Ruppe wrote: On Sunday, 25 December 2011 at 04:44:57 UTC, Andrei Alexandrescu wrote: I see every letter beyond this as a liability: Yeah, though like Jacob said, the implementation doesn't matter as much as the idea of giving it a name. But, if you're curious about why I wrote it long form, it's just a quirk I sometimes do; if I read something out loud differently than how I wrote it, I'll sometimes rewrite it. Here, I dictated it to myself as if it's divisible by two, return false (not odd), otherwise, return true (is odd), and decided to write it almost exactly like that. I understand. Personally, I am unable to bring myself to fathom why writing this has any advantage: if (condition) return true; // or false else return false; // or true, respectively as much as I'm unable to fathom why writing this has any advantage: if (expression == true) ... or if (expression == false) ... I might be biased, but I ask for it to be corrected if I see it during a code review at work, and if someones writes such during an interview, it doesn't gain them any points. Usually, these little things are meaningless once compiled, but check this out: [snip] I'd say that's a good enhancement request for the backend. Andrei
Re: Carmack about static analysis
On Saturday, December 24, 2011 21:37:12 Kapps wrote: The two biggest things that I believe D would benefit from, in terms of readability, are UFCS and C#-style lambdas (which were already discussed). Something like 'array(find(test, select!(a.name)(filter!(a.num 3)(Elements' Could be rewritten much cleaner as Elements.filter!a.num 3 .select!a.name .find(test) .array(); The first one is very confusing, hard to split across multiple lines without losing meaning, and difficult to see what it even operates on (assume Elements is not an array and thus can not be used with the current implementation of UFCS). The second one, you can clearly see operates on Elements, you can clearly split it across multiple lines, and it's very obvious where things stop and end. Plus, it eliminates at least one set of parenthesis per function call, not to mention being much easier to write as it's in order of steps instead of opposite order. I think that that's partially a function of how much programming you've done in functional languages. I personally find the first one to be far more readable. The second one seems horribly backwards. - Jonathan M Davis
Re: Carmack about static analysis
On Monday, 26 December 2011 at 02:31:52 UTC, Adam D. Ruppe wrote: I like an individual line of code to be dumb as a rock anyway. I'm reminded of a discussion I had on another forum, where someone suggested this might be because I started with assembly language. It might be that those first few years shape your brain and old habits die hard.
Re: Carmack about static analysis
On Monday, 26 December 2011 at 00:40:47 UTC, Andrei Alexandrescu wrote: I understand. Personally, I am unable to bring myself to fathom why writing this has any advantage: if(condition) return false just avoids a negation in the condition. It's easier to miss a ! entirely than to miss a return false, and personally, I've been finding it takes an extra brain cycle to invert a condition anyway. I like an individual line of code to be dumb as a rock anyway.
Re: Carmack about static analysis
Le 25/12/2011 05:27, Andrei Alexandrescu a écrit : Got the GC book this morning, already read 2.5 chapters :o). Andrei You'll find that you can do a lot when GC is aware of immutability (in functionnal languages, most things tends to be immutable). In the actual state of druntime, the GC is bad, but worse, cannot be imporoved to take advantage of this, and never will unless a major changement is done in GC interface.
Re: Carmack about static analysis
On 12/25/11 8:56 PM, deadalnix wrote: Le 25/12/2011 05:27, Andrei Alexandrescu a écrit : Got the GC book this morning, already read 2.5 chapters :o). Andrei You'll find that you can do a lot when GC is aware of immutability (in functionnal languages, most things tends to be immutable). In the actual state of druntime, the GC is bad, but worse, cannot be imporoved to take advantage of this, and never will unless a major changement is done in GC interface. What changes do you have in mind? I've gone through a larger portion of the book and one thing has become very clear to me: we must improve the precision of the collector. This would be a gating feature towards taking the GC pretty much anywhere. As a first step, we must make all allocations except stack type-aware, and leave only the stack to be imprecise. A tactical detail of the above has become equally clear to me - we MUST use D's reflection and introspection capabilities in increasing the precision of the collector. It's fairly easy to write a generic function mark() (as in the mark stage of the mark/sweep collector) that accepts any object type with indirections (class, array, associative array, pointer to struct). The function marks the pointer and then iterates through all fields of the object and recurses to itself to mark other reference/pointer/etc fields of the object. This one generic function obviates a lot of the code necessary today for conservatively marking pointers, and also reduces and simplifies enormously the new code that would otherwise be needed in order to define precise collection. Pointers to instances of that generic functions would be accessible from each allocated block; to transitively mark the block, the function is called via pointer. Essentially instances of that function (which I estimate to a 50-liner) would take care of mark() for the entire universe of D types - a humongous success of compile-time reflection. Andrei
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 12:42:41 UTC, bearophile wrote: Printf format string errors were the second biggest issue in A problem that people didn't want to reduce in D. printf and writef are entirely different beasts.
Re: Carmack about static analysis
On Sat, 24 Dec 2011 23:42:41 +1100, bearophile bearophileh...@lycos.com
wrote:
A new blog post by the very good John Carmack, I like how well readable
this post is:
http://altdevblogaday.com/2011/12/24/static-code-analysis/
... human minds and the logic of lint programs are two cognitive (or
pre-cognitive) systems that work in very different ways. Often what's
obvious and easy for a mammalian brain can't be seen by a lint tool and
what a lint tool finds easy to spot is hard to find for a human brain.
So they are both needed. The difference in the way computers and people
think is one of reasons of the usefulness of computers for our society.
But isn't lint written by humans? I'm pretty certain 'lint' is not a
cognitive entity, but just a program written by people, who as we all
agree, can and do make mistakes. I'm under the impression that a //lint//
program is an attempt to emulate a very pedantic (if not anal-retentive)
person who in real life would have no real friends, due to their
obsessive–compulsive habit for extreme nit-picking.
So in saying that, I would have to side with John Carmack in so far as
thinking that if a near-perfect lint program cannot understand some source
code, then a regular coder would also have difficulty in groking it.
In short, it does no harm to make one's code as readable (such that your
intention is exceedingly obvious) as possible.
Here there are some bad examples about /Analyze:
http://randomascii.wordpress.com/2011/09/13/analyze-for-visual-studio-the-ugly-part-5/
In the same blog you see many other similar cases.
The article you reference is primarily saying that this specific lint-like
functionality contains bugs. It is not an argument to convince us to
abandon lint functionality.
In theory functional-style is a good to shrink the code, but in D
functional-style code is a jungle of (({}){()}) so it's hard to write
and hard to read.
I once, long ago, suggested to Walter/Andrei that D is approaching the
ASCII equivalent of APL. The actual text of D source code can be a
hindrance to reading the code, let alone understanding what has been
written. I know it is hyperbole, but some D source modules are close to
write-only code, meaning that it takes specialist/expert D knowledge to
understand some source code. I'm not so certain that this is a desirable
feature of any programming language.
--
Derek Parnell
Melbourne, Australia
Re: Carmack about static analysis
On 12/24/2011 06:42 AM, bearophile wrote: A new blog post by the very good John Carmack, I like how well readable this post is: http://altdevblogaday.com/2011/12/24/static-code-analysis/ Nice! One nice snippet that I can't quote strongly enough: Automation is necessary. It is common to take a sort of smug satisfaction in reports of colossal failures of automatic systems, but for every failure of automation, the failures of humans are legion. This is a very important tidbit. We've improved participation of automation to D's development process a fair amount, but we still have a long time to go. Anything that isn’t crystal clear to a static analysis tool probably isn’t clear to your fellow programmers, either. This I disagree with; I think it comes with the assumption that static analysis tools Carmack tried don't even try to approach certain problems. There are plenty of things that are relatively simple but extremely difficult to automatically prove correct. Andrei
Re: Carmack about static analysis
On 12/24/2011 09:11 AM, Derek wrote:
On Sat, 24 Dec 2011 23:42:41 +1100, bearophile
bearophileh...@lycos.com wrote:
In theory functional-style is a good to shrink the code, but in D
functional-style code is a jungle of (({}){()}) so it's hard to write
and hard to read.
As I mentioned once, the proposition of writing functional code in D is
quite a bit different from other languages. In languages dedicated to
pure or almost pure functional semantics, there may literally be no
other way to achieve anything except in functional style; if there is,
the style is almost invariably stilted.
In D, functional style code must compete with a well honed imperative
paradigm. And as unpleasant as that may be, sometimes code that uses
mutation may be better than functional code by some metrics.
I once, long ago, suggested to Walter/Andrei that D is approaching the
ASCII equivalent of APL. The actual text of D source code can be a
hindrance to reading the code, let alone understanding what has been
written. I know it is hyperbole, but some D source modules are close to
write-only code, meaning that it takes specialist/expert D knowledge to
understand some source code. I'm not so certain that this is a desirable
feature of any programming language.
We recently discussed this a bit in connection with the standard
library. It definitely could be written in a manner that is easier to
understand for the more casual reader, but that would mean reducing the
capabilities (e.g. no reduce with multiple parameters) and making it slower.
That being said, I personally find contemporary application D code very
readable. It's a pleasure to get to it after using C++ at work.
Anyhow, is there anything you have in mind that we have the chance of
improving at this point?
Thanks,
Andrei
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 15:33:04 UTC, Andrei
Alexandrescu wrote:
Anyhow, is there anything you have in mind that we have the
chance of improving at this point?
Thanks,
Andrei
For one, we should follow up on:
foo!(a = a * 2)(bar);
vs.
foo!((a) { return a * 2; })(bar);
Re: Carmack about static analysis
On Sun, 25 Dec 2011 02:33:01 +1100, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: We recently discussed this a bit in connection with the standard library. It definitely could be written in a manner that is easier to understand for the more casual reader, but that would mean reducing the capabilities (e.g. no reduce with multiple parameters) and making it slower. It's the syntax and not the semantics that troubles me. I'm sure that some syntax changes could be possible to make readability improvements with no loss of capabilities. ... I personally find contemporary application D code very readable. It's a pleasure to get to it after using C++ at work. I'm sure you are totally correct; I'm not really a C++ coder. And I'm sure you also process the specialist/expert level of D knowledge to make reading contemporary D code a non-issue. But when compared to spoken language text, D code can appear quite obtuse to average coders. And I believe this is main do to the very high use of non-alphabetic symbols and a level of overloading of both punctuation characters and reserved words. Anyhow, is there anything you have in mind that we have the chance of improving at this point? I have nothing concrete right now, and I suspect nothing that a C/C++/D aficionado would appreciate, so I'll remain silent about this and try harder to appreciate D syntax. There is no point in starting bike-shed/flame-wars about syntax issues. I understand Walter's viewpoints on the desire for C compatibility and on the use of reserved words. -- Derek Parnell Melbourne, Australia
Re: Carmack about static analysis
On 2011-12-24 13:42, bearophile wrote:
There was a paper recently that noted that all of the various code quality metrics
correlated at least as strongly with code size as error rate, making code size
alone give essentially the same error predicting ability. Shrink your important
code.
In theory functional-style is a good to shrink the code, but in D
functional-style code is a jungle of (({}){()}) so it's hard to write and hard
to read.
There have been suggestion for a new lambda syntax, among them the one
used by C# and Scala:
foo(a = a * 2);
--
/Jacob Carlborg
Re: Carmack about static analysis
On 12/24/2011 6:35 AM, Adam D. Ruppe wrote: On Saturday, 24 December 2011 at 12:42:41 UTC, bearophile wrote: Printf format string errors were the second biggest issue in A problem that people didn't want to reduce in D. printf and writef are entirely different beasts. D's charter is not to fix bad usage of C functions like printf and memcpy. Use the corresponding D features like writef and array copy, and you won't need static analysis on them.
Re: Carmack about static analysis
On 12/24/2011 11:30 AM, Walter Bright wrote: On 12/24/2011 6:35 AM, Adam D. Ruppe wrote: On Saturday, 24 December 2011 at 12:42:41 UTC, bearophile wrote: Printf format string errors were the second biggest issue in A problem that people didn't want to reduce in D. printf and writef are entirely different beasts. D's charter is not to fix bad usage of C functions like printf and memcpy. Use the corresponding D features like writef and array copy, and you won't need static analysis on them. You still might if you want to make sure they work without running them. Andrei
Re: Carmack about static analysis
On 12/24/11 9:45 AM, Jakob Ovrum wrote:
On Saturday, 24 December 2011 at 15:33:04 UTC, Andrei Alexandrescu wrote:
Anyhow, is there anything you have in mind that we have the chance of
improving at this point?
Thanks,
Andrei
For one, we should follow up on:
foo!(a = a * 2)(bar);
vs.
foo!((a) { return a * 2; })(bar);
OK.
Andrei
Re: Carmack about static analysis
On 12/24/11 9:54 AM, Derek wrote: I'm sure you are totally correct; I'm not really a C++ coder. And I'm sure you also process the specialist/expert level of D knowledge to make reading contemporary D code a non-issue. Well I'm also a specialist in C++, actually more so than D as I have longer experience with C++ and wrote more code in it. But when compared to spoken language text, D code can appear quite obtuse to average coders. And I believe this is main do to the very high use of non-alphabetic symbols and a level of overloading of both punctuation characters and reserved words. This issue (analogy with human language) has been a long preoccupation of me. I have ended up at an odd point - I lost interest. Analogies of programming language with human language are flawed and take nowhere interesting. The only text that gets close to the level of precision and non-ambiguity needed is legal text; reading any amount of legal text blunts one's desire to be more like it. Second, natural language text has a very different use pattern. A body of natural language text is meant to be written once and then read many times; the notion that people need to maintain, enhance, and modify a large body of natural language text is quite foreign. It follows that programming language code must optimize for different directions. Third, analogy with math is inevitable; math is the most concise means known by humankind to encode theories. A good part of mathematics is dedicated to inventing good notation, and math notations have inevitably shunned natural language. Andrei
Re: Carmack about static analysis
Andrei Alexandrescu: the notion that people need to maintain, enhance, and modify a large body of natural language text is quite foreign. http://www.wikipedia.org/ ? Bye, bearophile
Re: Carmack about static analysis
Derek Parnell: But isn't lint written by humans? Right, but this doesn't mean a lot, because most programs don't work the same way humans and human brains think. I'm under the impression that a //lint// program is an attempt to emulate a very pedantic (if not anal-retentive) person who in real life would have no real friends, due to their obsessiveâcompulsive habit for extreme nit-picking. But the mind of such person and a lint work in very different ways. So they are often able to spot different classes of problems. The article you reference is primarily saying that this specific lint-like functionality contains bugs. It is not an argument to convince us to abandon lint functionality. Both my post and that link weren't meant to abadon lints, I like the idea of lints :-) I have shown that blog post as an example of the wide difference between lints skills and human programmers skills. I once, long ago, suggested to Walter/Andrei that D is approaching the ASCII equivalent of APL. I have used the K language a bit, that is one ASCII equivalent of APL, and it's nowhere D both in semantics, readability, and conciseness :-) Writing K code is more like solving a puzzle. And reading it is like solving a different puzzle. Bye, bearophile
Re: Carmack about static analysis
On 12/24/2011 12:38 PM, bearophile wrote: Andrei Alexandrescu: the notion that people need to maintain, enhance, and modify a large body of natural language text is quite foreign. http://www.wikipedia.org/ ? Wikipedia's text often shows an effect of maintenance similar to source code - additions are often jarring and break up the flow.
Re: Carmack about static analysis
Andrei Alexandrescu: Anyhow, is there anything you have in mind that we have the chance of improving at this point? Two things that I think are able to improve the D code (beside a shorter lambda syntax) are Python Lazy/eager array comps (I'd like to write a post about this...). If this is not possible then I think introducing amap and afilter (that means array(map()) and array(filter())) in Phobos is able to remove several parentheses from my current D code. Bye, bearophile
Re: Carmack about static analysis
On 12/24/11 2:55 PM, bearophile wrote: If this is not possible then I think introducing amap and afilter (that means array(map()) and array(filter())) in Phobos is able to remove several parentheses from my current D code. I've never gotten why you wouldn't define amap and afilter for your own code, thus avoiding parentheses. Andrei
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 12:42:41 UTC, bearophile wrote:
In theory functional-style is a good to shrink the code, but in
D functional-style code is a jungle of (({}){()}) so it's hard
to write and hard to read.
I think code would have a lot fewer bugs if we forced all
identifiers
to be one character long. Then it'd be a lot shorter.
Let's enforce this in the language. Walter? Andrei?
Re: Carmack about static analysis
On Sun, 25 Dec 2011 07:19:47 +1100, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 12/24/11 9:54 AM, Derek wrote: I'm sure you are totally correct; I'm not really a C++ coder. And I'm sure you also process the specialist/expert level of D knowledge to make reading contemporary D code a non-issue. Well I'm also a specialist in C++, actually more so than D as I have longer experience with C++ and wrote more code in it. LOL ... that went without saying ... your reputation precedes you. But when compared to spoken language text, D code can appear quite obtuse to average coders. And I believe this is main do to the very high use of non-alphabetic symbols and a level of overloading of both punctuation characters and reserved words. This issue (analogy with human language) has been a long preoccupation of me. I have ended up at an odd point - I lost interest. I'm sorry that I wasn't clear enough (irony?) but I'm not advocating that programming languages resemble human languages, just that in comparison - comparing D text with English text for example - D source code can be harder to read. Probably because D relies much more on the precise use of punctuation symbols than English text does. Our latin-alphabet focused training has to take in a larger character set, and with nearly all D punctuation being single-character entities, one has to read the text more carefully than English text. I am not suggesting that D change any of this because that would turn D into something else and thus alienate most of its adherents. -- Derek Parnell Melbourne, Australia
Re: Carmack about static analysis
Adam D. Ruppe:
I think code would have a lot fewer bugs if we forced all
identifiers
to be one character long. Then it'd be a lot shorter.
I was talking about the abundance of (({}){()}) and not about identifiers
length.
When they think about code conciseness (that is quite correlated with bug
count, as Carmack too says) they refer to something more like tokens number,
instead of character count.
Bye,
bearophile
Re: Carmack about static analysis
Andrei Alexandrescu: I've never gotten why you wouldn't define amap and afilter for your own code, thus avoiding parentheses. In D1 I did write a non-standard library to be used with Phobos. In D2 I'd like to avoid writing a dlibs2 that is used only by me. There are some functions that are probably useful for lot of people. Bye, bearophile
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 23:12:27 UTC, bearophile wrote:
I was talking about the abundance of (({}){()}) and not about
identifiers length.
It's the same fallacy. I can't read Carmack's mind, but
I'm sure he's talking about shortening code the same way
I would mean it if I said it - simpler concepts, fewer cases,
less repetition.
It's about how much you have to think about, now how much you
have to read/write.
Re: Carmack about static analysis
On 12/25/2011 12:27 AM, Adam D. Ruppe wrote:
On Saturday, 24 December 2011 at 23:12:27 UTC, bearophile wrote:
I was talking about the abundance of (({}){()}) and not about
identifiers length.
It's the same fallacy.
Not really. Functional style code tends to be conceptually simpler.
Having code that is more readable can help. Getting rid of (({return
{return}}){return()}) makes the code more readable, whereas excessively
shortening identifiers does the opposite.
See here for an example of what bearophile is talking about:
http://pastebin.com/2rEdx0RD
However, I think the slow druntime GC is more of a show stopper for
functional D than any syntactic issues there may be.
I can't read Carmack's mind, but
I'm sure he's talking about shortening code the same way
I would mean it if I said it - simpler concepts, fewer cases,
less repetition.
It's about how much you have to think about, now how much you
have to read/write.
I am quite sure he is talking about character count. I still think you
are right, because for reasonable code with average identifier lengths
etc. character count correlates a good bit with what you suggest are
good measures for code length.
Re: Carmack about static analysis
Adam D. Ruppe: It's the same fallacy. I can't read Carmack's mind, but I'm sure he's talking about shortening code the same way I would mean it if I said it - simpler concepts, fewer cases, less repetition. In a medium sized program there are many different ways to reduce redundancy and repetition, at various levels. Example: modules found online are a good way to avoid writing some chunks of your program yourself. Using certain abstractions sometimes helps to write one idea only once in a program. Etc. Another significant way to remove repetition at a very different level is to use micro-patterns like higher order functions, like map and filter, and other std.algorithm functions like canFind, count, etc, plus iterators like zip, etc. If you use them you are often able to reduce the code _significantly_ and make it more explicit (because if you use std.algorithm.count the person that reads the code knows you are counting). But a problem is that such D code is full of braces and parentheses that make such code hard to read and write (I agree with Andrei that D is not very designed for such kind of code, but the fact doesn't change). Bye, bearophile
Re: Carmack about static analysis
On Sun, 25 Dec 2011 01:51:57 +0200, bearophile bearophileh...@lycos.com wrote: But a problem is that such D code is full of braces and parentheses that make such code hard to read and write (I agree with Andrei that D is not very designed for such kind of code, but the fact doesn't change). Absolutely agree braces and parens are ugly and unreadable. Yet if your solution to this particular case is functional syntax, i disagree. It doesn't make it readable, at least when you are not writing sample code. When things get ugly (they do) braces and parens are your only friends. Whenever i see those that suggest tabs instead of braces, i can't help thinking that they just write sample code. I would love to have a = a*a for only lambdas but for generic case i haven't seen anything that could replace parens and braces.
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 23:44:11 UTC, Timon Gehr wrote:
Functional style code tends to be conceptually simpler.
I find bugs creep into things with lots of parameters
or conditions. For instance:
if((flag request.isSomething()) || request.isSomethingElse())
{}
If you have a lot of stuff like that, it's hard to tell what
code is actually running, which makes working with it hard,
even if the other stuff is simple.
This tends to have a good chunk of repetition too, inflating
the size, since you're doing many custom paths for all the
conditions. And that's where you miss stuff.
(On the other hand, I don't have a lot of experience working with
non-example functional code, so maybe this tends to be avoided
there?)
Re: Carmack about static analysis
On Saturday, 24 December 2011 at 23:51:57 UTC, bearophile wrote:
Using certain abstractions sometimes helps to write one idea
only once in a program. Etc.
This is the biggest one, and applies to all kinds of code.
I like to write little functions with meaningful names.
bool isOdd(int i) {
if((i % 2) == 0)
return false;
else
return true;
}
filter!isOdd(myCollection);
I find that nicer than
filter!i % 2 != 0(myCollection);
despite it being longer.
With the former, it says very simply that it wants odd
numbers at the usage location.
With the latter, you have to think for a second about
what the modulus operator actually does and what the
definition of an odd number is before you can get to
why the filter is there at all.
It gets even worse if the other function has non-trivial
code. Best to just give it a name so you don't have to think
about the implementation at all at the usage site.
Re: Carmack about static analysis
The two biggest things that I believe D would benefit from, in terms of readability, are UFCS and C#-style lambdas (which were already discussed). Something like 'array(find(test, select!(a.name)(filter!(a.num 3)(Elements' Could be rewritten much cleaner as Elements.filter!a.num 3 .select!a.name .find(test) .array(); The first one is very confusing, hard to split across multiple lines without losing meaning, and difficult to see what it even operates on (assume Elements is not an array and thus can not be used with the current implementation of UFCS). The second one, you can clearly see operates on Elements, you can clearly split it across multiple lines, and it's very obvious where things stop and end. Plus, it eliminates at least one set of parenthesis per function call, not to mention being much easier to write as it's in order of steps instead of opposite order. On 24/12/2011 9:33 AM, Andrei Alexandrescu wrote: Anyhow, is there anything you have in mind that we have the chance of improving at this point?
Re: Carmack about static analysis
On 12/24/2011 05:44 PM, Timon Gehr wrote:
On 12/25/2011 12:27 AM, Adam D. Ruppe wrote:
On Saturday, 24 December 2011 at 23:12:27 UTC, bearophile wrote:
I was talking about the abundance of (({}){()}) and not about
identifiers length.
It's the same fallacy.
Not really. Functional style code tends to be conceptually simpler.
Having code that is more readable can help. Getting rid of (({return
{return}}){return()}) makes the code more readable, whereas excessively
shortening identifiers does the opposite.
See here for an example of what bearophile is talking about:
http://pastebin.com/2rEdx0RD
I looked over that code. It creates an entire lazy evaluation
environment. It's quite remarkable it's actually so concise, and I don't
see a lot of ways to significantly improve it.
However, I think the slow druntime GC is more of a show stopper for
functional D than any syntactic issues there may be.
Got the GC book this morning, already read 2.5 chapters :o).
Andrei
Re: Carmack about static analysis
On 12/24/2011 07:20 PM, Adam D. Ruppe wrote:
On Saturday, 24 December 2011 at 23:51:57 UTC, bearophile wrote:
Using certain abstractions sometimes helps to write one idea only once
in a program. Etc.
This is the biggest one, and applies to all kinds of code.
I like to write little functions with meaningful names.
bool isOdd(int i) {
if((i % 2) == 0)
return false;
else
return true;
}
filter!isOdd(myCollection);
I find that nicer than
filter!i % 2 != 0(myCollection);
despite it being longer.
Different strokes for different folks. IMHO it would be difficult to
justify the verboseness and the unnecessary (twice) flow of control. I
see every letter beyond this as a liability:
bool isOdd(int i)
{
return (i 1) != 0;
}
With the former, it says very simply that it wants odd
numbers at the usage location.
Giving an operation, however trivial, a symbol, is often a good thing.
There are of course limits, and each engineer has their own ideas where
to draw the line.
Andrei
Re: Carmack about static analysis
On 12/24/2011 10:37 PM, Kapps wrote: The two biggest things that I believe D would benefit from, in terms of readability, are UFCS and C#-style lambdas (which were already discussed). Something like 'array(find(test, select!(a.name)(filter!(a.num 3)(Elements' Could be rewritten much cleaner as Elements.filter!a.num 3 .select!a.name .find(test) .array(); The first one is very confusing, hard to split across multiple lines without losing meaning, and difficult to see what it even operates on (assume Elements is not an array and thus can not be used with the current implementation of UFCS). The second one, you can clearly see operates on Elements, you can clearly split it across multiple lines, and it's very obvious where things stop and end. Plus, it eliminates at least one set of parenthesis per function call, not to mention being much easier to write as it's in order of steps instead of opposite order. Pardon, what does UFCS stand for?
