Re: DMD svn and contract inheritance

[Stewart Gordon]

Walter Bright wrote:
snip
There are some tricky bits to doing contract inheritance, I hope gdc and 
ldc don't have trouble with it.


What tricky bits are those?

Stewart.

Re: DMD svn and contract inheritance

[Walter Bright]

Stewart Gordon wrote:

Walter Bright wrote:
snip
There are some tricky bits to doing contract inheritance, I hope gdc 
and ldc don't have trouble with it.


What tricky bits are those?


It's outlined in the code comments, but it's implemented by making the 
contract code a nested function. The overriding function calls those 
nested functions of the overridden function(s). In order for this to 
work successfully, the 'this' pointer and the stack parameters must wind 
up in the same relative position on the stack.

Re: It's official: One-day D tutorial at the ACCU Conference 2010 in Oxford, England

[Graham St Jack]

I would love to get my hands on the transcript and video of the event...
Will it be recorded?

Re: It's official: One-day D tutorial at the ACCU Conference 2010 in Oxford, England

[Andrei Alexandrescu]

Graham St Jack wrote:

I would love to get my hands on the transcript and video of the event...
Will it be recorded?


ACCU has not recorded its conferences in the past.

Andrei

Re: DMD svn and contract inheritance

[Stewart Gordon]

Walter Bright wrote:

Stewart Gordon wrote:

Walter Bright wrote:
snip
There are some tricky bits to doing contract inheritance, I hope gdc 
and ldc don't have trouble with it.


What tricky bits are those?


It's outlined in the code comments, but it's implemented by making the 
contract code a nested function. The overriding function calls those 
nested functions of the overridden function(s). In order for this to 
work successfully, the 'this' pointer and the stack parameters must wind 
up in the same relative position on the stack.


I'm still none the wiser about why it absolutely has to be done like 
this instead of the simpler solution I proposed years ago.  But in any 
case, thank you for finally getting round to it!


Stewart.

Code injection

[Joe]

Can anybody tell, what i`m doing wrong?
I ported it from c, where it works well.

code:
private {
import tango.sys.win32.Types;
import tango.sys.win32.UserGdi;
import tango.sys.win32.Macros;

import tango.stdc.stringz : fromString16z;
import tango.stdc.stringz : toStringz;
import tango.text.convert.Integer : toString;
import tango.text.convert.Utf : toString;


import tango.stdc.stringz : toString16z;
import tango.text.convert.Integer : toString16;
import tango.text.convert.Utf : toString16;

extern(Windows) LPVOID VirtualAllocEx(HANDLE, LPVOID, DWORD, DWORD, DWORD);
}

void main() {
try
{
injSelfDelete(0);
}
catch(Exception x)
{
Report(x);
}
}

void Report(Exception x) {
wchar[] msg;
msg.length = 256;
int errcode = GetLastError();

FormatMessageW(FORMAT_MESSAGE_FROM_SYSTEM, null, errcode, 0, msg.ptr, 
msg.length, null);

wchar[] rep =   toString16(x.toString) ~
\nline:  ~ toString16(x.line) ~
\nlast err: [ ~ toString16(errcode) ~ ]  ~ msg ~ 
\0;

MessageBoxW(null, toString16z(rep), null, 0);
}

alias bool (*DeleteFileT)(char*);
alias void (*ExitProcessT)(uint);
alias void (*BeepT)(uint, uint);
alias void (*MessageBoxT)(void*, char*, char*, uint);

struct DeleteInjectData
{
DeleteFileT DeleteFile;
ExitProcessTExitProcess;
char szFileName [MAX_PATH];
};


static void DeleteInjectProc (DeleteInjectData *id) {
//~ id.DeleteFile(id.szFileName.ptr);
id.ExitProcess(0);

//~ asm
//~ {
//~ push id.szFileName.ptr;
//~ call id.DeleteFile;
//~ push 0;
//~ call id.ExitProcess;
//~ }
}

static void DeleteInjectProc_End () { }

void injSelfDelete (int exitCode) {
DeleteInjectData id;
int threadSize = cast(void*)DeleteInjectProc_End - 
cast(void*)DeleteInjectProc;

HMODULE hKernel32 = LoadLibraryA(Kernel32.dll);
assert(hKernel32 != null);
id.DeleteFile = cast(DeleteFileT) GetProcAddress(hKernel32, 
DeleteFileA);
id.ExitProcess = cast(ExitProcessT) GetProcAddress(hKernel32, 
ExitProcess);
assert(id.DeleteFile != null);
assert(id.ExitProcess != null);

id.szFileName[0..$] = 0;
GetModuleFileNameA(GetModuleHandleA(null), id.szFileName.ptr, 
id.szFileName.length);

injectNew(cast(void*)DeleteInjectProc, threadSize, cast(void*)id, 
id.sizeof);

ExitProcess(exitCode);
}

bool injectNew(void* threadProc, uint codeLength, void* data, uint dataSize) {
PROCESS_INFORMATION pi;
STARTUPINFO si;
ZeroMemory(pi, pi.sizeof);
ZeroMemory(si, si.sizeof);
si.cb = si.sizeof;

char[] cmdExe;
cmdExe.length = MAX_PATH;
cmdExe.length = GetSystemDirectoryA(cmdExe.ptr, cmdExe.length);
cmdExe ~= \\cmd.exe\0;

assert(CreateProcessA(cmdExe.ptr, null, null, null, false, 
CREATE_SUSPENDED /*CREATE_NO_WINDOW */,
null, null, si, pi));

void *lpDataMem = VirtualAllocEx(pi.hProcess, null, dataSize, 
MEM_COMMIT, PAGE_EXECUTE_READWRITE);
assert(lpDataMem != null);

void *lpThreadMem = VirtualAllocEx(pi.hProcess, null, codeLength, 
MEM_COMMIT,
PAGE_EXECUTE_READWRITE);
assert(lpThreadMem != null);

DWORD bytesWritten = 0;
assert(WriteProcessMemory(pi.hProcess, lpThreadMem, threadProc, 
codeLength, bytesWritten));

assert(WriteProcessMemory(pi.hProcess, lpDataMem, data, dataSize, 
bytesWritten));

DWORD dwThreadId = 0;
HANDLE hRemote = CreateRemoteThread(pi.hProcess, null, codeLength, 
lpThreadMem, lpDataMem, 0,
dwThreadId);
assert(hRemote != INVALID_HANDLE_VALUE);

ResumeThread(pi.hThread);

return true;
}

void injRedExitProcess (int exitCode) {
}

void injRedUse(wchar[] rcName, wchar[] rcType) {
}

void[] injRedGet() {
return null;
}

void injRedSet(void[] data) {
}

Re: Null references redux

[Don]

Jeremie Pelletier wrote:

Don wrote:

Jeremie Pelletier wrote:

Yigal Chripun wrote:

On 29/09/2009 16:41, Jeremie Pelletier wrote:

What I argued about was your view on today's software being too big 
and

complex to bother optimize it.



that is not what I said.
I was saying that hand optimized code needs to be kept at minimum 
and only for visible bottlenecks, because the risk of introducing 
low-level unsafe code is bigger in more complex and bigger software.


What's wrong with taking a risk? If you know what you're doing where 
is the risk, and if now how will you learn? If you write your 
software correctly, you could add countless assembly optimizations 
and never compromise the security of the entire thing, because these 
optimizations are isolated, so if it crashes there you have only a 
narrow area to debug within.


There are some parts where hand optimizing is almost useless, like 
network I/O since latency is already so high having a faster code 
won't make a difference.


And sometimes the optimization doesn't even need assembly, it just 
requires using a different high level construct or a different 
algorithm. The first optimization is to get the most efficient data 
structures with the most efficient algorithms for a given task, and 
THEN if you can't optimize it more you dig into assembly.


People seem to think assembly is something magical and incredibly 
hard, it's not.


Jeremie


Also, if you're using asm on something other than a small, simple 
loop, you're probably doing something badly wrong. Therefore, it 
should always be localised, and easy to test thoroughly. I don't think 
local extreme optimisation is a big risk.


That's also how I do it once I find the ideal algorithm, I've never had 
any problems or seen any risk with this technique, I did see some good 
performance gains however.


Greater risks come from using more complicated algorithms. Brute-force 
algorithms are always the easiest ones to get right g.


I'm not sure I agree with that. Those algorithms are pretty isolated and 
really easy to write unittests for so I don't see where the risk is when 
writing more complex algorithms, it's obviously harder, but not riskier.


By riskier I mean more chance of containing an error.

I'm partly basing this on my recent experience with writing BigInt. The 
low-level asm routines are easy to get right, and it's easy to tell when 
you've go them wrong. They do brute-force stuff, like schoolbook O(n^2) 
multiplication, and importantly, _there are no special cases_ because it 
needs to be fast.
But the higher-level O(n^1.3) multiplication algorithms are full of 
special cases, and that's where the bugs are.

Re: Video Codecs?

[grauzone]

Jacob Carlborg wrote:

On 9/30/09 22:42, Benji Smith wrote:
Does anybody know of any D libraries implementing or wrapping video 
codecs?


I need to read video files (AVI or MPEG would be fine) using DIVX,
XVID, or any other popular codec. In addition to playing those files in
a media player control, I need to extract individual frames and perform
various filtration and processing operations on them, for a computer
vision project I'm about to start working on.

I looked around at DSource but didn't find anything there. Any ideas?

--benji


There is rulesPlayer: http://www.dsource.org/projects/rulesplayer it 
uses MPlayer so I guess it has to have bindings to it but I don't know 
what you can do with MPlayer.


He's probably better off using ffmpeg directly. Or DirectShow. Maybe 
there are even bindings for those on dsource.

Re: Code injection

[Daniel Keep]

Joe wrote:
 Can anybody tell, what i`m doing wrong?

http://catb.org/~esr/faqs/smart-questions.html#forum (D.learn in this case)

http://catb.org/~esr/faqs/smart-questions.html#beprecise

http://catb.org/~esr/faqs/smart-questions.html#goal

http://catb.org/~esr/faqs/smart-questions.html#explicit

http://catb.org/~esr/faqs/smart-questions.html#code

A possible leak

[bearophile]

Time ago Jon Harrop has found a memory leak in a F# program running on Mono, he 
has reduced the program to a minimal test case. I have translated that code to 
D2 and Python2:

-

struct Node {
int data;
Node* next;

this(int data, Node* next) {
this.data = data;
this.next = next;
}
}

void main() {
Node* lh = new Node(1, null);
lh.next = lh;

while (true) {
Node* lh2 = lh;
lh2.next = new Node(2, lh2.next);
lh = lh2.next;
lh2 = lh;
lh2.next = lh2.next.next;
}
}


-

class Node:
def __init__(self, data, next):
self.data = data
self.next = next

def main():
lh = Node(1, None)
lh.next = lh

while True:
lh2 = lh
lh2.next = Node(2, lh2.next)
lh = lh2.next
lh2 = lh
lh2.next = lh2.next.next

main()

-

It just creates a circular single-linked list that represents a queue. And then 
adds an item and pops another out. The popped out items have the next field 
that point to the struct itself.

The D version of the code eats more and more RAM, while the Python version runs 
at constant memory (probably because CPython GC is a reference count augmented 
with a cycle detector, that helps in such situations).

This was an answer on the mono list regarding this GC problem (that the dotnet 
GC doesn't share, so that F# program doesn't leak on dotnet):
http://lists.ximian.com/pipermail/mono-list/2009-February/041236.html

This topic may interest Lucarella too.

Can the D GC be improved to avoid such problem, maybe like the CPython GC? And 
is such improvement worth it (= useful in practical programs)?

Bye,
bearophile

Re: A possible leak

[Leandro Lucarella]

bearophile, el  1 de octubre a las 07:23 me escribiste:
 Can the D GC be improved to avoid such problem, maybe like the CPython
 GC? And is such improvement worth it (= useful in practical programs)?

I've tested it with LDC (using classes instead of structs, because D1
doesn't support struct constructors) and it works with a perfectly stable
memory usage.

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
A lo que Peperino respondióles: aquel que tenga sabañones que se los
moje, aquel que padece calvicie no padece un osito, no es bueno comer
lechón en día de gastritis, no mezcleis el vino con la sandía, sacad la
basura después de las ocho, en caso de emergencia rompa el vidrio con
el martillo, a cien metros desvio por Pavón.
-- Peperino Pómoro

Re: restructuring name hiding around the notion of hijacking

[Andrei Alexandrescu]

Michel Fortin wrote:
On 2009-09-30 22:01:54 -0400, Andrei Alexandrescu 
seewebsiteforem...@erdani.org said:


Today's D has a very strong, principled notion of hijacking: for any 
given function call, if the call candidates are found in different 
modules, the call is invalid. I think that works great.


Lately I've been thinking of using the same notion of hijacking as a 
replacement for symbol hiding in inheritance hierarchies. Right now, 
if a derived class defines a symbol, that symbol simply hides whatever 
homonym symbol (unless it overrides it). There are some warnings about 
hiding sometimes, but it's all kind of fuzzy.


How about just using hijacking? The basic idea is that a use of a 
symbol in a class should not hijack a homonym symbol defined in a 
different module.



What do you think?


I think it's a good idea, but there should be a way to *override* static 
functions.


That has the same risks. The problem right now is that in order to use a 
class, you must absorb the definition of that class and that of each 
superclass of it, all the way up to Object. With hijacking thwarted, you 
can specify stuff in the base class that you can be sure will continue 
to work the same in derived classes. I believe this makes using classes 
quite a lot easier and safer.


In fact I sometime wonder if it'd be a good idea to disallow hijacking 
of global variables with local variables inside functions too, but 
that's more triky to do.


I explain in TDPL that that's not a good idea. Let me paste the text:

=
A symbol  defined inside  a scope hides  a homonym symbol  hanging out
outside all scopes:

\begin{D}
uint widgetCount;
...
void main() {
   writeln(widgetCount); // writes the global symbol
   auto widgetCount = getWidgetCount();
   writeln(widgetCount); // writes the local symbol
}
\end{D}

The first call to @writeln@ prints the global @widgetCount@ symbol and
the second accesses the locally-defined @widgetco...@. Should there be
a  need for  accessing the  global symbol  after it  has  been masked,
prefixing it  with a ``.''---as  in @writeln(.widgetCount)@---will do,
as first  mentioned on page~\ref{pg:dotSyntaxForScoping}.  However, it
is illegal to define a symbol that would mask a symbol in an enclosing
compound statement:

\begin{D}
void main() {
   auto widgetCount = getWidgetCount();
   // let's now open a nested block
   {
  auto widgetCount = getWidgetCount(); // error!
   }
}
\end{D}

As long as masking does not occur, it's legal to reuse the same symbol
in different compound statements:

\begin{D}
void main() {
   {
  auto i = 0;
  ...
   }
   {
  auto i = eye; // fine
  ...
   }
   double i = 3.14; // fine too
}
\end{D}

The rationale of this setup is simple.  Allowing global symbol masking
is necessary  for writing good,  modular code that's assembled  out of
separately-compiled  parts; you don't  want the  addition of  a global
variable to suddenly  render various innocent bystanders uncompilable.
On the other hand, enclosing-scope  masking is useless as a modularity
device (as there's never the  case a compound statement spans multiple
modules in~\dee) and most often indicates either an oversight aspiring
to become a bug, or a cancerous function that's grown out of control.



Andrei

Defining some stuff for each class in turn

[Andrei Alexandrescu]

I am becoming increasingly aware that we need to provide some means to 
define certain members (data and functions) for each class as if they 
were pasted there.


Right now that right is reserved to the compiler, which generates e.g. 
one static classinfo object for each class. But users would want to also 
define members for each class automatically. This is often the case with 
contravariant-argument functions (that we discussed recently) and other 
important cases such as factories and cloning.


For starters, assume I want to define one static int for each class in a 
hierarchy.


class Counted {
static uint counter;
...
}

Then subclasses of Counted would all share counter, something I don't 
want. I want each subclass to have its own counter, so I need to ask 
derived classes to *also* define counter:


class A : Counted {
static uint counter;
...
}

With the suggested feature, there would be the ability to define counter 
 for each class in turn.


class Counted {
mixin(Derived)
{
// Insert here stuff that must be pasted for each subclass
// of Counted (including Counted itself).
// Use Derived as the name of the current subtype of Counter
static uint counter;
static if (is(Counted == Derived))
ref uint getCounter() { return counter; }
else
override ref uint getCounter() { return counter; }
}
...
}

The code above does something quite neat - it defines an overridable 
function Counted.getCounter in the base class, and then overrides it in 
*every* class inheriting that base class, to return that class' own 
counter.


The same should go in interface definitions - the feature should allow 
you to


There are quite a few immediate applications of this: opEquals, opCmp, 
clone, various factories and object pools, and most importantly 
reflection. To enable custom reflection with today's D, we'd have to 
require each class to insert some code inside the class body. With the 
mechanism described above, we allow the base class or an interface (e.g. 
Reflectable) to inject the code into the derived class' body.


What do you think?


Andrei

Re: A possible leak

[Leandro Lucarella]

bearophile, el  1 de octubre a las 12:39 me escribiste:
 Leandro Lucarella:
 
  I've tested it with LDC (using classes instead of structs, because D1
  doesn't support struct constructors) and it works with a perfectly stable
  memory usage.
 
 That's a different situation, the compiler is different, the data structure 
 is different, and the runtime too may be a little different.
 I have tested with LDC with both structs and classes and the two programs 
 don't leak.

Sure, the point was: this is not a problem inherently to the D GC, it's
just a bug in a particular (version of the) compiler you are testing.

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
Si pudiera acercarme un poco más, hacia vos
Te diría que me tiemblan los dos pies, cuando me mirás
Si supieras todo lo que me costó, llegar
Hoy sabrías que me cuesta respirar, cuando me mirás

Re: Defining some stuff for each class in turn

[Jarrett Billingsley]

On Thu, Oct 1, 2009 at 12:25 PM, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
 I am becoming increasingly aware that we need to provide some means to
 define certain members (data and functions) for each class as if they were
 pasted there.

 Right now that right is reserved to the compiler, which generates e.g. one
 static classinfo object for each class. But users would want to also define
 members for each class automatically. This is often the case with
 contravariant-argument functions (that we discussed recently) and other
 important cases such as factories and cloning.

 For starters, assume I want to define one static int for each class in a
 hierarchy.

 class Counted {
    static uint counter;
    ...
 }

 Then subclasses of Counted would all share counter, something I don't want.
 I want each subclass to have its own counter, so I need to ask derived
 classes to *also* define counter:

 class A : Counted {
    static uint counter;
    ...
 }

 With the suggested feature, there would be the ability to define counter
  for each class in turn.

 class Counted {
    mixin(Derived)
    {
        // Insert here stuff that must be pasted for each subclass
        // of Counted (including Counted itself).
        // Use Derived as the name of the current subtype of Counter
        static uint counter;
        static if (is(Counted == Derived))
            ref uint getCounter() { return counter; }
        else
            override ref uint getCounter() { return counter; }
    }
    ...
 }

 The code above does something quite neat - it defines an overridable
 function Counted.getCounter in the base class, and then overrides it in
 *every* class inheriting that base class, to return that class' own counter.

 The same should go in interface definitions - the feature should allow you
 to

 There are quite a few immediate applications of this: opEquals, opCmp,
 clone, various factories and object pools, and most importantly reflection.
 To enable custom reflection with today's D, we'd have to require each class
 to insert some code inside the class body. With the mechanism described
 above, we allow the base class or an interface (e.g. Reflectable) to inject
 the code into the derived class' body.

 What do you think?

I think it sounds interesting enough, but I can't help but wonder if
this is a feature that you've really thought through (especially wrt.
how it interacts with mechanisms such as template mixins and normal
symbol inheritance), or if you just want it to support some pattern
you want to use in Phobos 2.

Re: The Non-Virtual Interface idiom in D

[Jérôme M. Berger]

Jeremie Pelletier wrote:

Jérôme M. Berger wrote:

Jeremie Pelletier wrote:

Jérôme M. Berger wrote:

Michel Fortin wrote:
I fully support having a way to specify a default implementation 
for a function in an interface. It might get handy for a few things 
(like implementing the delegate pattern you see everywhere in 
Cocoa). But it's a bad replacement for contracts.


Then what's the difference between an interface and an abstract 
class? I thought that the whole point of interfaces was that you 
couldn't have implementations of the methods so that you had no 
problem choosing an implementation when inheriting from multiple 
interfaces.


Jerome


The interface supports multiple inheritance since it doesn't add to 
the vtable of the class using it,


?? The interface adds as much (or as little) to the vtable as 
another class would. The reason why interfaces support multiple 
inheritance is that since the interface does not contain the code for 
any of its methods, there is never any doubt which method should be 
called even if several ancestors have methods with the same signature. 
Once you add code to the interface you loose that advantage.


Wrong, the implementation of the interface makes the vtable, the 
interface is only a skeleton for the methods. That interface code would 
only provide a default implementation for the class implementing it, 
which is really useful if you intend to have multiple implementations 
and get some generic code from the interface, unless of using template 
mixins.


	And how does it work if you implement several interfaces, with 
functions of the same name and different interface code. This is 
precisely the reason why we don't have multiple inheritance of 
classes...


and its code would be implemented on the classes implementing the 
interface, not overridden by subclasses.


I don't see how that's different from standard class inheritance. 
After all there is nothing that forces you to override methods in 
subclasses if they already have an implementation in the parent class 
either.


Jerome


Yes but you can only extend a single base class, while you may implement 
multiple interfaces, how would you implement generic methods for these 
interfaces under that model? The only way is through template mixins and 
that's not the most convenient method.


Basically interface code would be just like templates that gets mixed in 
the implementing class. You could easily write generic contracts for 
interfaces that way, instead of repeating the same contracts in every 
implementation. It's just another tool to help generic programming.


	I'll ask it again: how is that different from multiple class 
inheritance? I know we don't have it in D (and for good reasons) but 
once we add what you're describing, we might as well call the 
interfaces classes and be done with it.


Jerome

PS: Contracts are a different issue. I'd be all in favour of adding 
contracts to the interfaces now that we have contract inheritance 
for classes.

--
mailto:jeber...@free.fr
http://jeberger.free.fr
Jabber: jeber...@jabber.fr

Re: Defining some stuff for each class in turn

[Andrei Alexandrescu]

Jarrett Billingsley wrote:

On Thu, Oct 1, 2009 at 12:25 PM, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:

[code injection]

What do you think?


I think it sounds interesting enough, but I can't help but wonder if
this is a feature that you've really thought through (especially wrt.
how it interacts with mechanisms such as template mixins and normal
symbol inheritance), or if you just want it to support some pattern
you want to use in Phobos 2.


I've known for a long time this was in store if we want to define decent 
reflection. It's also been a perennial source of trouble with a lot of 
code that needs to inject members.


Let me give another example. When we discussed opCmp around here, people 
said that's the old way of doing things and that the right way is to use 
an interface Comparator!T, akin to Java's ComparatorT:


http://java.sun.com/j2se/1.5.0/docs/api/java/util/Comparator.html

That only solves exactly one level of inheritance. It's a more elaborate 
solution that doesn't quite help that much. Consider:


interface Comparator(T)
{
int opCmp(T rhs); // yum, exact type
}

class Widget : Comparator!Widget
{
override opCmp(Widget rhs) { ... } // yum, exact type
}

class Gadget : Widget
{
override opCmp(Gadget rhs) { ... } // ERROR
}

Of course that didn't work. Gadget.opCmp doesn't override anything. 
Gadget needs to remember to inherit Comparator!Gadget:


class Gadget : Widget, Comparator!Gadget
{
override opCmp(Gadget rhs) { ... } // oh ok
}

So any class X in Widget's hierarchy that forgets to inherit 
Comparator!X undergoes the risk of having the wrong opCmp called for it.


With injection, Comparator can be made to work for any interface:

interface Comparator(T)
{
int opCmp(Comparator!T rhs);
mixin(Impl) // for each implementation Impl
{
int opCmp(Impl rhs);
override int opCmp(Comparator!T rhs)
{
return opCmp(cast(Impl) rhs);
}
}
}

class Widget : Comparator!Widget { ... }

Now every derived class of Widget, including Widget itself, commits to 
define a opCmp with the proper signature (failure to do so results in a 
linker error), and also defines the interface function in terms of it.



Andrei

Re: Defining some stuff for each class in turn

[grauzone]

Andrei Alexandrescu wrote:

Jarrett Billingsley wrote:

I think it sounds interesting enough, but I can't help but wonder if
this is a feature that you've really thought through (especially wrt.
how it interacts with mechanisms such as template mixins and normal
symbol inheritance), or if you just want it to support some pattern
you want to use in Phobos 2.


I've known for a long time this was in store if we want to define decent 
reflection. It's also been a perennial source of trouble with a lot of 
code that needs to inject members.


I don't see how that would be needed for reflection, and I've written 
some custom reflection/serialization myself. Yay for tupleof being able 
to read even private members.

What does Coverity/clang static analysis actually do?

[Walter Bright]

I've been interested in having the D compiler take advantage of the flow 
analysis in the optimizer to do some more checking. Coverity and clang 
get a lot of positive press about doing this, but any details of exactly 
*what* they do have been either carefully hidden (in Coverity's case) or 
undocumented (clang's page on this is blank). All I can find is 
marketing hype and a lot of vague handwaving.


Here is what I've been able to glean from much time spent with google on 
what they detect and my knowledge of how data flow analysis works:


1. dereference of NULL pointers (all reaching definitions of a pointer 
are NULL)


2. possible dereference of NULL pointers (some reaching definitions of a 
pointer are NULL)


3. use of uninitialized variables (no reaching definition)

4. dead assignments (assignment of a value to a variable that is never 
subsequently used)


5. dead code (code that can never be executed)

6. array overflows

7. proper pairing of allocate/deallocate function calls

8. improper use of signed integers (who knows what this actually is)


Frankly, this is not an impressive list. These issues are discoverable 
using standard data flow analysis, and in fact are part of Digital Mars' 
optimizer. Here is the current state of it for dmd:


1. Optimizer discovers it, but ignores the information. Due to the 
recent thread on it, I added a report for it for D (still ignored for 
C). The downside is I can no longer use *cast(char*)0=0 to drop me into 
the debugger, but I can live with that as assert(0) will do the same thing.


2. Optimizer collects the info, but ignores this, because people are 
annoyed by false positives.


3. Optimizer detects and reports it. Irrelevant for D, though, because 
variables are always initialized. The =void case is rare enough to be 
irrelevant.


4. Dead assignments are automatically detected and removed. I'm not 
convinced this should be reported, as it can legitimately happen when 
generating source code. Generating false positives annoy the heck out of 
users.


5. Dead code is detected and silently removed by optimizer. dmd front 
end will complain about dead code.


6. Arrays are solidly covered by a runtime check. There is code in the 
optimizer to detect many cases of overflows at compile time, but the 
code is currently disabled because the runtime check covers 100% of the 
cases.


7. Not done because it requires the user to specify what the paired 
functions are. Given this info, it is rather simple to graft onto 
existing data flow analysis.


8. D2 has acquired some decent checking for this.


There's a lot of hoopla about these static checkers, but I'm not 
impressed by them based on what I can find out about them. What do you 
know about what these checkers do that is not on this list? Any other 
kinds of checking that would be great to implement?


D's dead code checking has been an encouraging success, and I think 
people will like the null dereference checks. More along these lines 
will be interesting.

Re: SoftBound

[Walter Bright]

bearophile wrote:

SoftBound: Highly Compatible and Complete Spatial Memory Safety for
C by  Santosh Nagarakatte, Jianzhou Zhao, Milo M K Martin and Steve
Zdancewic: http://llvm.org/pubs/2009-06-PLDI-SoftBound.html

It's a block of about 6000 lines of C++ code that augment the LLVM
compiler, adding extra tests. It avoids out-of-bound situations with
pointers. It works after the compilation stage, on the LL code
produced by LLVM, so it can be used equally for C or D. Conceptually
looks like a simple thing. Currently can't be used yet, but I'd like
to have a compilation flag in LDC to activate this. In in nonrelease
mode some of the tests are already present (the bound tests of
arrays), so it can avoid to put them in twice (in LDC you can disable
only bound tests, and keep assertions, etc).


I don't think there's much point to this in D. You rarely need to deal 
with pointers directly. Arrays are already checked.

Re: Defining some stuff for each class in turn

[Leandro Lucarella]

Andrei Alexandrescu, el  1 de octubre a las 11:25 me escribiste:
 What do you think?

I think this is close to Python metaclasses[1], but much more specific.
It's amazing how many cool things one can do with Python metaclasses
(there are several articles discussing them[2]) and decorators.  I would
love to see something similar in D but operating at compile-time.  I think
it would be nice to point to a more general solution (like AST macros)
than implementing all sort of small and specific tricks for each problem
instead.

[1] http://docs.python.org/reference/datamodel.html#customizing-class-creation
[2] http://en.wikipedia.org/wiki/Metaclass
http://www.ibm.com/developerworks/linux/library/l-pymeta.html

-- 
Leandro Lucarella (AKA luca)  http://llucax.com.ar/
--
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
--
Wake from your sleep,
the drying of your tears,
Today we escape, we escape.

Re: Defining some stuff for each class in turn

[Denis Koroskin]

On Thu, 01 Oct 2009 20:25:03 +0400, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:

I am becoming increasingly aware that we need to provide some means to  
define certain members (data and functions) for each class as if they  
were pasted there.


Right now that right is reserved to the compiler, which generates e.g.  
one static classinfo object for each class. But users would want to also  
define members for each class automatically. This is often the case with  
contravariant-argument functions (that we discussed recently) and other  
important cases such as factories and cloning.


For starters, assume I want to define one static int for each class in a  
hierarchy.


class Counted {
 static uint counter;
 ...
}

Then subclasses of Counted would all share counter, something I don't  
want. I want each subclass to have its own counter, so I need to ask  
derived classes to *also* define counter:


class A : Counted {
 static uint counter;
 ...
}

With the suggested feature, there would be the ability to define counter  
  for each class in turn.


class Counted {
 mixin(Derived)
 {
 // Insert here stuff that must be pasted for each subclass
 // of Counted (including Counted itself).
 // Use Derived as the name of the current subtype of Counter
 static uint counter;
 static if (is(Counted == Derived))
 ref uint getCounter() { return counter; }
 else
 override ref uint getCounter() { return counter; }
 }
 ...
}

The code above does something quite neat - it defines an overridable  
function Counted.getCounter in the base class, and then overrides it in  
*every* class inheriting that base class, to return that class' own  
counter.


The same should go in interface definitions - the feature should allow  
you to


There are quite a few immediate applications of this: opEquals, opCmp,  
clone, various factories and object pools, and most importantly  
reflection. To enable custom reflection with today's D, we'd have to  
require each class to insert some code inside the class body. With the  
mechanism described above, we allow the base class or an interface (e.g.  
Reflectable) to inject the code into the derived class' body.


What do you think?


Andrei


This is cool. I'd also add Serialization task to the application list  
(currently I manually insert mixin Serializable!(); into every class.


There is one thing that I dislike about it, though. I believe mixin(Foo)  
{} syntax is a bit far-fetched, it is not obvious that Foo represents  
derived type inside a mixin scope.


I'd like to remind you that we could just use typeof(this) instead of  
Foo/Derived/etc:


static if (is (typeof(this) == Counted) {
ref uint getCounter() { return counter; }
} else {
override ref uint getCounter() { return counter; }
}

If you remember, we had a similar discussion about a half year ago. It was  
about ICloneable interface that forces all derived class to implement  
proper clone() method. We could define it as:


// forces implementor class and all the derivatives to implement proper  
clone() method

interface ICloneable
{
scope mixin {   // alternative syntax
typeof(this) clone();
}
}

and also be able to declare a method non-recursively (so that derived  
class don't have to implement that method):


interface ICloneableNonRecursive
{
typeof(this) clone();
}

Other possible solution could be to use a special derived keyword. We  
already have a class-level compile-time constant - super. We could also  
introduce its counter-part (derived), which will expand into a concrete  
derived class type. This will simplify the syntax a bit and will allow to  
get rid of mixin and an addition level of indentation:


interface ICloneable
{
derived clone();
}

interface IComparable
{
int opCmp(derived other);
}

As a downside, I don't know how to define a field in every class this way  
(int derived.counter; maybe?).

Re: What does Coverity/clang static analysis actually do?

[BCS]

Hello Walter,


Frankly, this is not an impressive list. These issues are discoverable
using standard data flow analysis, and in fact are part of Digital
Mars' optimizer. Here is the current state of it for dmd:

1. Optimizer discovers it, but ignores the information. Due to the
recent thread on it, I added a report for it for D (still ignored for
C). The downside is I can no longer use *cast(char*)0=0 to drop me
into the debugger, but I can live with that as assert(0) will do the
same thing.


nice


4. Dead assignments are automatically detected and removed. I'm not
convinced this should be reported, as it can legitimately happen when
generating source code. Generating false positives annoy the heck out
of users.


vote++ on silent


6. Arrays are solidly covered by a runtime check. There is code in the
optimizer to detect many cases of overflows at compile time, but the
code is currently disabled because the runtime check covers 100% of
the cases.



I'd advocate for any compile time checks that never generate false positives 
running even if the runtime checks would get it also. I'd rather known sooner 
than later.

Re: Defining some stuff for each class in turn

[Andrei Alexandrescu]

Denis Koroskin wrote:

On Thu, 01 Oct 2009 20:25:03 +0400, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:

[...]

What do you think?


Andrei


This is cool. I'd also add Serialization task to the application list 
(currently I manually insert mixin Serializable!(); into every class.


Yah, serialization is close to reflection so it's definitely part of the 
target applications of the features.


There is one thing that I dislike about it, though. I believe mixin(Foo) 
{} syntax is a bit far-fetched, it is not obvious that Foo represents 
derived type inside a mixin scope.


I'd like to remind you that we could just use typeof(this) instead of 
Foo/Derived/etc:


static if (is (typeof(this) == Counted) {
ref uint getCounter() { return counter; }
} else {
override ref uint getCounter() { return counter; }
}


Yah, I thought a lot along the likes of typeof(this). We need some 
symbolic alias for the struct or class currently being defined and 
typeof(this) is an obvious choice. I just fear that it might be 
confusing that typeof(this) works even outside any method, i.e. when 
there is no this.


If you remember, we had a similar discussion about a half year ago. It 
was about ICloneable interface that forces all derived class to 
implement proper clone() method. We could define it as:


// forces implementor class and all the derivatives to implement proper 
clone() method

interface ICloneable
{
scope mixin {// alternative syntax
typeof(this) clone();
}
}


Yah, I remember.

and also be able to declare a method non-recursively (so that derived 
class don't have to implement that method):


interface ICloneableNonRecursive
{
typeof(this) clone();
}

Other possible solution could be to use a special derived keyword. We 
already have a class-level compile-time constant - super. We could 
also introduce its counter-part (derived), which will expand into a 
concrete derived class type. This will simplify the syntax a bit and 
will allow to get rid of mixin and an addition level of indentation:


interface ICloneable
{
derived clone();
}

interface IComparable
{
int opCmp(derived other);
}

As a downside, I don't know how to define a field in every class this 
way (int derived.counter; maybe?).


I dislike adding yet another keyword. In fact, I want to propose Walter 
to eliminate super as a keyword and make it just an alias for the base 
class as if you defined it by hand.



Andrei

Re: SoftBound

[bearophile]

Walter Bright:

 I don't think there's much point to this in D. You rarely need to deal 
 with pointers directly. Arrays are already checked.

In D pointers are quite less common than in C, but half of the point of using D 
is to be able to use pointers too, when you want to implement your own data 
structures, otherwise it may be better to just use Java in the first place. So 
in my opinion adding optional safeties to D pointers can be useful.

The good things of the design of SoftBound is that it looks simple to implement 
(and probably LDC may just use/adapt the already existing implementation), it's 
logically sound, it doesn't change the behaviour of the C/D program and works 
with most or all programs, it's safe, it doesn't need changes to the source 
code of programs to be used, and the performancememory overhead it introduces 
is usually acceptable in nonrelease mode (there are two different usage modes). 
I have seen more than ten similar systems for C, this one looks like being 
simple and effective enough.

Bye,
bearophile

Can D compile for PowerPC Architecture?

[Snake]

For some reason, about an hour ago I was googling stuff about X86 and PowerPC 
and right now I'm not even sure if this is a serious question nor do I know if 
I know what I'm talking about or if the question is even valid.

If that doesn't make sense, I want to know if you guys are developing D for use 
with the PowerPC Instruction Set

The question: Can you program for PowerPC architecture using D? In my mind I'm 
having these crazy ideas about things that are remotely relevant to programming 
but...whatever _ This is me rambling at this point.

Re: Can D compile for PowerPC Architecture?

[Jarrett Billingsley]

On Thu, Oct 1, 2009 at 4:16 PM, Snake loasolidsnake...@gmail.com wrote:
 For some reason, about an hour ago I was googling stuff about X86 and PowerPC 
 and right now I'm not even sure if this is a serious question nor do I know 
 if I know what I'm talking about or if the question is even valid.

 If that doesn't make sense, I want to know if you guys are developing D for 
 use with the PowerPC Instruction Set

 The question: Can you program for PowerPC architecture using D? In my mind 
 I'm having these crazy ideas about things that are remotely relevant to 
 programming but...whatever _ This is me rambling at this point.


AFAIK yes. I think GDC can do it, but it's long been out-of-date. I
haven't heard much about LDC on that front.

Should certain abstract classes be instantiable?

[Andrei Alexandrescu]

Consider:

class A {
abstract void fun() {}
}

The class defines a function that is at the same time abstract (so it 
requires overriding in derivees) and has implementation.


Currently the compiler disallows creation of objects of type A, although 
technically that is feasible given that A defines the abstract method.


Should A be instantiable? What designs would that help or hinder?


Andrei

Re: What does Coverity/clang static analysis actually do?

[Nick Sabalausky]

Walter Bright newshou...@digitalmars.com wrote in message 
news:ha2run$275...@digitalmars.com...

 1. dereference of NULL pointers (all reaching definitions of a pointer are 
 NULL)
 1. Optimizer discovers it, but ignores the information. Due to the recent 
 thread on it, I added a report for it for D (still ignored for C). The 
 downside is I can no longer use *cast(char*)0=0 to drop me into the 
 debugger, but I can live with that as assert(0) will do the same thing.


Sounds like a nice start.

 2. possible dereference of NULL pointers (some reaching definitions of a 
 pointer are NULL)
 2. Optimizer collects the info, but ignores this, because people are 
 annoyed by false positives.


If you mean something like this:

Foo f;
if(cond)
f = new Foo();
f.bar();

Then I *want* the compiler to tell me. C# does this and I've never been 
annoyed by it, in fact I've always appreciated it. I'm not aware of any 
other C# user that has a problem with that either. If that's not what you 
mean though, then could you elaborate?

 3. use of uninitialized variables (no reaching definition)
 3. Optimizer detects and reports it. Irrelevant for D, though, because 
 variables are always initialized. The =void case is rare enough to be 
 irrelevant.


D variable default-initialization is absolutely no different from your 
scenario of a programmer blindly tossing in =0 to shut up a compiler, 
*except* that the programmer is never even given the opportunity to do the 
right thing. This is *bad*. I *want* variables that haven't meen manually 
inited to be statically treated as uninited. C# does this and it works 
great.

 4. dead assignments (assignment of a value to a variable that is never 
 subsequently used)
 4. Dead assignments are automatically detected and removed. I'm not 
 convinced this should be reported, as it can legitimately happen when 
 generating source code. Generating false positives annoy the heck out of 
 users.


I'll agree with you here. But it might be nice to have an option to just 
simply report them anyway for when the programmer wants to see if there's 
any of these around that he can clean up.

 5. dead code (code that can never be executed)
 5. Dead code is detected and silently removed by optimizer. dmd front end 
 will complain about dead code.


Cool.

 6. array overflows
 6. Arrays are solidly covered by a runtime check. There is code in the 
 optimizer to detect many cases of overflows at compile time, but the code 
 is currently disabled because the runtime check covers 100% of the cases.


I'm puzzled by why you would prefer to leave this entirely runtime when some 
of it can be detected at compile-time. Normally you agree that catching 
something at compile-time is better whenever possible. So shouldn't the 
array overflows that can be detected at compile-time be detected at 
compile-time? I would certainly prefer that.

 7. proper pairing of allocate/deallocate function calls
 7. Not done because it requires the user to specify what the paired 
 functions are. Given this info, it is rather simple to graft onto existing 
 data flow analysis.


Interesting idea. D's scope(){} does help a lot, but I wouldn't be opposed 
to this extra protection to make sure I remember to use scope.

 8. improper use of signed integers (who knows what this actually is)
 8. D2 has acquired some decent checking for this.


Hooray for D :)

 There's a lot of hoopla about these static checkers, but I'm not impressed 
 by them based on what I can find out about them. What do you know about 
 what these checkers do that is not on this list?

Sorry, I don't know anything about them.

 Any other kinds of checking that would be great to implement?


http://d.puremagic.com/issues/show_bug.cgi?id=2197

Re: Should certain abstract classes be instantiable?

[Jeremie Pelletier]

Andrei Alexandrescu wrote:

Consider:

class A {
abstract void fun() {}
}

The class defines a function that is at the same time abstract (so it 
requires overriding in derivees) and has implementation.


Currently the compiler disallows creation of objects of type A, although 
technically that is feasible given that A defines the abstract method.


Should A be instantiable? What designs would that help or hinder?


Andrei


What's the point of marking fun() abstract if it has an implementation, 
I thought the compiler disallowed that.

Re: What does Coverity/clang static analysis actually do?

[Bill Baxter]

On Thu, Oct 1, 2009 at 1:38 PM, Nick Sabalausky a...@a.a wrote:

 2. possible dereference of NULL pointers (some reaching definitions of a
 pointer are NULL)
 2. Optimizer collects the info, but ignores this, because people are
 annoyed by false positives.


 If you mean something like this:

 Foo f;
 if(cond)
    f = new Foo();
 f.bar();

Probably more like

Foo f;
createAndSetOutParam(f);
f.bar();

or

Foo f;
if (a  10) { f = new Foo(10); }
...
if (a  10) {
   f.bar();
}

Or some complex intertwined gotos or any number of other things that
would be hard for the compiler to verify without a lot of effort.

I think I'd want to try this out as an optional warning for a while to
see how annoying the false positives really are in practice.  Still,
it seems like there's a subset of cases where it can be proved with
100% certainty that the code is wrong.  Just reporting those cases
would be a big win, I think.

 Then I *want* the compiler to tell me. C# does this and I've never been
 annoyed by it, in fact I've always appreciated it. I'm not aware of any
 other C# user that has a problem with that either. If that's not what you
 mean though, then could you elaborate?

How does C# handle the cases above?


 6. array overflows
 6. Arrays are solidly covered by a runtime check. There is code in the
 optimizer to detect many cases of overflows at compile time, but the code
 is currently disabled because the runtime check covers 100% of the cases.


 I'm puzzled by why you would prefer to leave this entirely runtime when some
 of it can be detected at compile-time. Normally you agree that catching
 something at compile-time is better whenever possible. So shouldn't the
 array overflows that can be detected at compile-time be detected at
 compile-time? I would certainly prefer that.

I agree. Run-time checks are only useful if the code is on a path that
actually runs.  Compile-time checks are useful even if the code is on
the branch not taken.

--bb

Re: Should certain abstract classes be instantiable?

[Andrei Alexandrescu]

Jeremie Pelletier wrote:

Andrei Alexandrescu wrote:

Consider:

class A {
abstract void fun() {}
}

The class defines a function that is at the same time abstract (so it 
requires overriding in derivees) and has implementation.


Currently the compiler disallows creation of objects of type A, 
although technically that is feasible given that A defines the 
abstract method.


Should A be instantiable? What designs would that help or hinder?


Andrei


What's the point of marking fun() abstract if it has an implementation, 
I thought the compiler disallowed that.


It may offer incomplete functionality that is to be reused and enhanced 
by descendants.


Andrei

Re: What does Coverity/clang static analysis actually do?

[Andrei Alexandrescu]

Walter Bright wrote:

[snip]

We need to have passable flow control to typecheck creation of immutable 
objects.


Andrei

Re: What does Coverity/clang static analysis actually do?

[David Gileadi]

Walter Bright wrote:
There's a lot of hoopla about these static checkers, but I'm not 
impressed by them based on what I can find out about them. What do you 
know about what these checkers do that is not on this list? Any other 
kinds of checking that would be great to implement?


I'm not familiar with C/C++ static checkers.

Eclipse's Java compiler has decent support for code checks.  I'm copying 
the list of items it can check here (except for those that seem 
Java-specific), in case it's of interest.  For each of the below, you 
can choose whether it's an error, a warning, or ignored.  I've included 
their defaults.


Code Style:

Non-static access to static member: Warning
Indirect access to static member: Ignored
Unqualified access to instance field: Ignored
Undocumented empty block: Ignored
Method with a constructor name: Warning
Parameter assignment: Ignored

Potential programming problems:

Assignment has no effect (e.g. 'x = x'): Warning
Possible accidental boolean assignment (e.g. 'if (a = b)'): Ignored
'finally' does not complete normally: Warning
Empty statement: Ignored
Hidden catch block: Warning
Inexact type match for vararg argments: Warning
Enum type constant not covered on 'switch': Ignored
'switch' case fall-through: Ignored
Null pointer access: Warning
Potential null pointer access: Ignored
Comparing identical values ('x == x'): Warning
Missing synchronized modifier on inherited method: Ignored
Class overrides 'equals()' but not 'hashCode()': Ignored
Dead code (e.g. 'if (false)'): Warning

Name shadowing and conflicts:

Field declaration hides another field or variable: Ignored
Local variable declaration hides another field or variable: Ignored
(If not ignored) Include constructor or setter method parameters
Type parameter hides another type: Warning
Method does not override package visible method: Warning
Interface method conflicts with protected 'Object' method: Warning

Unnecessary code:

Local variable is never read: Warning
Parameter is never read: Ignored
(If not ignored) Ignore in overriding and implementing methods
Unused import: Warning
Unused local or private member: Warning
Redundant null check: Ignored
Unnecessary 'else' statement: Ignored
Unnecessary cast or 'instanceof' operation: Ignored
Unused 'break' or 'continue' label: Warning
Redundant super interface: Ignored

I understand DMD's policy on warnings, but some of the above checks are 
reasons why I've grown to like some warnings.  Of course it helps that 
Eclipse's compiler is most often used with its IDE.


-Dave

Re: Should certain abstract classes be instantiable?

[dsimcha]

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
 Jeremie Pelletier wrote:
  Andrei Alexandrescu wrote:
  Consider:
 
  class A {
  abstract void fun() {}
  }
 
  The class defines a function that is at the same time abstract (so it
  requires overriding in derivees) and has implementation.
 
  Currently the compiler disallows creation of objects of type A,
  although technically that is feasible given that A defines the
  abstract method.
 
  Should A be instantiable? What designs would that help or hinder?
 
 
  Andrei
 
  What's the point of marking fun() abstract if it has an implementation,
  I thought the compiler disallowed that.
 It may offer incomplete functionality that is to be reused and enhanced
 by descendants.
 Andrei

If you are in that situation, then don't declare the class abstract.  I thought
the whole point of abstract classes was that they can't be instantiated.  If it
can be instantiated, then what does abstract even mean?

Re: What does Coverity/clang static analysis actually do?

[Walter Bright]

Nick Sabalausky wrote:

3. use of uninitialized variables (no reaching definition)
3. Optimizer detects and reports it. Irrelevant for D, though, because 
variables are always initialized. The =void case is rare enough to be 
irrelevant.




D variable default-initialization is absolutely no different from your 
scenario of a programmer blindly tossing in =0 to shut up a compiler, 
*except* that the programmer is never even given the opportunity to do the 
right thing. This is *bad*. I *want* variables that haven't meen manually 
inited to be statically treated as uninited. C# does this and it works 
great.


The difference is the maintenance programmer won't be left puzzling why 
there is an explicit assignment to the variable that is never used. The 
point to default initialization is consistency in the resulting 
behavior. Also, the optimizer will remove nearly all of the default 
initializers if they are dead assignments.


Anyhow, I think this issue was beaten to death in the previous thread on 
null dereference. I don't wish to divert this thread into rediscussing 
it, but rather stick with what other kinds of bug-detecting data flow 
analyses there are?



4. dead assignments (assignment of a value to a variable that is never 
subsequently used)
4. Dead assignments are automatically detected and removed. I'm not 
convinced this should be reported, as it can legitimately happen when 
generating source code. Generating false positives annoy the heck out of 
users.




I'll agree with you here. But it might be nice to have an option to just 
simply report them anyway for when the programmer wants to see if there's 
any of these around that he can clean up.


I congenitally dislike optional warnings, as I've pontificated at length 
about here before g. The problem is it makes for a wishy-washy 
definition of the language, and muddies what is legal versus illegal. D 
needs to advance the state of the art with clear thinking about what is 
legal and what isn't. Warnings and false positives are failures of 
language design.




6. array overflows
6. Arrays are solidly covered by a runtime check. There is code in the 
optimizer to detect many cases of overflows at compile time, but the code 
is currently disabled because the runtime check covers 100% of the cases.




I'm puzzled by why you would prefer to leave this entirely runtime when some 
of it can be detected at compile-time. Normally you agree that catching 
something at compile-time is better whenever possible. So shouldn't the 
array overflows that can be detected at compile-time be detected at 
compile-time? I would certainly prefer that.


Because when I implemented it I discovered that a compile time check 
rarely (if ever) caught actual bugs, the real problems could only be 
caught at runtime. Normally one uses things like foreach which 
automatically generate code that won't array overflow.




I generally regard as evil:

1. bugs that have erratic, random, irreproducible symptoms

2. source code that doesn't have an obvious purpose (this includes code 
inserted solely to suppress a false positive or warning)


I regard as undesirable:

3. wishy-washy warnings

4. overzealous compiler messages that are more akin to nagging than 
finding actual bugs

Re: Should certain abstract classes be instantiable?

[Andrei Alexandrescu]

dsimcha wrote:

== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article

Jeremie Pelletier wrote:

Andrei Alexandrescu wrote:

Consider:

class A {
abstract void fun() {}
}

The class defines a function that is at the same time abstract (so it
requires overriding in derivees) and has implementation.

Currently the compiler disallows creation of objects of type A,
although technically that is feasible given that A defines the
abstract method.

Should A be instantiable? What designs would that help or hinder?


Andrei

What's the point of marking fun() abstract if it has an implementation,
I thought the compiler disallowed that.

It may offer incomplete functionality that is to be reused and enhanced
by descendants.
Andrei


If you are in that situation, then don't declare the class abstract.  I thought
the whole point of abstract classes was that they can't be instantiated.  If it
can be instantiated, then what does abstract even mean?


They can't be instantiated.

Andrei

Re: What does Coverity/clang static analysis actually do?

[grauzone]

Walter Bright wrote:
Anyhow, I think this issue was beaten to death in the previous thread on 
null dereference. I don't wish to divert this thread into rediscussing 


I don't want to make this discussion even more complicated, but some 
people (such as me) like the existing default initialization behavior, 
and find the C# behavior annoying.  It doesn't really have to do with 
the null pointer problem either.


I especially like that it's consistent with the initialization of member 
variables in structs/classes.

Re: What does Coverity/clang static analysis actually do?

[Lutger]

Walter Bright wrote:

 Nick Sabalausky wrote:
 Walter Bright newshou...@digitalmars.com wrote in message
 2. possible dereference of NULL pointers (some reaching definitions of a
 pointer are NULL)
 2. Optimizer collects the info, but ignores this, because people are
 annoyed by false positives.

 
 If you mean something like this:
 
 Foo f;
 if(cond)
 f = new Foo();
 f.bar();
 
 Then I *want* the compiler to tell me. C# does this and I've never been
 annoyed by it, in fact I've always appreciated it. I'm not aware of any
 other C# user that has a problem with that either. If that's not what you
 mean though, then could you elaborate?
 
 The problem crops up when there are two connected variables:
 
void foo(bool flag)
{
  char* p = null;
  if (flag)
 p = hello;
  ...
  if (flag)
 bar(*p);
}
 
 The code is logically correct, there is no null pointer dereference
 possible. However, the data flow analysis will see the *p and see two
 reaching definitions for p: null and hello, even though only one
 actually reaches.
 
 Hence the false positive. To eliminate the false error report, the user
 would have to insert a redundant null check.
 
 Does this happen in practice? Yes.

How hard is this to implement? I ask this because I would suggest to try it 
out and see how much it catches vs. how annoying it is. In VB.NET I have 
quite some false positives, but in C# less. It's all about how it fits with 
the rest of the language. VB.NET doesn't have a ternary operator for 
example. In D you have less need for pointers and generally a much more 
expressive vocabulary at your disposal than other C family languages. 

Re: Should certain abstract classes be instantiable?

[Lutger]

Andrei Alexandrescu wrote:

 Jeremie Pelletier wrote:
 Andrei Alexandrescu wrote:
 Consider:

 class A {
 abstract void fun() {}
 }

 The class defines a function that is at the same time abstract (so it
 requires overriding in derivees) and has implementation.

 Currently the compiler disallows creation of objects of type A,
 although technically that is feasible given that A defines the
 abstract method.

 Should A be instantiable? What designs would that help or hinder?


 Andrei
 
 What's the point of marking fun() abstract if it has an implementation,
 I thought the compiler disallowed that.
 
 It may offer incomplete functionality that is to be reused and enhanced
 by descendants.
 
 Andrei

Then again, if it offers incomplete functionality why would you want it to 
be instantiable? Instantiable should mean you can use this guy no? I tried 
hard to think of a reason but can't find any.

 

Re: Can D compile for PowerPC Architecture?

[Jesse Phillips]

Snake Wrote:

 For some reason, about an hour ago I was googling stuff about X86 and PowerPC 
 and right now I'm not even sure if this is a serious question nor do I know 
 if I know what I'm talking about or if the question is even valid.
 
 If that doesn't make sense, I want to know if you guys are developing D for 
 use with the PowerPC Instruction Set
 
 The question: Can you program for PowerPC architecture using D? In my mind 
 I'm having these crazy ideas about things that are remotely relevant to 
 programming but...whatever _ This is me rambling at this point.

You will have to look at GDC and LDC.

http://www.dsource.org/projects/ldc/wiki/PlatformSupport

* LDC compiles, but bugs in frontend
* porting of GDC fixes suggested
* runtime, inline asm and exception handling need work
* contact: none 

Re: What does Coverity/clang static analysis actually do?

[BCS]

Hello Nick,


Walter Bright newshou...@digitalmars.com wrote in message
news:ha2run$275...@digitalmars.com...


2. possible dereference of NULL pointers (some reaching definitions
of a
pointer are NULL)
2. Optimizer collects the info, but ignores this, because people are
annoyed by false positives.

If you mean something like this:

Foo f;
if(cond)
f = new Foo();
f.bar();
Then I *want* the compiler to tell me. C# does this and I've never
been annoyed by it, in fact I've always appreciated it. I'm not aware
of any other C# user that has a problem with that either. If that's
not what you mean though, then could you elaborate?


I think it is important to note that the check DMD is not using checks for 
null where as C# checks for uninitialized


legal in c# but flagged by this test:

Foo f  = null;
if(cond) f = new Foo();
f.bar();


6. array overflows
6. Arrays are solidly covered by a runtime check. There is code in
the
optimizer to detect many cases of overflows at compile time, but the
code
is currently disabled because the runtime check covers 100% of the
cases.


I'm puzzled by why you would prefer to leave this entirely runtime
when some of it can be detected at compile-time. Normally you agree
that catching something at compile-time is better whenever possible.
So shouldn't the array overflows that can be detected at compile-time
be detected at compile-time? I would certainly prefer that.



Ditto on this; Walter, what is your rational on this?

Re: Should certain abstract classes be instantiable?

[Jeremie Pelletier]

Lutger wrote:

Andrei Alexandrescu wrote:


Jeremie Pelletier wrote:

Andrei Alexandrescu wrote:

Consider:

class A {
abstract void fun() {}
}

The class defines a function that is at the same time abstract (so it
requires overriding in derivees) and has implementation.

Currently the compiler disallows creation of objects of type A,
although technically that is feasible given that A defines the
abstract method.

Should A be instantiable? What designs would that help or hinder?


Andrei

What's the point of marking fun() abstract if it has an implementation,
I thought the compiler disallowed that.

It may offer incomplete functionality that is to be reused and enhanced
by descendants.

Andrei


Then again, if it offers incomplete functionality why would you want it to 
be instantiable? Instantiable should mean you can use this guy no? I tried 
hard to think of a reason but can't find any.




Exactly, I only mark members as abstract if they have no 
implementations, if any members have an incomplete implementation then i 
mark the class as abstract.


I never had any problems that way. Besides it just makes no sense to 
have an implemented member as abstract.

Re: Should certain abstract classes be instantiable?

[Jarrett Billingsley]

On Thu, Oct 1, 2009 at 4:30 PM, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
 Consider:

 class A {
    abstract void fun() {}
 }

 The class defines a function that is at the same time abstract (so it
 requires overriding in derivees) and has implementation.

 Currently the compiler disallows creation of objects of type A, although
 technically that is feasible given that A defines the abstract method.

 Should A be instantiable? What designs would that help or hinder?

Uh... why?

Re: What does Coverity/clang static analysis actually do?

[Brad Roberts]

On Thu, 1 Oct 2009, Walter Bright wrote:

 I've been interested in having the D compiler take advantage of the flow
 analysis in the optimizer to do some more checking. Coverity and clang get a
 lot of positive press about doing this, but any details of exactly *what* they
 do have been either carefully hidden (in Coverity's case) or undocumented
 (clang's page on this is blank). All I can find is marketing hype and a lot of
 vague handwaving.
 
 Here is what I've been able to glean from much time spent with google on what
 they detect and my knowledge of how data flow analysis works:
 

Snipped a lot of the detail, because that's not really what makes the 
tools interesting.  There's a couple things that do, im my opinion -- with 
a little experience having used Fortify and looked at Coverity a couple 
times over the years (and would be using if it wasn't so much more 
expensive than Fortify).

1) Rich flow control.  They go well beyond what's typically done by 
compiliers during their optimization passes.  They tend to be whole-code 
in scope and actually DO the parts that are hard, like cross expression 
variable value tracking similar to a couple examples in this thread.  
Function boundaries are no obstacle to them.  The only obstacle is where 
source isn't provided.

2) Due to working with whole source bases, the UI for managing the data 
produced is critical to overall usability.  A lot of time goes into making 
it easy to manage the output.. both for single runs and for cross-run flow 
of data.  Some examples:

   * suppression of false positives, 
   * graphing of issue trends
   * categorization of issue types

3) Rule creation.  The core engine usually generates some digested dataset 
upon rules are evaluated.  The systems come with a builtin set that do the 
sorts of things already talked about.  In addition they come with the 
ability to develop new rules specific to your application and business 
needs.  For example:

   * tracking of taint from user data
   * what data is acceptable to log to files (for example NOT credit-cards)

4) They're expected to be slower than compilation, so it's ok to do things 
that are computationally prohibitive to do during compilation cycles.



I've seen these tools detect some amazing subtle bugs in c and c++ code.  
They're particularly handy in messy code.   They can help find memory 
leaks where the call graphs are arbitrarily obscure.  Sites where NULL 
pointers are passed into a function that dereferences without a null check 
even when the call graph has many layers.

Yes, rigid contract systems and richer type systems can help reduce the 
need for some of these sorts of checks, but as we all know, there's 
tradeoffs.


That help?

Later,
Brad

Re: It's official: One-day D tutorial at the ACCU Conference 2010 in Oxford, England

[Graham St Jack]

I would love to get my hands on the transcript and video of the event...
Will it be recorded?

Re: Defining some stuff for each class in turn

[Phil Deets]

On Thu, 01 Oct 2009 12:53:46 -0500, Andrei Alexandrescu  
seewebsiteforem...@erdani.org wrote:


interface Comparator(T)
{
 int opCmp(Comparator!T rhs);
 mixin(Impl) // for each implementation Impl
 {
 int opCmp(Impl rhs);
 override int opCmp(Comparator!T rhs)
 {
 return opCmp(cast(Impl) rhs);
 }
 }
}



I like that you can use this in interfaces too. It seems to allow for  
something similar to multiple inheritance, but presumably without the  
implementation difficulty.


Phil Deets

Re: Should certain abstract classes be instantiable?

[Lionello Lunesu]

On 2-10-2009 4:30, Andrei Alexandrescu wrote:

Consider:

class A {
abstract void fun() {}
}

The class defines a function that is at the same time abstract (so it
requires overriding in derivees) and has implementation.

Currently the compiler disallows creation of objects of type A, although
technically that is feasible given that A defines the abstract method.

Should A be instantiable? What designs would that help or hinder?


Andrei


If it were instantiable, what would be the difference between abstract 
and virtual?


L.

Re: Should certain abstract classes be instantiable?

[Andrei Alexandrescu]

Jarrett Billingsley wrote:

On Thu, Oct 1, 2009 at 4:30 PM, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:

Consider:

class A {
   abstract void fun() {}
}

The class defines a function that is at the same time abstract (so it
requires overriding in derivees) and has implementation.

Currently the compiler disallows creation of objects of type A, although
technically that is feasible given that A defines the abstract method.

Should A be instantiable? What designs would that help or hinder?


Uh... why?


Because I want to give a good argument one way or another in TDPL. FWIW, 
I can't imagine why you'd ever... or Never needed that are not 
strong enough arguments.


Andrei

Re: Defining some stuff for each class in turn

[Andrei Alexandrescu]

Phil Deets wrote:
On Thu, 01 Oct 2009 12:53:46 -0500, Andrei Alexandrescu 
seewebsiteforem...@erdani.org wrote:


interface Comparator(T)
{
 int opCmp(Comparator!T rhs);
 mixin(Impl) // for each implementation Impl
 {
 int opCmp(Impl rhs);
 override int opCmp(Comparator!T rhs)
 {
 return opCmp(cast(Impl) rhs);
 }
 }
}



I like that you can use this in interfaces too. It seems to allow for 
something similar to multiple inheritance, but presumably without the 
implementation difficulty.


Phil Deets


That's a great point. To compare this feature with Scala's mixins (which 
I studied and now feel are a good feature), my understanding is that 
Scala mixins are an interface plus a default implementation. The feature 
proposed above is an interface with a prescribed implementation. That 
way, Scala mixins seem to be somewhat more flexible.


Andrei

Re: It's official: One-day D tutorial at the ACCU Conference 2010 in Oxford, England

[Andrei Alexandrescu]

Graham St Jack wrote:

I would love to get my hands on the transcript and video of the event...
Will it be recorded?


ACCU has not recorded its conferences in the past.

Andrei

Re: Defining some stuff for each class in turn

[Christopher Wright]

Andrei Alexandrescu wrote:
I am becoming increasingly aware that we need to provide some means to 
define certain members (data and functions) for each class as if they 
were pasted there.


Most of the examples given would be well served by decent builtin 
reflection. Walter thinks reflection is too expensive to be active by 
default. Find a cheaper way to provide runtime reflection. It's not as 
sexy as using templates, but it's DRY and easier to use. Slower, but you 
don't pay the cost of reflection multiple times if you have multiple 
libraries requiring reflection.

Re: Defining some stuff for each class in turn

[Andrei Alexandrescu]

Christopher Wright wrote:

Andrei Alexandrescu wrote:
I am becoming increasingly aware that we need to provide some means to 
define certain members (data and functions) for each class as if they 
were pasted there.


Most of the examples given would be well served by decent builtin 
reflection. Walter thinks reflection is too expensive to be active by 
default. Find a cheaper way to provide runtime reflection. It's not as 
sexy as using templates, but it's DRY and easier to use. Slower, but you 
don't pay the cost of reflection multiple times if you have multiple 
libraries requiring reflection.


What cheaper way would be than allowing a base class to prescribe 
reflection for its hierarchy? Where do templates even enter the mix? 
What's slower and why? Why do reflection as a language feature 
(increases base language size, buggy, rigid) instead of allowing it as a 
library if we so can? I'm totally against that.


Andrei

Re: Should certain abstract classes be instantiable?

[Jarrett Billingsley]

On Thu, Oct 1, 2009 at 8:49 PM, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
 Jarrett Billingsley wrote:

 On Thu, Oct 1, 2009 at 4:30 PM, Andrei Alexandrescu
 seewebsiteforem...@erdani.org wrote:

 Consider:

 class A {
   abstract void fun() {}
 }

 The class defines a function that is at the same time abstract (so it
 requires overriding in derivees) and has implementation.

 Currently the compiler disallows creation of objects of type A, although
 technically that is feasible given that A defines the abstract method.

 Should A be instantiable? What designs would that help or hinder?

 Uh... why?

 Because I want to give a good argument one way or another in TDPL. FWIW, I
 can't imagine why you'd ever... or Never needed that are not strong
 enough arguments.

But.. you mark something abstract when you want it to be .. abstract.
How would you argue that abstract is basically a no-op when used on
methods with bodies? And there's a reasonable use for it, so why
suddenly allow something that.. doesn't really make all that much
sense to begin with?

Re: restructuring name hiding around the notion of hijacking

[Michel Fortin]

On 2009-10-01 12:29:39 -0400, Andrei Alexandrescu 
seewebsiteforem...@erdani.org said:


I think it's a good idea, but there should be a way to *override* 
static functions.


That has the same risks. The problem right now is that in order to use 
a class, you must absorb the definition of that class and that of each 
superclass of it, all the way up to Object. With hijacking thwarted, 
you can specify stuff in the base class that you can be sure will 
continue to work the same in derived classes. I believe this makes 
using classes quite a lot easier and safer.


But it breaks one pattern of mine. In the D/Objective-C bridge I have a 
few static functions and variables that must be redefined for each 
subclass defining an Objective-C interface. With your proposal I'd have 
to give them a different name for each subclass.


For instance, the objcClass function in:

NSObject.objcClass

will give you a pointer to the Objective-C class NSObject, while in:

NSString.objcClass

it will give you a pointer to the Objective-C class NSString, because 
objcClass has been reimplemented in the D version of the NSString class 
even though it derives from NSObject which has its own.


If you can't override a static function, how do you implement this?

I'd suggest that a static function could be made final which would 
remove the possibility of redefining it in a subclass. But in abscence 
of final, you should still be able to override a static function in 
a subclass (perhaps the override keyword should be required).


--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: Defining some stuff for each class in turn

[Michel Fortin]

On 2009-10-01 14:05:10 -0400, Max Samukha spam...@d-coding.com said:


For example, the upcoming reflection mechanism in QtD (which uses its
own metaobjects instead of the limited classinfo) requires manual
insertion of a mixin into derived classes unless the mixin is inserted
implicitly when a signal/property/etc is declared.


I'm using the exact same solution here for the D/Objective-C bridge and 
it's working surprisingly well.


I certainly wouldn't turn down the possibility of inheriting an 
implementation in subclasses, it'd make things a little less hackish.



--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: What does Coverity/clang static analysis actually do?

[Nick Sabalausky]

 Walter Bright newshou...@digitalmars.com wrote in message 
news:ha37cf$4l...@digitalmars.com...
 Nick Sabalausky wrote:
 3. use of uninitialized variables (no reaching definition)
 3. Optimizer detects and reports it. Irrelevant for D, though, because 
 variables are always initialized. The =void case is rare enough to be 
 irrelevant.


 D variable default-initialization is absolutely no different from your 
 scenario of a programmer blindly tossing in =0 to shut up a compiler, 
 *except* that the programmer is never even given the opportunity to do 
 the right thing. This is *bad*. I *want* variables that haven't meen 
 manually inited to be statically treated as uninited. C# does this and it 
 works great.

 The difference is the maintenance programmer won't be left puzzling why 
 there is an explicit assignment to the variable that is never used.

Why would there be one? If a value is assigned but never used, then a must 
assign value before reading rule wouldn't have tripped over it before the 
initial programmer wrote the assignment in the first place.

 The point to default initialization is consistency in the resulting 
 behavior.

Yes, I'm aware that D's default-initing came about as a better alternative 
to C's let's introduce unreliable randomness approach. But we're way past 
that and no one's comparing it to C. But more importantly, it does nothing 
to address my argument that compares it to a more C#-way. I'll explain with 
a scenario you may initially find very familiar:

Johnny is using a language that statically requires variables to have been 
verifiably written to before they can be read. Johnny compiles and gets the 
error Var 'x' read without having been written to. Johnny jerks his knee 
and blindly throws in =0. Bad Johnny. Bad, bad Johnny. scolds Kenneth 
Compiler Writer quite justifiably. Two seconds later, Kenneth changes the 
compiler to blindly throw in =0. Kenneth is a big hypocrite. Johnny smiles 
because his bad behavior is now automated.

Notice that story has no relevance to C's way of handling (or rather 
ignoring) uninitialized vars. It's D-style vs C#-style, not D-style vs 
C-style. So can you please address how the current D-style is supposed to be 
better than *C#-style*, rather than once again telling us it's better than 
C-style. We know it's better than C-style. That's not the issue.

 Also, the optimizer will remove nearly all of the default initializers if 
 they are dead assignments.

 Anyhow, I think this issue was beaten to death in the previous thread on 
 null dereference. I don't wish to divert this thread into rediscussing it, 
 but rather stick with what other kinds of bug-detecting data flow analyses 
 there are?


 4. dead assignments (assignment of a value to a variable that is never 
 subsequently used)
 4. Dead assignments are automatically detected and removed. I'm not 
 convinced this should be reported, as it can legitimately happen when 
 generating source code. Generating false positives annoy the heck out of 
 users.


 I'll agree with you here. But it might be nice to have an option to just 
 simply report them anyway for when the programmer wants to see if there's 
 any of these around that he can clean up.

 I congenitally dislike optional warnings, as I've pontificated at length 
 about here before g. The problem is it makes for a wishy-washy 
 definition of the language, and muddies what is legal versus illegal.

The way you've implemented them (as errors rather than real warnings), yes, 
it absolutely does that, as I've pontificated about at length g.

 D needs to advance the state of the art with clear thinking about what is 
 legal and what isn't. Warnings and false positives are failures of 
 language design.


If you define a failure of language design to be anything that isn't total 
perfection, then sure.

Warnings and false positives are measures that are taken to mitigate 
problems from the *inevitable* imperfections. Many of the imperfections in 
language design are like cliffs, they're potentially hazardous and you can't 
always *realistically* eliminate them. So when we can't realistically 
eliminate them, what we do is we swallow our pride and we put up a railing, 
warning and even a potential false positive.

In fact, D has *already* conceded to being imperfect and accordingly added 
false positives and...well..something warning-like (but more problematic):

- There *are* situations where an implicit narrowing cast would be perfectly 
OK. But does D warn about these false positives for the sake of the 
narrowing casts that aren't OK? Absolutely. And in these cases it forces the 
programmer to cast...but what does cast(byte)someInt really do if not your 
Evil #2 below?

- No return at end of function. It should be pretty clear why a non-void 
function with no return is potentially hazardous. It's also clear that there 
are certain times it would just happen to work out just fine, for instance, 
to just assume 0 or something. 

Re: Should certain abstract classes be instantiable?

[Andrei Alexandrescu]

Jarrett Billingsley wrote:

On Thu, Oct 1, 2009 at 8:49 PM, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:

Jarrett Billingsley wrote:

On Thu, Oct 1, 2009 at 4:30 PM, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:

Consider:

class A {
  abstract void fun() {}
}

The class defines a function that is at the same time abstract (so it
requires overriding in derivees) and has implementation.

Currently the compiler disallows creation of objects of type A, although
technically that is feasible given that A defines the abstract method.

Should A be instantiable? What designs would that help or hinder?

Uh... why?

Because I want to give a good argument one way or another in TDPL. FWIW, I
can't imagine why you'd ever... or Never needed that are not strong
enough arguments.


But.. you mark something abstract when you want it to be .. abstract.
How would you argue that abstract is basically a no-op when used on
methods with bodies?


It's not a no-op. Try it.

Andrei

Re: restructuring name hiding around the notion of hijacking

[Andrei Alexandrescu]

Michel Fortin wrote:
On 2009-10-01 12:29:39 -0400, Andrei Alexandrescu 
seewebsiteforem...@erdani.org said:


I think it's a good idea, but there should be a way to *override* 
static functions.


That has the same risks. The problem right now is that in order to use 
a class, you must absorb the definition of that class and that of each 
superclass of it, all the way up to Object. With hijacking thwarted, 
you can specify stuff in the base class that you can be sure will 
continue to work the same in derived classes. I believe this makes 
using classes quite a lot easier and safer.


But it breaks one pattern of mine. In the D/Objective-C bridge I have a 
few static functions and variables that must be redefined for each 
subclass defining an Objective-C interface.


I'd say that's a questionable practice (but then I don't know any more 
details).


Andrei

Re: What does Coverity/clang static analysis actually do?

[Brad Roberts]

Walter Bright wrote:
 Brad Roberts wrote:
 1) Rich flow control.  They go well beyond what's typically done by
 compiliers during their optimization passes.  They tend to be
 whole-code in scope and actually DO the parts that are hard, like
 cross expression variable value tracking similar to a couple examples
 in this thread.  Function boundaries are no obstacle to them.  The
 only obstacle is where source isn't provided.
 
 Modern compiler optimizers (including dmc and dmd) DO do
 cross-expression variable tracking. They just don't often do it
 inter-function (called inter-procedural analysis) because of time and
 memory constraints, not that it is technically more difficult.

Exactly my point.. compilers tend to make the opposite trade off that tools like
Coverity do.  Not that compilers can't or don't.  They just usually do a small
subset of what is possible in the grander sense of 'possible'.

 C and C++ do have some issues with inter-procedural analysis because the
 compiler view of them, at heart, is single file. D is much more
 conducive to that because the compiler sees an arbitrarily large part of
 the source code, and can obviously see all of it if the user so desires.

Neither c nor c++ as languages are _required_ to do single file compilation.
That's just what most compilers do.  In fact, gcc/g++ have been capable of doing
whole-app compilation for a couple years now -- though not many people use it
that way as far as I can tell.  See also, exact same issue as above.. it's a
trade off.

 2) Due to working with whole source bases, the UI for managing the
 data produced is critical to overall usability.  A lot of time goes
 into making it easy to manage the output.. both for single runs and
 for cross-run flow of data.  Some examples:

* suppression of false positives, 
 
 I'd rather do a better job and not have false positives.

Of course you would.. everyone would.  It's a meaningless statement since no one
would ever contradict it with any seriousness.  At the risk of repeating
myself.. same tradeoffs again.

* graphing of issue trends
 
 That's a crock g.

Uh, whatever.  Most of the rest of us humans respond much better to pictures and
trends than to raw numbers.  Show me some visual indication of the quality of my
code (ignoring the arguments about the validity of such graphs) and I can pretty
much guarantee that I'll work to improve that measure.  Nearly everyone I've
ever worked with behaves similarly.. once they agree that the statistic being
measured is useful.  One of the best examples is percent of code covered by unit
tests.  The same applies to number of non-false positive issues discovered
through static analysis.

A specific case of this:  At informix we had a step in our build process that
ran lint (yes, it's ancient, but this was a decade ago and the practice was at
least a decade old before I got there).  Any new warnings weren't tolerated.
The build automatically reported any delta over the previous build.  It was
standard practice and kept the code pretty darned clean.

* categorization of issue types
 
 I'm not convinced that is of critical value.

You don't need to be.  You view too many things in black/white terms.

 3) Rule creation.  The core engine usually generates some digested
 dataset upon rules are evaluated.  The systems come with a builtin set
 that do the sorts of things already talked about.  In addition they
 come with the ability to develop new rules specific to your
 application and business needs.  For example:

* tracking of taint from user data
* what data is acceptable to log to files (for example NOT
 credit-cards)
 
 There have been several proposals for user-defined attributes for types,
 I think that is better than having some external rule file.

Again, this is where trade offs come in.  If it can be done cheaply enough to
warrant being done during compilation and is accurate enough in small scoped
analysis.. yay.  But sometimes you still want to do things that take more time
and more completely.

 4) They're expected to be slower than compilation, so it's ok to do
 things that are computationally prohibitive to do during compilation
 cycles.
 
 I agree.
 
 

 I've seen these tools detect some amazing subtle bugs in c and c++
 code.  They're particularly handy in messy code.   They can help find
 memory leaks where the call graphs are arbitrarily obscure.  Sites
 where NULL pointers are passed into a function that dereferences
 without a null check even when the call graph has many layers.
 
 Once you get the data flow analysis equations right, they'll detect it
 every time regardless of how subtle or layered the call graph is.
 
 Yes, rigid contract systems and richer type systems can help reduce
 the need for some of these sorts of checks, but as we all know,
 there's tradeoffs.


 That help?
 
 Yes, very much. In particular, I wasn't sure coverity did
 inter-procedural analysis.

That's essentially all it does, if you 

Re: What does Coverity/clang static analysis actually do?

[Walter Bright]

Nick Sabalausky wrote:
If you accept the idea of a compiler (like DMD) having rudimentary built-in 
optional versions of normally separate tools like profiling, unittesting, 
doc generation, etc., and you accept that lint tools are perfectly fine 
tools to use (as I think you do, or am I mistaken?), then I don't see what 
would make lint tools an exception to the built-ins are ok attitude 
(especially since a separate one would require a lot of redundant 
parsing/analysis.)


This is a more general comment on your post (and similar ones by others, 
it's a recurring theme):


Consider the Bible. It's long and complicated, and by careful 
examination of it you can find a verse here and there to justify *any* 
behavior.


D is complicated, and is founded on principles that are not orthogonal - 
they are often at odds with each other. Any attempt to take one 
particular aspect of D's behavior and use it as a rule to impose 
elsewhere is surely doomed to conflict with some other rule.


The only reasonable way forward is to evaluate each idea not only in 
terms of all of D's principles, but also on its own merits, and throw in 
one's best judgment.


Nearly a decade with D has now shown that some ideas and choices were 
dead wrong, but others were more right than I even dreamed g.

Re: Access original data from array

[Moritz Warning]

On Thu, 01 Oct 2009 05:02:10 +0100, Tom S wrote:

 Moritz Warning wrote:
 Hi,
 
 how can I access the original value for xs?
[..]
 
 You were probably looking for the old meaning of .init, but it's gone
 now, so I present these alternative fixes:
 
 1) You need to load the state of the game from before overriding xs. I
 recommend Quick Save and Quick Load. Often bound to F5 and F9.
I tried F5 and F9, but I only accidently activated the bonus level.
Quirky game...

 2) Perhaps a custom-fit Delorian will do.
- A FluxCapacitator would be an overkill solution.
- I don't do coding while driving at  88Mph.

 
 3) If all else fails, I'm afraid you'll have to resort to copying the
 contents of xs prior to overwriting them.
I will give it a try.

Thanks!

Re: implicit ubyte casting

[Jeremie Pelletier]

Saaa wrote:

I think is very bug-prone, isn't it obvious iub should be -5?

ubyte ub = 5;
int iub = -ub; // iub now is 251

What is the reasoning to do it this way? 


Minus toggles the most significant bit, be it on a signed or unsigned 
type. When converting it to an int, the byte being signed or unsigned 
does make a difference: when unsigned the number is copied as is, when 
signed the most significant bit (bit 7) is shifted to the most 
significant bit of the int (bit 31).


Its therefore pretty standard logic, no warning is given since the 
entire ubyte range fits within an int


Jeremie

Re: implicit ubyte casting

[Jeremie Pelletier]

Saaa wrote:

Jeremie Pelletier wrote

Saaa wrote:

I think is very bug-prone, isn't it obvious iub should be -5?

ubyte ub = 5;
int iub = -ub; // iub now is 251

What is the reasoning to do it this way?
Minus toggles the most significant bit, be it on a signed or unsigned 
type. When converting it to an int, the byte being signed or unsigned does 
make a difference: when unsigned the number is copied as is, when signed 
the most significant bit (bit 7) is shifted to the most significant bit of 
the int (bit 31).


Its therefore pretty standard logic, no warning is given since the entire 
ubyte range fits within an int


Jeremie

Thanks, but it is not that I do not know how it occurs more that
I should have asked whether people use this kind of logic.
For me it resulted in annoying bug like this:
for(int i = nloop;i10;i++);//ubyte nloop is created quite a few lines 
above.


Then why use an ubyte instead of a byte or an int?

You could also just do:
for(int i = cast(byte)nloop; i  10; i++)

Jeremie

Re: implicit ubyte casting

[Saaa]

Jeremie Pelletier wrote

 Then why use an ubyte instead of a byte or an int?
I wasn't me who wrote that part of the code :)


 You could also just do:
 for(int i = cast(byte)nloop; i  10; i++)
I forgot the minus sign:
for(int i = -cast(int)nloop;i 10; i++)

Still think it is unnecessary bug-prone.

Re: implicit ubyte casting

[Saaa]

Moritz Warning wrote

 This is a troublesome behavior:

 ubyte z = 5;
 int x = -z; // x now is 251
 int y = -1 * z; // y is now -5

Yes, troublesome is the correct word :)
Does anybody ever use the =-z behaviour? 

Re: implicit ubyte casting

[bearophile]

Saaa:
Does anybody ever use the =-z behaviour?

Sometimes C programmers use something like:
unsigned int x = -1;

The interaction of signed-unsigned integral numbers in D is very error-prone, 
so much that I suggest to use unsigned integrals in D only where strictly 
necessary (generally when you need bitfields for bitwise operations (so not for 
arithmetic operations), or the less common situations where you need the full 
range of 1,2,4,8 bytes).

Sometimes I even cast array lengths to an int and then I keep and use only such 
int around because in D that's safer than using a size_t (example: if you 
compare an unsigned int length with a negative int, your code will have a bug).

I have discussed such topics several times in the main D newsgroup, and in the 
end no good solution has being found/accepted so far. But eventually some 
better solution must be found...

Bye,
bearophile

Re: implicit ubyte casting

[Jarrett Billingsley]

On Thu, Oct 1, 2009 at 2:00 PM, bearophile bearophileh...@lycos.com wrote:

 I have discussed such topics several times in the main D newsgroup, and in 
 the end no good solution has being found/accepted so far. But eventually some 
 better solution must be found...

Fucking A, bearophile. Bugzilla. How many fucking times do we have to tell you.

Re: implicit ubyte casting

[Brad Roberts]

On Thu, 1 Oct 2009, Saaa wrote:

 I think is very bug-prone, isn't it obvious iub should be -5?
 
 ubyte ub = 5;
 int iub = -ub; // iub now is 251
 
 What is the reasoning to do it this way? 

The inclusion of the 'int' part obscures what I think the real problem 
is.. 

   Does it make sense to use uniary-minus on a unsigned type?

My answer.. no.  But the counter argument that will likely come up is 
generic behavior.  So, to prempt that.. does unary minus have any useful 
meaning for MOST types?  My answer is still no. :)

Later,
Brad

[Issue 2942] asm fadd; accepted, but generates code for faddp.

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=2942


Walter Bright bugzi...@digitalmars.com changed:

   What|Removed |Added

   Severity|normal  |minor


--- Comment #3 from Walter Bright bugzi...@digitalmars.com 2009-09-30 
23:03:03 PDT ---
These pseudo-ops *are* documented in older Intel manuals, like the 387 DX
User's Manual. I'm reluctant to change it. The last issue should be in a
separate report.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 2942] asm fadd; accepted, but generates code for faddp.

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=2942


Don clugd...@yahoo.com.au changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||INVALID


--- Comment #4 from Don clugd...@yahoo.com.au 2009-10-01 00:27:56 PDT ---
(In reply to comment #3)
 These pseudo-ops *are* documented in older Intel manuals, like the 387 DX
 User's Manual. I'm reluctant to change it. The last issue should be in a
 separate report.

Interesting. They aren't present in any manual which is still available. I
found a website with material which was copied from the 386 manual (not 387), 
but it said that even in 1997, the manual was no longer officially available.
I suspect that a lot of those pseudo-ops were bugs in DEBUG. (DEBUG also
accepts fld addr, ST(6);).

However, I just checked MSVC, and it _does_ accept fadd;
(But it doesn't accept the legal faddp; !!)
Pretty useless, and I think they should be abandoned, but no big deal if you
want to keep them.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3354] New: asm fld x, ST(6); accepted.

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3354

   Summary: asm fld x, ST(6); accepted.
   Product: D
   Version: 1.00
  Platform: x86
OS/Version: All
Status: NEW
  Keywords: accepts-invalid
  Severity: minor
  Priority: P2
 Component: DMD
AssignedTo: nob...@puremagic.com
ReportedBy: clugd...@yahoo.com.au


--- Comment #0 from Don clugd...@yahoo.com.au 2009-10-01 00:29:53 PDT ---
This garbage compiles, but shouldn't.


void main() {
 double x;
  asm {
  fld x, ST(6);
 }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3355] New: std.string.cmp works incorrectly for mixed-type and different-length strings

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3355

   Summary: std.string.cmp works incorrectly for mixed-type and
different-length strings
   Product: D
   Version: 2.032
  Platform: x86
OS/Version: Linux
Status: NEW
  Keywords: patch
  Severity: normal
  Priority: P2
 Component: Phobos
AssignedTo: nob...@puremagic.com
ReportedBy: acehr...@yahoo.com


--- Comment #0 from Ali Cehreli acehr...@yahoo.com 2009-10-01 01:08:40 PDT ---
cmp fails unit tests when added this one

result = cmp(aa, aaad);
assert(result  0);


The patch is trivial:

109c109
 if (i1 == s1.length) return s2.length - i2;
---
 if (i1 == s1.length) return i2 - s2.length;
136a137,143
 
 result = cmp(aa, aaad);
 assert(result  0);
 result = cmp(aaa, aad);
 assert(result  0);
 result = cmp(aa, aad);
 assert(result == 0);

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3355] std.string.cmp works incorrectly for mixed-type and different-length strings

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3355


Andrei Alexandrescu and...@metalanguage.com changed:

   What|Removed |Added

 Status|NEW |ASSIGNED
 CC||and...@metalanguage.com


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3356] New: Make pure functions require immutable parameters

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3356

   Summary: Make pure functions require immutable parameters
   Product: D
   Version: future
  Platform: All
OS/Version: All
Status: NEW
  Severity: enhancement
  Priority: P2
 Component: DMD
AssignedTo: nob...@puremagic.com
ReportedBy: dfj1es...@sneakemail.com


--- Comment #0 from Sobirari Muhomori dfj1es...@sneakemail.com 2009-10-01 
07:39:29 PDT ---
Illustration:
---
int foo(const int[] bar) pure
{
return bar[1];
}

void goo()
{
int[2] a;
a[1]=1;
foo(a);
a[1]=2;
foo(a);
}
---

1. This doesn't affect functions with value type parameters.
2. When a function takes a reference type parameter, the chanses are slim, that
the return value doesn't depend on the referenced data. So the referenced data
must be immutable.
3. This doesn't require complex flow analysis, only a formal function signature
check.
4. (??) Replace immutability of explicit pointer type with constness, since
even if the referenced data is immutable, the code doesn't know, where the
immutable data ends and can access subsequent possibly mutating data. This will
instantly make any function, taking a pointer, impure. This should not apply to
objects and byref data.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3352] RangeError in std.conv

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3352



--- Comment #1 from Sobirari Muhomori dfj1es...@sneakemail.com 2009-10-01 
07:43:35 PDT ---
Also I think, it's an overkill to fine tune allocation as it's done in
to!string(uint)

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3356] Make pure functions require immutable parameters

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3356


Don clugd...@yahoo.com.au changed:

   What|Removed |Added

 CC||clugd...@yahoo.com.au


--- Comment #1 from Don clugd...@yahoo.com.au 2009-10-01 08:09:24 PDT ---
(In reply to comment #0)
 2. When a function takes a reference type parameter, the chanses are slim, 
 that
 the return value doesn't depend on the referenced data. 

Yes.

 So the referenced data must be immutable.

That conclusion does not follow. I don't think you're seeing all of the
benefits of 'pure'.
Consider foo(a) + foo(a). This can be changed into 2*foo(a), even though a is
not immutable.

It is true that in the case where all parameters are immutable, additional
optimisations (such as caching) can be performed. But there's more to pure than
that.

 4. (??) Replace immutability of explicit pointer type with constness, since
 even if the referenced data is immutable, the code doesn't know, where the
 immutable data ends and can access subsequent possibly mutating data. This 
 will
 instantly make any function, taking a pointer, impure. This should not apply 
 to
 objects and byref data.

That's a memory integrity issue, not a purity issue. That could only happen in
an unsafe module.

You are asking for a feature to be removed from the language, but I'm not
really sure why.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3357] New: ICE with aa that use static char array as key

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3357

   Summary: ICE with aa that use static char array as key
   Product: D
   Version: 1.047
  Platform: Other
OS/Version: Mac OS X
Status: NEW
  Severity: normal
  Priority: P2
 Component: DMD
AssignedTo: nob...@puremagic.com
ReportedBy: fa...@gmx.ch


--- Comment #0 from Fawzi Mohamed fa...@gmx.ch 2009-10-01 08:05:44 PDT ---
I was trying to reduce an error, namely
  Internal error: e2ir.c 4026
and I generated another one
{{{
struct Particle{
char[16] name;
}

class ReadSystem{
size_t[char[16]] pKindsIdx;

void t(Particle p){
auto idx=p.name in pKindsIdx; // fails (Internal error: ../ztc/cod1.c
2636)
}
}

void main(){
char[16] n;
size_t[char[16]] aa;
auto r=n in aa; // works
}
}}}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3357] ICE with aa that use static char array as key

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3357


Don clugd...@yahoo.com.au changed:

   What|Removed |Added

 CC||clugd...@yahoo.com.au


--- Comment #1 from Don clugd...@yahoo.com.au 2009-10-01 08:11:41 PDT ---
Is this the same as bug 1934?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 1934] ICE(e2ir.c) using static array as AA key

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=1934


Fawzi Mohamed fa...@gmx.ch changed:

   What|Removed |Added

 CC||fa...@gmx.ch


--- Comment #4 from Fawzi Mohamed fa...@gmx.ch 2009-10-01 08:21:28 PDT ---
*** Issue 3357 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3357] ICE with aa that use static char array as key

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3357


Fawzi Mohamed fa...@gmx.ch changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||DUPLICATE


--- Comment #2 from Fawzi Mohamed fa...@gmx.ch 2009-10-01 08:21:28 PDT ---
Yes it looks like it, the line number changed (probably due to changes in the
code) and I hadn't found it, but it really looks like the same issue.

*** This issue has been marked as a duplicate of issue 1934 ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3357] ICE with aa that use static char array as key

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3357



--- Comment #4 from Fawzi Mohamed fa...@gmx.ch 2009-10-01 08:35:40 PDT ---
thanks for catching it...

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 2998] ICE(expression.c) with floating point enum

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=2998


Walter Bright bugzi...@digitalmars.com changed:

   What|Removed |Added

 CC||bugzi...@digitalmars.com


--- Comment #4 from Walter Bright bugzi...@digitalmars.com 2009-10-01 
10:42:10 PDT ---
The order in which those two functions are called shouldn't matter. The actual
problem is the TypeEnum doesn't have proper overrides for isreal, isimaginary,
etc. Will fix.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 3358] New: Several typeof(Class.method).stringof incoherence

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=3358

   Summary: Several typeof(Class.method).stringof incoherence
   Product: D
   Version: 2.032
  Platform: x86
OS/Version: Linux
Status: NEW
  Severity: major
  Priority: P2
 Component: DMD
AssignedTo: nob...@puremagic.com
ReportedBy: jul...@onandon.be


--- Comment #0 from Julien Leclercq jul...@onandon.be 2009-10-01 15:25:31 PDT 
---
Hello,

   the results of `typeof(Class.method).stringof` for different methods
sharing the same signature but different parameters name seems identical.

  Please view the attachment.

Thank you,
Julian.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---