Re: Incremental compilation with DMD
Tom S wrote: Walter Bright wrote: I don't really understand why the -lib approach is not working for your needs. I'm not sure what you mean by "the -lib approach". Just how do you exactly apply it to incremental compilation? If my project has a few hundred modules and I change just one line in one function, I don't want to rebuild it with -lib all again. I thought you were referring to the proof-of-concept incremental build tool I posted yesterday which used -multiobj, as it should be possible to optimize it using -lib... I just haven't tried that yet. You only have to build one source file with -lib, not all of them.
Re: Incremental compilation with DMD
Walter Bright wrote: I don't really understand why the -lib approach is not working for your needs. I'm not sure what you mean by "the -lib approach". Just how do you exactly apply it to incremental compilation? If my project has a few hundred modules and I change just one line in one function, I don't want to rebuild it with -lib all again. I thought you were referring to the proof-of-concept incremental build tool I posted yesterday which used -multiobj, as it should be possible to optimize it using -lib... I just haven't tried that yet. -- Tomasz Stachowiak http://h3.team0xf.com/ h3/h3r3tic on #D freenode
Re: Incremental compilation with DMD
Tom S wrote: Walter Bright wrote: Tom S wrote: As for my own impressions of this idea, its biggest drawback probably is that the multitude of object files created via -multiobj strains the filesystem. Sure, but -multiobj and -lib generate exactly the same object files, it's just that -lib puts them all into a library so it doesn't strain the file system. Extracting the obj files from the lib is pretty simple, you can see the libomf.c for the format. You're right, I'm sorry. I must've overlooked something in the lib dumps and assumed one module overwrites the other. So with -lib, it should be possible to only extract the object files that contain static constructors and the main function and keep the rest packed up. Does that sound about right? All the .lib file is, is: [header] [all the object files concatenated together and aligned] [dictionary and index] Linux .a libraries are the same idea, just a different format for the header, dictionary and index. The obj files are unmodified in the library. You can extract them based on whatever criteria you need. By the way, using -lib causes DMD to eat a LOT of memory compared to the 'normal' mode - in one of my projects, it eats up easily > 1.2GB and dies. This could be a downside to this approach. I haven't tested whether it's the same with -multiobj Hmm. I build Phobos with -lib, and haven't experienced any problems, but it's possible as dmd doesn't ever discard any memory. Would it be hard to add an option to DMD to control template emission? Apparently GDC has -femit-templates, so it's doable ;) LDC outputs instantiations to all objects. I've found the LDC approach to be generally a poor one (having much experience with it for C++, where there is no choice). It generates huge object files and there are often linker problems trying to remove the duplicates. I really got tired of "COMDAT" problems with linkers, and no, it wasn't just with Optlink. Having each template instantiation in its own obj file works out great, eliminating all those problems. I don't really understand why the -lib approach is not working for your needs.
Re: Modern Windows GUI visual styles
Daniel Keep Wrote: > > > Tim M wrote: > > Microsoft says to use Application.EnableVisualStyles > > http://msdn.microsoft.com/en-us/library/system.windows.forms.application.enablevisualstyles.aspx > > Umm... you do realise that's for .NET, right? Run it through a debugger and you will probably find that both provide an abstraction over the same complicated win32 api functions.
Re: std.string phobos
== Quote from pc (peng2che...@yahoo.com)'s article
> Is there a way to make the functions in std.string, such as replace, pure?
> Many
pure functions are going to want to use these. Also, could some of them be
executable at compile time?
> For me, using D2.032, this did not compile.
> pure string replaceXX(string str){
> return replace(str,"XX","X");
> }
> If I am missing something, help!
For a function in D to pass the compiler checks for purity, it must only call
functions that are *marked as* being pure. If a function is not marked as pure
but is de facto pure, it won't work. For example:
uint nPlus2(uint n) pure {
return nPlus1( nPlus1( n)); // Not pure.
}
uint nPlus1(uint n) {
return n + 1;
}
Many functions that are, in fact, pure, have not been annotated as such yet in
Phobos, since pure was implemented fairly recently. If you want to help out,
this
is fairly low hanging fruit.
Also, purity is very restrictive right now and is designed partly with thread
safety in mind. A function that truly has no side effects from an observable
behavior in a single thread point of view won't necessarily pass the compiler
as pure:
__gshared uint foo;
/* wasteTime() is impure even though it has no observable side
* effects in a single thread because it still (at least
* temporarily) manipulates global state, and thus could
* cause problems in multithreaded code. Furthermore, even if
* it were thread safe, it would be hard to prove for all but
* the simplest cases that functions like these have no
* observable side effects.*/
void wasteTime() pure { // Won't compile.
foo++;
foo--;
}
Re: XML ecosystem wrt D
On Sat, Sep 12, 2009 at 8:34 PM, div0 wrote: > It seems on causal thought that it ought to be possible to machine > translate it into D, unless Java has under gone massive change since the > last time I played with it. > > So who's up for writing a Java to D translator? You mean Tioport? http://www.dsource.org/projects/tioport
Re: XML ecosystem wrt D
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Michel Fortin wrote: > On 2009-09-12 11:07:38 -0400, Justin Johansson > said: > >> What's the current state of the nation with respect to the XML (W3C >> compliant) ecosystem for D? >> >> Last months D forum archive had some discussions about std.text.xml or >> similar (for Phobos) but all in all seems to be no overall plan. >> (Whoever was developing seems to have gone AWOL; has Andrei been left >> with picking up from whoever left off?) Tango doesn't seem to have a >> grand plan either. Please correct me if I'm wrong. >> >> To qualify my question, by XML ecosystem I mean all things to do with >> XML processing and I/O, including parsing, serialization (complete >> suite of output formats XML/XHTML/HTML/plain text), memory-resident >> tree model(s), and particularly XSLT 2.0. > > There's a huge jump between XSLT 2.0 and the capability of parsing, > serialization and having a tree model. Yar. And XSLT is the only thing about all that xml bullshit that is in anyway interesting. Writing a XSLT processor is a massively none trivial task. Mind you saxon is written in Java, which was the only XSLT processor I found which did a proper job of it. It seems on causal thought that it ought to be possible to machine translate it into D, unless Java has under gone massive change since the last time I played with it. So who's up for writing a Java to D translator? - -- My enormous talent is exceeded only by my outrageous laziness. http://www.ssTk.co.uk -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKrD39T9LetA9XoXwRAinFAJ9nwgyeZURwLQYH6kjrpYWrfACkZgCfaPjq K/PoOAU8IHoCSp+DioHUVG0= =pYim -END PGP SIGNATURE-
Re: XML ecosystem wrt D
Justin Johansson wrote: What's the current state of the nation with respect to the XML (W3C compliant) ecosystem for D? Last months D forum archive had some discussions about std.text.xml or similar (for Phobos) but all in all seems to be no overall plan. (Whoever was developing seems to have gone AWOL; has Andrei been left with picking up from whoever left off?) Tango doesn't seem to have a grand plan either. Please correct me if I'm wrong. To qualify my question, by XML ecosystem I mean all things to do with XML processing and I/O, including parsing, serialization (complete suite of output formats XML/XHTML/HTML/plain text), memory-resident tree model(s), and particularly XSLT 2.0. Now I can here you all saying, "what planet is this guy on?" but given that D is supposed to be a systems programming language, real world (web) I/O, processing and glue that address contemporary W3C standards are of paramount importance if D is to be taken seriously. (IMHO) Has Java become the defacto language for doing any of this stuff -- all the tools are available? Having spent the last week trying to leverage non-Java libraries to get there in D, I'm not even close aside from a successful 2 hour exercise in getting James Clark's Expat XML parser working in D. Are any of these issues on D developer radars? Justin You might want to consider posting this question on the Tango IIRC you can use http://webchat.freenode.net/ to access this. The channel is #D.tango Nick B
Re: Incremental compilation with DMD
Walter Bright wrote: Tom S wrote: As for my own impressions of this idea, its biggest drawback probably is that the multitude of object files created via -multiobj strains the filesystem. Sure, but -multiobj and -lib generate exactly the same object files, it's just that -lib puts them all into a library so it doesn't strain the file system. Extracting the obj files from the lib is pretty simple, you can see the libomf.c for the format. You're right, I'm sorry. I must've overlooked something in the lib dumps and assumed one module overwrites the other. So with -lib, it should be possible to only extract the object files that contain static constructors and the main function and keep the rest packed up. Does that sound about right? By the way, using -lib causes DMD to eat a LOT of memory compared to the 'normal' mode - in one of my projects, it eats up easily > 1.2GB and dies. This could be a downside to this approach. I haven't tested whether it's the same with -multiobj Would it be hard to add an option to DMD to control template emission? Apparently GDC has -femit-templates, so it's doable ;) LDC outputs instantiations to all objects. -- Tomasz Stachowiak http://h3.team0xf.com/ h3/h3r3tic on #D freenode
Re: Incremental compilation with DMD
Tom S wrote: As for my own impressions of this idea, its biggest drawback probably is that the multitude of object files created via -multiobj strains the filesystem. Sure, but -multiobj and -lib generate exactly the same object files, it's just that -lib puts them all into a library so it doesn't strain the file system. Extracting the obj files from the lib is pretty simple, you can see the libomf.c for the format.
Re: shared adventures in the realm of thread-safety.
I'm also having the same problems.
As Jeremie said, as soon as you start introducing shared methods (via
synchronized for example), you rapidly get into trouble that can only be
overcome by excessive casting.
It may be possible to contain the problem by refactoring multi-threaded
code so that the shared objects are very small and simple, but even then
the casting required is too much. This approach might be ok if you could
define classes as being shared or immutable, and ALL instance of them
were then implicitly shared or immutable. Also, immutable objects should
be implicitly shareable.
On Sat, 12 Sep 2009 15:32:05 -0400, Jason House wrote:
> I'm glad to see I'm not the only one trying to use shared. I tried to
> use it with 2.031 and rapidly hit bug after bug... I submitted several
> bug reports for basic functionality, and none of it appeared in the
> changelog.
>
> http://d.puremagic.com/issues/show_bug.cgi?id=3089
> http://d.puremagic.com/issues/show_bug.cgi?id=3090
> http://d.puremagic.com/issues/show_bug.cgi?id=3091
>
>
>
> Jeremie Pelletier Wrote:
>
>> I decided to play once again with shared and see what 2.032 is capable
>> of. Turns out a lot of the previous issues I was having last time are
>> gone, however, there are still a few things left which prevent me from
>> rewriting my code.
>>
>> The first issue that jumped to my face straight away was how 'shared
>> const' methods are not callable from 'shared' objects.
>>
>> shared class Foo {
>> void bar() const;
>> }
>> auto foo = new Foo; // foo is of type shared(Foo) foo.bar; // Error:
>> function Foo.bar () shared const is not callable using argument types
>> () shared
>>
>> Considering how 'const' methods can be called from mutable objects,
>> this looks like either a bug or a really awkward feature to me. Sending
>> a shared(Foo) to a method expecting a shared(const(Foo)) also triggers
>> a similar error from the compiler.
>>
>> The other issue may be an intended feature, but it doesn't sound
>> practical to me. Marking a method as shared assumes all used properties
>> in the method's scope are also shared. Here is an example to illustrate
>> my point:
>>
>> class SimpleReader {
>>this(LocalFile file) { _stream = new FileInputStream(file); } ...
>> private:
>> synchronized void read(ubyte[] buf, long offset) {
>> _stream.seek(offset);
>> _stream.read(buf);
>> }
>> FileInputStream _stream;
>> }
>>
>> The FileInputStream here is a generic blocking binary stream which is
>> not thread-safe by design. The reader is a composite class where every
>> instance has its own unique stream instance and use it to implement
>> asynchronous reads over the file format it abstracts, which in my case
>> is a specialized read-only archive using a lot of random accesses from
>> different threads.
>>
>> This is where the issue shows its ugly head. The 'synchronized' keyword
>> tags the read method as shared, which in itself is quite neat, what is
>> annoying however is that it also changes the type of _stream in the
>> method's scope to shared(FileInputStream) and therefore triggers
>> compiler errors because _stream.seek and _stream.read are not shared:
>>
>> Error: function FileInputStream.read (ubyte[]) is not callable using
>> argument types (ubyte[]) shared
>>
>> While it may be an attempt to keep shared usage safe, it isn't very
>> practical. The stream object here is not shared because it is not
>> thread-safe. While it may be used by different threads, it is unique to
>> the reader's context and its accesses are synchronized by the reader,
>> the stream should therefore be completely oblivious to the fact it is
>> being used by different threads.
>>
>> Maybe this could be the time to implement an unique qualifier; this is
>> a context where having _stream be of type unique(FileInputStream) would
>> solve the problem and allow further compiler optimizations. I don't
>> know if it can be done with templates, and without any overhead
>> whatsoever. I know I would much rather see unique(Foo) than Unique!Foo,
>> and it would allow the use of 'is(foo : unique)'.
>>
>> Furthermore, tagging a method with shared does not make it thread-safe,
>> it may however use synchronized within its scope to protect its shared
>> or unique data. This may be confusing when calling shared methods vs
>> calling synchronized methods; one may think the shared one is not
>> thread-safe and optionally synchronize the call, resulting in another
>> monitor being used for nothing, or no monitor being used at all:
>>
>> class Foo {
>> shared void bar() {
>> // Do stuff with local or immutable data synchronized(this) {
>> /* do stuff with shared data */ }
>> }
>> shared void bar2() {
>> // Do stuff on shared data
>> }
>> }
>>
>> Someone seeing only the prototype of Foo.bar may assume the method is
>> not thread-safe and call it as 'synchronized(foo) foo.bar()'. Just like
>> they cou
std.string phobos
Is there a way to make the functions in std.string, such as replace, pure? Many
pure functions are going to want to use these. Also, could some of them be
executable at compile time?
For me, using D2.032, this did not compile.
pure string replaceXX(string str){
return replace(str,"XX","X");
}
If I am missing something, help!
Re: Simple bolt-on unittest improvement
"Bill Baxter" wrote in message
news:mailman.64.1252768975.20261.digitalmar...@puremagic.com...
> On Sat, Sep 12, 2009 at 7:09 AM, Justin Johansson
> wrote:
>> Lutger Wrote:
>>
>>> I'm rewriting my testing stuff at the moment, I hoped to use the runtime
>>> features to be able to user regular asserts too but that didn't work
>>> out.
>>>
>>> Do you know you can get rid of the need to pass __FILE__ and __LINE__?
>
> You can use a string mixin. That's the only way I know of in D1. But
> then the syntax becomes something like
>
> mixin(expectEquals("message"));
>
> The expectEquals then has to return a string of code which contains
> the __FILE__ and __LINE__.
>
Yea, that's what my stuff does.
Re: shared adventures in the realm of thread-safety.
I'm glad to see I'm not the only one trying to use shared. I tried to use it
with 2.031 and rapidly hit bug after bug... I submitted several bug reports for
basic functionality, and none of it appeared in the changelog.
http://d.puremagic.com/issues/show_bug.cgi?id=3089
http://d.puremagic.com/issues/show_bug.cgi?id=3090
http://d.puremagic.com/issues/show_bug.cgi?id=3091
Jeremie Pelletier Wrote:
> I decided to play once again with shared and see what 2.032 is capable of.
> Turns out a lot of the previous issues I was having last time are gone,
> however, there are still a few things left which prevent me from rewriting my
> code.
>
> The first issue that jumped to my face straight away was how 'shared const'
> methods are not callable from 'shared' objects.
>
> shared class Foo {
> void bar() const;
> }
> auto foo = new Foo; // foo is of type shared(Foo)
> foo.bar; // Error: function Foo.bar () shared const is not callable using
> argument types () shared
>
> Considering how 'const' methods can be called from mutable objects, this
> looks like either a bug or a really awkward feature to me. Sending a
> shared(Foo) to a method expecting a shared(const(Foo)) also triggers a
> similar error from the compiler.
>
> The other issue may be an intended feature, but it doesn't sound practical to
> me. Marking a method as shared assumes all used properties in the method's
> scope are also shared. Here is an example to illustrate my point:
>
> class SimpleReader {
>this(LocalFile file) { _stream = new FileInputStream(file); }
>...
> private:
> synchronized void read(ubyte[] buf, long offset) {
> _stream.seek(offset);
> _stream.read(buf);
> }
> FileInputStream _stream;
> }
>
> The FileInputStream here is a generic blocking binary stream which is not
> thread-safe by design. The reader is a composite class where every instance
> has its own unique stream instance and use it to implement asynchronous reads
> over the file format it abstracts, which in my case is a specialized
> read-only archive using a lot of random accesses from different threads.
>
> This is where the issue shows its ugly head. The 'synchronized' keyword tags
> the read method as shared, which in itself is quite neat, what is annoying
> however is that it also changes the type of _stream in the method's scope to
> shared(FileInputStream) and therefore triggers compiler errors because
> _stream.seek and _stream.read are not shared:
>
> Error: function FileInputStream.read (ubyte[]) is not callable using argument
> types (ubyte[]) shared
>
> While it may be an attempt to keep shared usage safe, it isn't very
> practical. The stream object here is not shared because it is not
> thread-safe. While it may be used by different threads, it is unique to the
> reader's context and its accesses are synchronized by the reader, the stream
> should therefore be completely oblivious to the fact it is being used by
> different threads.
>
> Maybe this could be the time to implement an unique qualifier; this is a
> context where having _stream be of type unique(FileInputStream) would solve
> the problem and allow further compiler optimizations. I don't know if it can
> be done with templates, and without any overhead whatsoever. I know I would
> much rather see unique(Foo) than Unique!Foo, and it would allow the use of
> 'is(foo : unique)'.
>
> Furthermore, tagging a method with shared does not make it thread-safe, it
> may however use synchronized within its scope to protect its shared or unique
> data. This may be confusing when calling shared methods vs calling
> synchronized methods; one may think the shared one is not thread-safe and
> optionally synchronize the call, resulting in another monitor being used for
> nothing, or no monitor being used at all:
>
> class Foo {
> shared void bar() {
> // Do stuff with local or immutable data
> synchronized(this) { /* do stuff with shared data */ }
> }
> shared void bar2() {
> // Do stuff on shared data
> }
> }
>
> Someone seeing only the prototype of Foo.bar may assume the method is not
> thread-safe and call it as 'synchronized(foo) foo.bar()'. Just like they
> could see the prototype of bar2 and assume it is thread-safe, calling it as
> 'foo.bar2()'.
>
> What could be a good design against this sort of misleading behavior?
>
> Phew, that's about enough issues and questions for now :)
Re: XML ecosystem wrt D
On 2009-09-12 11:07:38 -0400, Justin Johansson said: What's the current state of the nation with respect to the XML (W3C compliant) ecosystem for D? Last months D forum archive had some discussions about std.text.xml or similar (for Phobos) but all in all seems to be no overall plan. (Whoever was developing seems to have gone AWOL; has Andrei been left with picking up from whoever left off?) Tango doesn't seem to have a grand plan either. Please correct me if I'm wrong. To qualify my question, by XML ecosystem I mean all things to do with XML processing and I/O, including parsing, serialization (complete suite of output formats XML/XHTML/HTML/plain text), memory-resident tree model(s), and particularly XSLT 2.0. There's a huge jump between XSLT 2.0 and the capability of parsing, serialization and having a tree model. I started a XML tokenizer and DOM project for D2. In fact, the idea is that it could eventualy replace std.xml in Phobos 2. It's not advancing very fast as I don't have any urgent need of it and I'm busy working on too many things at once. If you're interested in looking at it, here's the documentation: Tokenizer part: http://michelf.com/docs/d/mfr/xmltok.html DOM part: http://michelf.com/docs/d/mfr/xml.html That's still far from XSLT (I don't have namespaces yet, and even less Xpath), but it's a start. If others want to help developing it, I can share the code and setup what's necessary for collaboration. Now I can here you all saying, "what planet is this guy on?" but given that D is supposed to be a systems programming language, real world (web) I/O, processing and glue that address contemporary W3C standards are of paramount importance if D is to be taken seriously. (IMHO) That depends in which fields you want D to be taken seriously. You don't need a full XML ecosystem everywhere. But a few things I'd like to do with D requires it. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Template Metaprogramming Made Easy (Huh?)
Fri, 11 Sep 2009 22:41:32 +, BCS thusly wrote: > Hello language_fan, > >> Fri, 11 Sep 2009 16:33:56 +, BCS thusly wrote: >> >>> First, I have zero interest in game development so that's not an issue >>> for me. >>> >> Game development is one of the largest users of systems programming >> languages. > > I would mandate the 10-25% test no mater what language is being used. > > The bulk of programming is done for Finance, Insurance and Real Estate > (and is done in COBOL ) The most common programs out there are > OSs and MS Office. As I said, I don't care about games. I was talking about systems programming languages like C or D. From wikipedia "The term system programming language is also (and perhaps more widely) used to mean "a language for system programming": that is, a language designed for writing system software as distinct from application software. In contrast with application languages, such system programming languages typically offer more direct access to the physical hardware of the machine: an archetypical system programming language in this sense was BCPL. The distinction between languages for system programming and applications programming became blurred with widespread popularity of C." The various application domains you listed actually often do not use systems programming languages, at least the majority of their code does not. Things like database engines, drivers, operating systems, firmware, virtual machines (and games) on the other hand have no choice. For instance the code I have seen on the finance industry used languages Java (J2EE), Awk, Perl, and Javascript. What is a bit confusing is that you mentioned operating systems and MS Office. 99.9% of companies worldwide do not develop any code even as plugins to those. For example MS Office is a native executable only for business reasons. There is nothing preventing them from providing it as an applet or web service (like google does). Office suites are in no way performance limited these days. In fact I think parts of the competitor OpenOffice.org has been written in Java. > >>> OK so the lead knows that they can make things x times faster. Well >>> then the demo on the 10-25th percentile machine must not be x times >>> slower than what you need on ship day. Exactly the same as would be >>> true if the demo were done on a 50th or 75th percentile machine. >>> >> Well basically you could do that. Usually it does not work that way. >> The idea is to prioritize the features and remove the worst ones. > > well that's also a way to make it run faster. > >> It >> cannot be known beforehand which features are unnecessary, and there is >> a hard limit on how much can be removed. So either you can remove say >> 30-50% of features > > Clearly you can't cut core features, but you can make some eye candy > features go away when there isn't enough power to run them. Making business decisions is not that easy, especially if you have no idea of the application domain. There are several stakeholders and various contracts involved. > >> or do a complete redesign. > > If a different design is practical and would be faster, you should have > used it in the first place or should be planning on doing it at some > point anyway (I have never seen a non trivial program that was fast > enough that I didn't whish it was faster). Large parts of software projects worldwide fail. Redesigning for instance a single iteration is not that bad. You seem to favor the top-down waterfall model. Unfortunately the waterfall model usually fails. If you had studied software engineering lately, you would know that. >> But if you end up >> using only 50% of the potential resources of the platform, your game >> will usually suck (if the audience is technology oriented as it usually >> is). >> >>> This is the classic "fast cheap or well done, pick two". For anything >>> that will ship, I'll always pick well done. >>> >> That is ok if you are a hobby programmer, but in real world e.g. in the >> game industry the contracts pretty much dictate the schedules and if >> you spend too much time on the project, the producer will not offer any >> extra money. So if $1000..$1500 / month is ok for you, then fine. >> >> > I will grant that games can legitimately require top of the line > hardware (scientific programs, and some things like photoshop can also) > but most anything that runs on a desktop should be written so that > people can run it with the hardware they have now, rather than have to > buy new hardware Nowadays, as the piracy is hindering PC sales quite a lot, the focus is on console, mobile, and online games. The hardware specifications do not change that often, but it is still a bit hard to foretell what kind of stuff works. If the producer decides to want a split screen game mode 2 months before deadline, it is not clear at all if the final frame rate will be below acceptable level in some parts of the game.
Re: Simple bolt-on unittest improvement
Lutger Wrote: > Justin Johansson wrote: > ... > > If I haven't misread you, sounds we are on the same track. > > For sure, one thing I have come to appreciate is how Walter made things so > simple: one file containing related code, documentation and the testsuite. > As soon as you have more than one of anything there is bound to be some > hassle involved :) Hear, hear (as they say in parliament .. dunno what they say in congress or the kremlin though). Thanks Walter for the unittest facility!
Re: Simple bolt-on unittest improvement
On Sat, Sep 12, 2009 at 7:09 AM, Justin Johansson
wrote:
> Lutger Wrote:
>
>> I'm rewriting my testing stuff at the moment, I hoped to use the runtime
>> features to be able to user regular asserts too but that didn't work out.
>>
>> Do you know you can get rid of the need to pass __FILE__ and __LINE__?
You can use a string mixin. That's the only way I know of in D1. But
then the syntax becomes something like
mixin(expectEquals("message"));
The expectEquals then has to return a string of code which contains
the __FILE__ and __LINE__.
--bb
XML ecosystem wrt D
What's the current state of the nation with respect to the XML (W3C compliant) ecosystem for D? Last months D forum archive had some discussions about std.text.xml or similar (for Phobos) but all in all seems to be no overall plan. (Whoever was developing seems to have gone AWOL; has Andrei been left with picking up from whoever left off?) Tango doesn't seem to have a grand plan either. Please correct me if I'm wrong. To qualify my question, by XML ecosystem I mean all things to do with XML processing and I/O, including parsing, serialization (complete suite of output formats XML/XHTML/HTML/plain text), memory-resident tree model(s), and particularly XSLT 2.0. Now I can here you all saying, "what planet is this guy on?" but given that D is supposed to be a systems programming language, real world (web) I/O, processing and glue that address contemporary W3C standards are of paramount importance if D is to be taken seriously. (IMHO) Has Java become the defacto language for doing any of this stuff -- all the tools are available? Having spent the last week trying to leverage non-Java libraries to get there in D, I'm not even close aside from a successful 2 hour exercise in getting James Clark's Expat XML parser working in D. Are any of these issues on D developer radars?
Re: Simple bolt-on unittest improvement
Justin Johansson wrote: ... > If I haven't misread you, sounds we are on the same track. For sure, one thing I have come to appreciate is how Walter made things so simple: one file containing related code, documentation and the testsuite. As soon as you have more than one of anything there is bound to be some hassle involved :)
Re: Simple bolt-on unittest improvement
Lutger Wrote: > I'm rewriting my testing stuff at the moment, I hoped to use the runtime > features to be able to user regular asserts too but that didn't work out. > > Do you know you can get rid of the need to pass __FILE__ and __LINE__? > > Define the functions this way: > > void expectEquals(T)(T some_x, T some_y, > string file = __FILE__, > int line = __LINE__); > > Default parameter initialization does the right thing here (in D2, dunno > about D1). > Thanks to everyone for responses; to keep forum churn low if I haven't thanked you please consider yourself thanked ;-) Lutger, I'm using D1.0 and, being an old C++ hacker, immediately tried the default params trick for __FILE__ and __LINE__ but it gave me the info for the called function and not the point of call so seems I'm stuck with the noise unfortunately. Moving on. Extending the JUnit functions set which includes assertTrue, assertGreaterThan et. al. and taking the lead from DUnit to prefix the funcnames with "expect" rather than "assert", I've implemented things like "expectAssertFail" and "expectException" as added tests just to verify that my assert and exception code works. These latter functions take functions and/or delegates as parameters and are wrapped in a try/catch block themselves so that the tests can be continuous. All in all, and taking into account other replies on this thread, me still thinks that the stock-standard D idiom for unit testing is all you need for a decent unit testing regime. Its just a matter of fine tuning your set of "assert-but-don't-bail-out-on-just-one-error" functions. If I haven't misread you, sounds we are on the same track. Cheers Justin
Re: Incremental compilation with DMD
Tom S wrote: Walter Bright wrote: Tom S wrote: Thus my suggestion of adding an option to DMD so it may emit template instances to all object files that use them. If anyone has alternative ideas, I'd be glad to hear them, because I'm running out of options. Try compiling with -lib, which will put each template instance into its own obj file. Thanks for the suggestion. Unfortunately it's a no-go since -lib seems to have the same issue that compiling without -op does - if you have multiple modules with the same name (but different packages), one will overwrite the other in the lib. To clarify, this is not the only issue with -lib. The libs would either have to be expanded into objects or static ctors would not run. And why extract them if -multiobj already generates them extracted? -- Tomasz Stachowiak http://h3.team0xf.com/ h3/h3r3tic on #D freenode
Re: Incremental compilation with DMD
Walter Bright wrote:
Tom S wrote:
Thus my suggestion of adding an option to DMD so it may emit template
instances to all object files that use them. If anyone has alternative
ideas, I'd be glad to hear them, because I'm running out of options.
Try compiling with -lib, which will put each template instance into its
own obj file.
Thanks for the suggestion. Unfortunately it's a no-go since -lib seems
to have the same issue that compiling without -op does - if you have
multiple modules with the same name (but different packages), one will
overwrite the other in the lib. On the other hand, I was able to hack
DMD a bit and use -multiobj since your suggestion gave me an idea :)
Basically, the approach would be to compile the project with -multiobj
and move the generated objects to a local (per project) directory,
renaming them so no conflicts arise.
The next step is to determine all public and comdat symbols in all of
these object files - this might be done via a specialized program,
however I've used Burton Radons' exelib to optimally run libunres.exe
from DMC. The exports are saved to some sort of a database (a dumb
structured file is ok).
The following is done on the initial build - so the next time we have
some object files in a directory and a map of all their exported
symbols. In an incremental step, we'll compile the modified modules, but
don't move their object files immediately over to the special directory.
We'll instead scan their public and comdat symbols and figure out which
object files they replace from our already compiled set. For each symbol
in the newly compiled objects, find which object in the original set
defined it, then mark it. For all marked files, add them to a library (
I call it junk.lib ), then remove the source object. Finally, move the
newly compiled objects to the special object directory.
The junk.lib will be used if the newly compiled object files missed any
shared symbols that were in the old objects and that would be generated,
had more modules be passed to the compiler. In other words, it contains
symbols that the naive incremental compilation will lose.
When linking, all object files from the directory are passed explicitly
to the compiler and symbols are pulled eagerly from them, however
junk.lib will be queried only if a symbol cannot be found in the set of
objects in the special directory.
I've put up a proof-of-concept implementation at
http://h3.team0xf.com/increBuild.7z . It requires a slightly patched DMD
(well, hacked actually), so it prints out the names of all objects it
generates. Basically, uncomment the `printf("writing '%s'\n", fname);`
in glue.c at line 133 and add `printf("writing '%s'\n",
m->objfile->name->str);` after `m->genobjfile(global.params.multiobj);`
in mars.c. I'm compiling the build tool with a recent (SVN-ish) version
of Tango and DMD 1.047.
As for my own impressions of this idea, its biggest drawback probably is
that the multitude of object files created via -multiobj strains the
filesystem. Even when running on a ramdrive, my WinXP-based system took
a good fraction of a second to move a few hundred object files to their
destination directory. This can probably be improved on, as -multiobj
seems to produce some empty object files (at least according to libunres
and ddlinfo). It might also be possible to use specialized storage for
object files by patching up dmd and hooking OPTLINK's calls to
CreateFile. I'm not sure about Linux, but perhaps something based on
FUSE might work. These last options are probably long shots, so I'm
still quite curious how DMD might perform with outputting template
instantiations into each object file that uses them.
--
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
Re: Simple bolt-on unittest improvement
I'm rewriting my testing stuff at the moment, I hoped to use the runtime
features to be able to user regular asserts too but that didn't work out.
Do you know you can get rid of the need to pass __FILE__ and __LINE__?
Define the functions this way:
void expectEquals(T)(T some_x, T some_y,
string file = __FILE__,
int line = __LINE__);
Default parameter initialization does the right thing here (in D2, dunno
about D1).
I like the testing overhead to be as minimal as possible, for that purpose I
have something hackish right now. Basically something like this:
void test(string testname)(void delegate () testClosure,
int LineNumber = __LINE__,
string FileName = __FILE__)
{
writeln("running test ", name, " in ", FileName,"(", LineNumber,")");
try
testClosure();
catch(Exception ex)
// print test failure
/* print test success. (actually first checks some flag set by expect***
functions to see if test passed)*/
}
And this:
void expectTrue(lazy bool exp, int LineNumber = __LINE__,
string FileName = __FILE__)
{
if (!exp())
writeln("not true! ", FileName,"(", LineNumber,")");
}
usage:
unittest
{
int a = 42;
test!"foo" = {
auto b = 42;
expectEquals( a, b );
expectTrue( a == a );
throw new Exception("fail!");
};
}
Re: Template Metaprogramming Made Easy (Huh?)
Fri, 11 Sep 2009 16:16:07 -0400, Justin Johansson thusly wrote: >> > Another point related to maintenance costs is the choice of language. >> > Many smaller companies choose "easier" languages like php, visual >> > basic, or python for their applications because they outsource >> > library writing and it is much cheaper and easier to hire new >> > workforce for business logic when the existing codebase does not use >> > any complex idioms or language constructs. Some indian schoolboy can >> > write e.g. php pages $1-2/ h, a hard-core C++/Haskell professional >> > easily costs $500/h. >> >> Haven't heard of many $500/h gigs. >> >> Andrei > > I have. Haven't you every been to see a lawyer :-) > > Seriously, the OP must have meant $500/day. Unfortunately no. I live in the ECU area and even my consulting costs are on average about 150..200 eur/h + travel expenses. A normal senior level developer on any field of computer engineering gets $500/day.
