Re: unzip parallel, 3x faster than 7zip
On Saturday, 7 April 2012 at 05:02:04 UTC, dennis luehring wrote: 7zip took 55 secs _on the same file_. that is ok but he still compares different implementations 7zip is the program. It unzips many formats, with the standard zip format being one of them. The parallel d program is three times faster at decoding the zip format than 7zip decodes the same file on the same ssd drive. That is an appropriate comparison since 7zip has been my utility of choice for unzipping zip format files on windows for many years. I provided the source code in the examples folder for the complete command line utility that I used, so you may build it and compare it to whatever you like and report the results.
Re: I'll be in Seattle at Lang.NEXT
On 3/15/12 10:39 AM, Andrei Alexandrescu wrote: Hello, Somewhat unexpectedly even to me, I'll be in Seattle for the Lang.NEXT conference together with Walter. Both of us will give talks. http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012 Would be glad to redo a D Seattle meeting. Regardless, the conference is interesting and free so it's worth joining if time allows. Slides are online: http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Three-Unlikely-Successful-Features-of-D Talk will be up in due time, probably Monday. Philippe and Dmitry, you have a lot of work to do :o). Andrei
Re: I'll be in Seattle at Lang.NEXT
On 4/7/12, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Slides are online: http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Three-Unlikely-Successful-Features-of-D I don't know if Walter opened another thread for his presentation, but his slides are online now too: http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/The-D-Programming-Language
Re: I'll be in Seattle at Lang.NEXT
On 4/7/12, Andrej Mitrovic andrej.mitrov...@gmail.com wrote: I don't know if Walter opened another thread for his presentation, but his slides are online now too: http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/The-D-Programming-Language Heheh, I've noticed Walter is using FIle.byLine to demonstrate RAII on page 24. I guess he didn't run into this yet: http://d.puremagic.com/issues/show_bug.cgi?id=7022
Re: I'll be in Seattle at Lang.NEXT
On Sat, Apr 7, 2012 at 08:33, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Slides are online: http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/Three-Unlikely-Successful-Features-of-D Nice :) Thanks for choosing my project as one of your examples. Talk will be up in due time, probably Monday. Philippe and Dmitry, you have a lot of work to do :o). I love being stressed! What kind of work do you have in mind?
Re: unzip parallel, 3x faster than 7zip
On 4/7/2012 12:32 AM, Jay Norwood wrote: I got procmon to see what is going on. Win7 has doing indexing and thumbnails, and there was some virus checker going on, but you can get rid of those. Still, most of the problem just boils down to the duration of the delete on close being proportional to the size of the file, and apparently related to the access times of the disk. I sometimes see .25 sec duration for a single file during the close of the delete operations on the hard drive. Maybe it is the trim command being executed on the sectors previously occupied by the file. I've been using an intel 510 series 120GB drive for recording concerts. It is hooked up with an ineo usb3 adaptor to the front panel port of an rme ufx recorder. The laptop is just used as a controller ... the ufx does all the mixing and recording to the hard drive.
Re: unzip parallel, 3x faster than 7zip
On Saturday, 7 April 2012 at 11:41:41 UTC, Rainer Schuetze wrote: Maybe it is the trim command being executed on the sectors previously occupied by the file. No, perhaps I didn't make it clear that the rmdir slowness is only an issue on hard drives. I can unzip the 2GB archive in about 17.5 sec on the ssd drive, and delete it using the rmd multi-thread delete example program in less than 17 secs on the ssd drive. The same operations on a hard drive take around 60 seconds to extract, but 1.5 to 3 minutes to delete. H:\uzp tzip.zip tz unzipping: .\tzip.zip finished! time: 17405 ms H:\rmd tz removing: .\tz finished! time:16671 ms I've been doing some reading on the web and studying the procmon logs. I am convinced the slow hard drive delete is an issue with seek times, since it is not an issue on the ssd. It may be caused by fragmentation of the stored data or the mft itself, or else it could be that ntfs is doing some book-keeping journaling. You are right that it could be doing delete notifications to any application watching the disk activity. I've already turned off the virus checker and the indexing, but I'm going to try the tweaks in the second link and also try the mydefrag program in the third link and see if anything improves the hd delete times. http://ixbtlabs.com/articles/ntfs/index3.html http://www.gilsmethod.com/speed-up-vista-with-these-simple-ntfs-tweaks http://www.mydefrag.com/index.html That mydefrag has some interesting ideas about sorting folders by full pathname on the disk as one of the defrag algorithms. Perhaps using it, and also using unzip and zip algorithms that match the defrag algorithm, would be a nice combination. In other words, if the zip algorithm processes the files in a sorted-by-pathname order, and if the defrag algorithm has created folders that are sorted on disk by the same order, then you would expect optimally short seeks while processing the files in the order they are stored. The mydefrag program uses the ntfs defrag api. There is an article at the following link showing how to access it to get the Logical Cluster Numbers on disk for a file. I suppose you could sort your file operations by start LCN, of the file, for example during compression, and that might reduce the seek related delays. http://blogs.msdn.com/b/jeffrey_wall/archive/2004/09/13/229137.aspx
Re: unzip parallel, 3x faster than 7zip
On Saturday, 7 April 2012 at 17:08:33 UTC, Jay Norwood wrote: The mydefrag program uses the ntfs defrag api. There is an article at the following link showing how to access it to get the Logical Cluster Numbers on disk for a file. I suppose you could sort your file operations by start LCN, of the file, for example during compression, and that might reduce the seek related delays. http://blogs.msdn.com/b/jeffrey_wall/archive/2004/09/13/229137.aspx I did a complete defrag of the g hard drive, then did parallel unzip of the tz folder with rmd, then unzipped it again with the parallel uzp. Then analyzed the disk again with mydefrag. The analysis shows the unzip resulted in over 300 fragmented files created, even though I wrote each expanded file in a single operation. So, I did a complete defrag again, then removed the folder again, and get about the same 109 secs for the delete operation on the hd (vs about 17 sec on the ssd for the same operation). The uzp parallel unzip is bout 85 secs vs about 17.5 sec on the ssd. G:\rmd tz removing: .\tz finished! time:109817 ms G:\uzp tzip.zip tz unzipping: .\tzip.zip finished! time: 85405 ms G:\rmd tz removing: .\tz finished! time:108387 ms So ... it looks like the defrag helps, as the 109 sec values are at the low end of the range I've seen previously. Still it is totally surprising to me that deleting files should take longer than creating the same files. btw, here are the windows rmdir on the defragged hd and on the ssd drive, and the third measurement is the D parallel rmd on the ssd ... much faster on D. G:\cmd /v:on /c echo !TIME! rmdir /q /s tz echo !TIME! 14:34:09.06 14:36:23.36 H:\cmd /v:on /c echo !TIME! rmdir /q /s tz echo !TIME! 14:38:44.69 14:40:02.16 H:\rmd tz removing: .\tz finished! time:17536 ms
Re: D projects list
On Fri, Apr 06, 2012 at 02:52:38PM -0400, Nick Sabalausky wrote: H. S. Teoh hst...@quickfur.ath.cx wrote in message news:mailman.1417.1333721195.4860.digitalmar...@puremagic.com... On Fri, Apr 06, 2012 at 12:34:09PM +0400, Denis Shelomovskij wrote: And adobe Flash of course should also die. +1. It should have died a DECADE ago. Except that certain interests kept its decaying worm-infested corpse animating even till today. Funny, that's also how I feel about C++. As I've been saying for awhile, a decade of near-zero interest in anything but VM languages is what kept it on life support. I would've felt the same had I known of a suitable replacement for C++ earlier. But for many years it was the only thing I had (besides pure hardcore C) that was even remotely close to what my ideal language would be. Fortunately, D's quickly becoming the successor that's always been needed so C++ will finally be able to RIP. You have no idea how many times I swore off C++ because of its inherent stupidities (like hash tables not making the C++ standard before C++11, among many other sillinesses), only to have to crawl back on my knees when other alternatives sucked even more. When I finally discovered D, I was almost blown away by how many things it got right, in my book. (I had a list of things I wanted in an ideal language, and D fits so many items on that list it's quite uncanny.) [...] I can personally vouch for the fact that there is *nothing* productive about writing software in Flash. C++ is more productive. [...] From the very first day I heard about Flash, I've had my doubts. It just *smelled* wrong. That was before I even had any idea of how it's implemented. Time has proven that my gut feeling was correct. Flash is a festering wart that should be blotted from off the face of the internet for the sake of the future of humanity. T -- It is of the new things that men tire --- of fashions and proposals and improvements and change. It is the old things that startle and intoxicate. It is the old things that are young. -- G.K. Chesterton
Re: Discussion on Go and D
On 07.04.2012 2:08, Rainer Schuetze wrote: On 4/6/2012 8:01 PM, Walter Bright wrote: On 4/6/2012 10:37 AM, Rainer Schuetze wrote: I hope there is something wrong with my reasoning, and that you could give me some hints to avoid the memory bloat and the application stalls. A couple of things you can try (they are workarounds, not solutions): 1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free(). Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this. How about this: http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/ Or you can wrap-up something similar along the same lines. -- Dmitry Olshansky
Re: Goldie Parser Generator. Haxe language definition.
bls bizp...@orange.fr wrote in message news:jlnp76$2mg4$1...@digitalmars.com... Hi Nick, I guess :) some questions. Since there is no formal language description for Haxe (afaik) How do you implement the Gold Haxe grammar ? It seems that Goldie grammars are very similar to BNF ( not EBNF) is this true ? Since I like Haxe (at least the language, not the OCaml impl.) I would like to use your grammar to create a PEG port. (based on https://github.com/PhilippeSigaud/Pegged) So ...May I reuse your Gold Haxe grammar ? I would like to create Haxe to D and eventually HLA translator. The final goal is to create a factory pattern based Haxe2XXX translator. Bjoern.. Goldie uses the exact same grammar definition language as GOLD. It is very similar to BNF, and it doesn't include repetition operators for non-terminals, but such things are easy enough to accomplish manually. The full documentation of the format is on GOLD's site here: http://goldparser.org/doc/grammars/index.htm It's also explained in Goldie's Beginner's Tutorial and on GOLD's Getting Started: http://www.semitwist.com/goldie/Start/Tutorial/ http://goldparser.org/getting-started/index.htm However, I am working on a Haxe - D converter already, called HaxeD (which uses Goldie), so the easiest thing would probably be to just use that as a starting point, or contribute pull requests: http://www.dsource.org/projects/haxed It's zlib/libpng licensed open-source (except where otherwise noted, such as JsonViewer), so you can fork it, borrow and adapt any part of it, pretty much whatever you want. It's not really usable for real Haxe programs yet, but the current state of it is explained right there on its homepage. Out of all my hobby projects, HaxeD is actually my #1 top priority at this point (real world work still takes precedence, of course). So I'm pretty committed to it. Any big gaps of time between commits are usually just due to making Goldie improvements to help with HaxeD. For example, I just finished a manic 3-week stretch fixing some embarrassing problems in Goldie's GRMC: Grammar Compiler that prevented it from handling the Haxe grammar correctly. It's fixed in Goldie's Git/BitBucket now and will be out in Goldie v0.9 hopefully pretty soon after DMD 2.059 is out. Of course, if you want to adapt the grammar for Pegged, by all means, feel free!
Re: D projects list
H. S. Teoh hst...@quickfur.ath.cx wrote in message news:mailman.1438.1333780033.4860.digitalmar...@puremagic.com... On Fri, Apr 06, 2012 at 02:52:38PM -0400, Nick Sabalausky wrote: H. S. Teoh hst...@quickfur.ath.cx wrote in message news:mailman.1417.1333721195.4860.digitalmar...@puremagic.com... On Fri, Apr 06, 2012 at 12:34:09PM +0400, Denis Shelomovskij wrote: And adobe Flash of course should also die. +1. It should have died a DECADE ago. Except that certain interests kept its decaying worm-infested corpse animating even till today. Funny, that's also how I feel about C++. As I've been saying for awhile, a decade of near-zero interest in anything but VM languages is what kept it on life support. I would've felt the same had I known of a suitable replacement for C++ earlier. But for many years it was the only thing I had (besides pure hardcore C) that was even remotely close to what my ideal language would be. Getting fed up with C/C++ many years ago prompted me to seek out alternatives, which happily led me to D back around the 0.8x or 0.9x days. Heh, I remember being mildly disappointed that 0.99 was followed by 0.100 instead of 1.0 ;) Fortunately, D's quickly becoming the successor that's always been needed so C++ will finally be able to RIP. You have no idea how many times I swore off C++ because of its inherent stupidities (like hash tables not making the C++ standard before C++11, among many other sillinesses), only to have to crawl back on my knees when other alternatives sucked even more. Yea, the problem IMO is that for at least a full decade eveyone had drunk the Sun Kool-Aid and was obsessed with VMs and interpreted and all that silly nonsence. So outside Digital Mars, *real* language development took a ten year hiatus. Only now are there languages like Rust and Issue 9 trying to catch up with D. I can personally vouch for the fact that there is *nothing* productive about writing software in Flash. C++ is more productive. [...] From the very first day I heard about Flash, I've had my doubts. It just *smelled* wrong. That was before I even had any idea of how it's implemented. Time has proven that my gut feeling was correct. Flash is a festering wart that should be blotted from off the face of the internet for the sake of the future of humanity. Heh, you've got better intuition than me then :) Until I actually tried to use it, I don't remember having any problems with Flash beyond terrible, horrible abuses of it like Flash Sites, Flash Intros, and what turned out to essentially be the second coming of animated GIFs. The *one* good use of it is stupid/goofy/hilarious shit like And Whe! and other such things that make Newgrounds worthwhile. :) But, I had *no hint* of the endless mountains of technical problems with it until I tried it.
Re: IDE Support for D
On Friday, 6 April 2012 at 23:45:49 UTC, Manu wrote: Fair enough I guess, but I'm a customer. I work commercially, and I'd happily pay money for tools that work. On Windows, the Zeus editor/IDE has support for the D language: http://www.zeusedit.com/zforum/viewtopic.php?t=2645 Sadly, I can't offer any significant amount of my own time Any Zeus bug reports are more than welcome ;) NOTE: Zeus is shareware
Re: custom attribute proposal (yeah, another one)
Am Fri, 06 Apr 2012 16:33:21 -0400
schrieb Steven Schveighoffer schvei...@yahoo.com:
On Fri, 06 Apr 2012 15:03:39 -0400, Piotr Szturmaj
bncr...@jadamspam.pl wrote:
Steven Schveighoffer wrote:
What if I have 20 string attributes, I must define a new attribute
type for each one? This seems like unneeded bloat.
I don't see advantage of functions here, twenty of them is also a
bloat. Different types give you different _names_ for different
purposes. Those _names_ are crucial to support the attributes.
Unused function do not make it into the EXE.
But as long as you mark attribute structs in some special way
(@attribute struct Author), this can also be guaranteed for structs.
Afaik ignoring unused functions is currently done by the linker, but I
think functions/structs only used for attributes should probably never
make it to the linker. I think we already had cases were linkers didn't
strip unused functions for some reason?
Regarding code bloat: If struct initializers could be used with
attributes, a constructor isn't necessary and the minimal code is:
--
@attribute struct Author
{
string name;
}
@Author(Johannes Pfau) int test;
or
@Author{name: Johannes Pfau} int test;
--
That's exactly as much code as using functions:
--
@attribute string Author(string name)
{
return name;
}
@Author(Johannes Pfau) int test;
--
I don't have a strong opinion whether storable types should be limited
to structs, but I think classes add little benefit and complicate
things because of inheritance (at least if you query attributes by
type). What we want to do here is store _data_ and the D style is to
use structs for pure data. Basic types could be useful too but when
querying by type, those can't work well.
BTW: I think there's one thing both your and my proposals are missing:
The function/constructor returning the data must be pure: We can't
guarantee this function will be executed only once, but the value of
the attribute should always be the same. Consider this scenario:
a.d
@RandomNumber() int test;
b.d
---
auto value1 = __traits(getAttribute, a.test, RandomNumber);
---
c.d
---
auto value2 = __traits(getAttribute, a.test, RandomNumber);
---
All files are compiled individually. Now the compiler has to call
RandomNumber() 2 times: one time for b.d and one time for c.d, but
value1 should be the same as value2.
Re: Mascot for D
Apparently, then, when you have no substance, the cure is to get a mascot? Good products sell themselves. If D lacks market share after being introduced 10 years ago, do you REALLY think a cutsie-tootsie graphic is going to help? Hello. That said, I would like to know HOW MANY is this group of soldiers fighting for D's resurrection and saving of the world. Seriously. How many? Also, I would like to know, in a consolidated way, what the hopes are of those soldiers. (I think I may know, even if they don't). Lastly, I would like to know if there are any developers (or programmers, and yes, there is a difference) here that have not ever worked with C or C++. (Yeah, I tend to be blunt. Deal with it.) crudbug crud...@gmail.com wrote in message news:siedfnxxjyimkitsh...@forum.dlang.org... Hi All, I am just starting my D adventure, I chose D over Go ! I think it will kool to have some marketing behind D. A mascot Dragon will look good ? http://4.bp.blogspot.com/-0YqxfqC5Ero/TfCTFkcPHbI/CNY/UaKuBVRyOpw/s1600/Skeleton_Dragon_Knight_Tattoo.jpg What do you guys think ? -Adam
Re: Mascot for D
On 04/07/2012 10:35 AM, BillWendt wrote: Apparently, then, when you have no substance, the cure is to get a mascot? These are orthogonal concerns. Good products sell themselves. If D lacks market share after being introduced 10 years ago, 10 years ago, it was not a language I would have used. (In fact, the same probably holds for a time span of only 2 years.) do you REALLY think a cutsie-tootsie graphic is going to help? Why do you assume it is intended to? Hello. Hi. That said, I would like to know HOW MANY is this group of soldiers fighting for D's resurrection and saving of the world. It never died. Seriously. How many? That is completely unimportant. Also, I would like to know, in a consolidated way, what the hopes are of those soldiers. (I think I may know, even if they don't). I enjoy using D. Popularity contests are childish. Lastly, I would like to know if there are any developers (or programmers, and yes, there is a difference) here that have not ever worked with C or C++. So what? http://en.wikipedia.org/wiki/Argumentum_ad_populum http://en.wikipedia.org/wiki/Worse_is_better (Yeah, I tend to be blunt. Deal with it.) (Well, why do you tend to be blunt? Most people are not like that. Also, why do you care?)
Re: Mono-D GSoC proposal, hopefully the last thread about it
On Saturday, 7 April 2012 at 00:22:09 UTC, Adam Wilson wrote: Well, that was just the deadline for turning them in, you can still make changes. Now the mentor/student assignment process begins, i've listed myself on your proposal, I couldn't do that before today. :-) Ah ok, nice + thanks!
Re: custom attribute proposal (yeah, another one)
Le 07/04/2012 05:29, Kapps a écrit : I slightly prefer this function method over the struct method because: 1) No need to generate a custom struct for everything. Plenty of things are just a true or false, or a string. Saves a little bit of TypeInfo generation. If the type isn't used at runtime, the compiler should be able to remove that dead part of the code. If it doesn't, it is an implementation issue, and shouldn't be solved by language design decision.
Re: custom attribute proposal (yeah, another one)
Le 07/04/2012 05:29, Kapps a écrit : On the topic of type vs storage, it is useful to be able to apply attributes to a type, but this should /not/ change the type itself. It must be transparent to the user what attributes a type has unless they're actually accessing attributes. Then put the attribute at type declaration, not where it is used.
Re: custom attribute proposal (yeah, another one)
Le 07/04/2012 09:10, Johannes Pfau a écrit :
But as long as you mark attribute structs in some special way
(@attribute struct Author), this can also be guaranteed for structs.
Afaik ignoring unused functions is currently done by the linker, but I
think functions/structs only used for attributes should probably never
make it to the linker. I think we already had cases were linkers didn't
strip unused functions for some reason?
Regarding code bloat: If struct initializers could be used with
attributes, a constructor isn't necessary and the minimal code is:
--
@attribute struct Author
{
string name;
}
@Author(Johannes Pfau) int test;
or
@Author{name: Johannes Pfau} int test;
--
That's exactly as much code as using functions:
--
@attribute string Author(string name)
{
return name;
}
@Author(Johannes Pfau) int test;
--
I don't have a strong opinion whether storable types should be limited
to structs, but I think classes add little benefit and complicate
things because of inheritance (at least if you query attributes by
type). What we want to do here is store _data_ and the D style is to
use structs for pure data. Basic types could be useful too but when
querying by type, those can't work well.
For basic type :
alias int foobar;
@foobar(2) someDeclaration;
No benefit in introducing a new syntax.
BTW: I think there's one thing both your and my proposals are missing:
The function/constructor returning the data must be pure: We can't
guarantee this function will be executed only once, but the value of
the attribute should always be the same. Consider this scenario:
a.d
@RandomNumber() int test;
b.d
---
auto value1 = __traits(getAttribute, a.test, RandomNumber);
---
c.d
---
auto value2 = __traits(getAttribute, a.test, RandomNumber);
---
All files are compiled individually. Now the compiler has to call
RandomNumber() 2 times: one time for b.d and one time for c.d, but
value1 should be the same as value2.
RandomNumber is something that shouldn't be CTFEable, if it does what
the name says.
Re: custom attribute proposal (yeah, another one)
Le 06/04/2012 22:46, Mafi a écrit :
Also, if I see:
@square(5) int foo();
How do I know that I have to use __traits(getAttribute, foo, Area)?
Another possibility:
@attribute Area area(int w, int h) { return Area(w, h);}
@attribute Area area(Area a) { return a;}
Area square(int a) { return Area(a, a);}
@area(5, 5) int foo();
@area(square(5)) int bar();
-Steve
The second possibility looks good. Especially because the lack of
@attribute on square disallows @square.
Mafi
This is adding code just for the pleasure of adding more code. Why wan't
I construct Area directly as attribute ?
Re: custom attribute proposal (yeah, another one)
On 7 April 2012 06:29, Kapps opantm2+s...@gmail.com wrote: On Friday, 6 April 2012 at 13:23:03 UTC, Steven Schveighoffer wrote: OK, so I woke up this morning to find a huge discussion on attributes, and I'd like to once again propose the idea of how to define and use an attribute. I do not like the idea of: @attr(identifier) Why? Because what the hell did we create that @ syntax for? I though it was to avoid misinterpreting such things as normal symbols, and avoid creating new keywords. Why should the compiler be the only one able to use such symbols? Another thing I don't like is some idea that only a certain type of construct can be the identifier. An attribute should have one requirement -- that it can be created/determined at compile time. Either this or the one that's the same just with structs is the way to go. The original proposal by Walter is good, it's just a little verbose. I slightly prefer this function method over the struct method because: 1) No need to generate a custom struct for everything. Plenty of things are just a true or false, or a string. Saves a little bit of TypeInfo generation. 2) The more important one: The possibility to eventually include an alias template parameter. This allows things like looking up whether the symbol with the attribute has other attributes applied, or determining type. This allows things like constraints, and can be a nice benefit. On the topic of type vs storage, it is useful to be able to apply attributes to a type, but this should /not/ change the type itself. It must be transparent to the user what attributes a type has unless they're actually accessing attributes. Generating a struct for an attribute is fine. It's not like you go on a custom attribute frenzy attributing everything with different stuff. You may have a few useful attributes, and those given by libs that you just use. Why can't you use alias template parameters in a struct definition in just the same way? Structs are definitely preferable in my opinion, for the fact that they can have methods and properties and stuff. If you get an attribute of something, being about to use methods on it, or access calculated data via properties will be useful. I see no reason to name an attribute differently than the thing that happens to define it.
Re: custom attribute proposal (yeah, another one)
On Saturday, 7 April 2012 at 11:13:54 UTC, deadalnix wrote: Le 07/04/2012 05:29, Kapps a écrit : I slightly prefer this function method over the struct method because: 1) No need to generate a custom struct for everything. Plenty of things are just a true or false, or a string. Saves a little bit of TypeInfo generation. If the type isn't used at runtime, the compiler should be able to remove that dead part of the code. If it doesn't, it is an implementation issue, and shouldn't be solved by language design decision. This is not possible currently. The TypeInfo is required at runtime whether or not the type is used at compile-time, for reasons such as Object.factory.
Re: custom attribute proposal (yeah, another one)
On Saturday, 7 April 2012 at 11:25:15 UTC, Manu wrote: Generating a struct for an attribute is fine. It's not like you go on a custom attribute frenzy attributing everything with different stuff. You may have a few useful attributes, and those given by libs that you just use. Why can't you use alias template parameters in a struct definition in just the same way? Structs are definitely preferable in my opinion, for the fact that they can have methods and properties and stuff. If you get an attribute of something, being about to use methods on it, or access calculated data via properties will be useful. I see no reason to name an attribute differently than the thing that happens to define it. The calling methods is a valid point, however the method can return a struct as well. Ultimately, I don't think it makes a large difference at all which is used. I'm just leaning towards methods because there's less bloat, no issues with this() like with a struct, and can be slightly simpler in certain situations. Again, it's mostly minor things. I'd be quite happy with either approach.
Re: custom attribute proposal (yeah, another one)
On 7 April 2012 14:35, Kapps opantm2+s...@gmail.com wrote: On Saturday, 7 April 2012 at 11:25:15 UTC, Manu wrote: Generating a struct for an attribute is fine. It's not like you go on a custom attribute frenzy attributing everything with different stuff. You may have a few useful attributes, and those given by libs that you just use. Why can't you use alias template parameters in a struct definition in just the same way? Structs are definitely preferable in my opinion, for the fact that they can have methods and properties and stuff. If you get an attribute of something, being about to use methods on it, or access calculated data via properties will be useful. I see no reason to name an attribute differently than the thing that happens to define it. The calling methods is a valid point, however the method can return a struct as well. Ultimately, I don't think it makes a large difference at all which is used. I'm just leaning towards methods because there's less bloat, no issues with this() like with a struct, and can be slightly simpler in certain situations. Again, it's mostly minor things. I'd be quite happy with either approach. Yeah I'm happy either way. At the end of the day, I guess whoever actually implements the feature will just follow their preference ;)
Re: Custom attributes (again)
On 2012-04-06 15:48, Adam D. Ruppe wrote:
On Friday, 6 April 2012 at 12:52:27 UTC, Jacob Carlborg wrote:
Just as we have today with keywords and symbols, I don't see the problem.
Things like safe aren't keywords though. They are just
magic identifiers.
Think of the implementation:
switch(tok.ident) {
case Id::safe:
case Id::property:
// etc for built ins
default:
// handle user ones here
}
If you were to define a struct safe {}, which is perfectly
legal D, and try to use it, you won't get what you expect.
It will trigger the case safe before it goes to the default.
So, the user defined one will just be quietly hidden.
I mean that int is a keyword and int is in the same scope as the
rest of the symbols. So if we allow custom attributes, like @foo, it's
the same problem as with regular identifiers. Regular identifiers can
conflict with keywords.
--
/Jacob Carlborg
Re: custom attribute proposal (yeah, another one)
On 2012-04-06 19:37, Steven Schveighoffer wrote:
On Fri, 06 Apr 2012 12:53:51 -0400, Piotr Szturmaj
struct Author { string name = empty; }
// struct Author { string name; } - this works too
I think the point is, we should disallow:
@Author int x;
-Steve
Why?
--
/Jacob Carlborg
Re: Discussion on Go and D
On 2012-04-06 19:37, Rainer Schuetze wrote: GC issues like this are currently blocking development of Visual D (a Win32 project): when just adding spaces to a file, parsing the new file every other second often needs 10 or more parsings until an equal amount of memory is collected compared to the allocated memory. AFAICT Visual D just keeps a reference to the root of the most recent AST of a source file. What's even worse: when the allocated memory gets larger (say 200MB), the garbage collection itself takes more than a second stalling the application, which is a real pain if it happens while you are typing source text (it does happen quite often). Can you pause the GC when the user is typing? When you're finished with the processing you can start it again. -- /Jacob Carlborg
Re: custom attribute proposal (yeah, another one)
On 2012-04-06 19:36, Steven Schveighoffer wrote:
so now I must define a type for every attribute? I'd rather just define
a function.
What if I have 20 string attributes, I must define a new attribute type
for each one? This seems like unneeded bloat.
If we want to be able to pass a key-value list to the attribute, I think
a struct is needed.
@attribute struct Author
{
string name;
string email;
}
@Author(name = John Doe, email = j...@doe.com) struct Foo {}
BTW, could both structs and functions be allowed?
--
/Jacob Carlborg
Re: custom attribute proposal (yeah, another one)
On 2012-04-06 19:51, bls wrote:
Why not being more flexible .. Likewise
struct Annotation //Throw in all your annotations
{
Variant[] [string] map;
Variant[] opDispatch(string key)()
{
return map[key];
}
// Single value
Variant[] opDispatch(string key, T) (T t )
if ( !isArray!T !isTuple!T )
{
index ~= key;
map[key] ~= to!Variant(t);
return map[key];
}
Array, Tuple
}
well.. I am not sure about CTFE
Variant yet again. What with this Variant all the time.
--
/Jacob Carlborg
Re: custom attribute proposal (yeah, another one)
On 2012-04-06 20:52, Steven Schveighoffer wrote: Also, if I see: @square(5) int foo(); How do I know that I have to use __traits(getAttribute, foo, Area)? Isn't square the name of the attribute? In that case you would use: __traits(getAttribute, foo, square) -- /Jacob Carlborg
Re: custom attribute proposal (yeah, another one)
On 2012-04-07 00:40, Piotr Szturmaj wrote: Ok, but it needs more work in the compiler, comparing to identifier search and remembering expression tuple of a symbol. Also, I just found a major drawback of this approach: consider parameterless attributes like @NotNull. What would you return from function named NotNull()? void ? -- /Jacob Carlborg
Re: custom attribute proposal (yeah, another one)
Jacob Carlborg wrote: On 2012-04-07 00:40, Piotr Szturmaj wrote: Ok, but it needs more work in the compiler, comparing to identifier search and remembering expression tuple of a symbol. Also, I just found a major drawback of this approach: consider parameterless attributes like @NotNull. What would you return from function named NotNull()? void ? I know that's the answer, but how would you store void, and get it from __traits(getAttributes) ?
Re: Discussion on Go and D
On 4/7/2012 12:44 AM, Manu wrote: On 7 April 2012 01:08, Rainer Schuetze r.sagita...@gmx.de mailto:r.sagita...@gmx.de wrote: I don't think there are many places in the code where these hints might apply. Are there known ways of hunting down false references? Still, my main concern are the slow collections that stall the application when a decent amount of memory is used. Removing false pointers won't change that, just make it happen a little later. An obvious best-practise is to allocate in fewer-larger blocks. Ie, more compounds and aggregates where possible. I also expect you are doing a lot of string processing. Using D strings directly? I presume you have a stack-string class? Put as many working strings on the stack as possible... There isn't a lot of string processing involved: tokens take a slice on the original text, and nodes of the AST seldomly save more than the identifier which just the same slice. So the full text always remains in memory, but this is only small part of the actual footprint, the AST is a lot bigger. The nodes have child and parent references, so you keep the whole AST once there is a false pointer to any node. I could try breaking up this dependencies when I think the AST is no longer used, but that brings me back to manual memory management and thread synchronization (parsing uses std.parallelism).
Re: custom attribute proposal (yeah, another one)
Le 07/04/2012 13:32, Kapps a écrit : On Saturday, 7 April 2012 at 11:13:54 UTC, deadalnix wrote: Le 07/04/2012 05:29, Kapps a écrit : I slightly prefer this function method over the struct method because: 1) No need to generate a custom struct for everything. Plenty of things are just a true or false, or a string. Saves a little bit of TypeInfo generation. If the type isn't used at runtime, the compiler should be able to remove that dead part of the code. If it doesn't, it is an implementation issue, and shouldn't be solved by language design decision. This is not possible currently. The TypeInfo is required at runtime whether or not the type is used at compile-time, for reasons such as Object.factory. Object.factory is limited to classes IIRC.
Re: Discussion on Go and D
On 4/7/2012 8:24 AM, Dmitry Olshansky wrote: On 07.04.2012 2:08, Rainer Schuetze wrote: On 4/6/2012 8:01 PM, Walter Bright wrote: On 4/6/2012 10:37 AM, Rainer Schuetze wrote: I hope there is something wrong with my reasoning, and that you could give me some hints to avoid the memory bloat and the application stalls. A couple of things you can try (they are workarounds, not solutions): 1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free(). Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this. How about this: http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/ Or you can wrap-up something similar along the same lines. Thanks for your and other's hints on reducing garbage collected memory, but I find it hard to isolate larger blocks of memory for manual management. Most of the structs and classes are rather small. I'm rather unhappy to sell D with the hint Go back to manual memory management if you need more than 64MB of memory and want your application to be responsive.
Re: Discussion on Go and D
On 7 April 2012 17:03, Jacob Carlborg d...@me.com wrote: On 2012-04-06 19:37, Rainer Schuetze wrote: GC issues like this are currently blocking development of Visual D (a Win32 project): when just adding spaces to a file, parsing the new file every other second often needs 10 or more parsings until an equal amount of memory is collected compared to the allocated memory. AFAICT Visual D just keeps a reference to the root of the most recent AST of a source file. What's even worse: when the allocated memory gets larger (say 200MB), the garbage collection itself takes more than a second stalling the application, which is a real pain if it happens while you are typing source text (it does happen quite often). Can you pause the GC when the user is typing? When you're finished with the processing you can start it again. There's a bit of a problem there though, when you're coding, when are you NOT typing? :) I don't ever stop and sit there patiently for a few seconds for no reason.
Re: Discussion on Go and D
On 7 April 2012 17:38, Rainer Schuetze r.sagita...@gmx.de wrote: On 4/7/2012 12:44 AM, Manu wrote: On 7 April 2012 01:08, Rainer Schuetze r.sagita...@gmx.de mailto:r.sagita...@gmx.de wrote: I don't think there are many places in the code where these hints might apply. Are there known ways of hunting down false references? Still, my main concern are the slow collections that stall the application when a decent amount of memory is used. Removing false pointers won't change that, just make it happen a little later. An obvious best-practise is to allocate in fewer-larger blocks. Ie, more compounds and aggregates where possible. I also expect you are doing a lot of string processing. Using D strings directly? I presume you have a stack-string class? Put as many working strings on the stack as possible... There isn't a lot of string processing involved: tokens take a slice on the original text, and nodes of the AST seldomly save more than the identifier which just the same slice. So the full text always remains in memory, but this is only small part of the actual footprint, the AST is a lot bigger. The nodes have child and parent references, so you keep the whole AST once there is a false pointer to any node. I could try breaking up this dependencies when I think the AST is no longer used, but that brings me back to manual memory management and thread synchronization (parsing uses std.parallelism). Hmm. Well I have to say, thus far, I personally have basically no confidence in the D GC, and tend to avoid it where possible. I only use it for trivial/transient allocations, and for convenience where it makes sense. In my work so far, I still tend to perform manual memory management for all major resources (this stems largely from the fact that in my line of work, allocations need to be predictable, and often pooled). If D supported allocators, and I had control over the allocation source, then maybe I'd change my usage patterns...
Re: Discussion on Go and D
On 07.04.2012 18:43, Rainer Schuetze wrote: On 4/7/2012 8:24 AM, Dmitry Olshansky wrote: On 07.04.2012 2:08, Rainer Schuetze wrote: On 4/6/2012 8:01 PM, Walter Bright wrote: On 4/6/2012 10:37 AM, Rainer Schuetze wrote: I hope there is something wrong with my reasoning, and that you could give me some hints to avoid the memory bloat and the application stalls. A couple of things you can try (they are workarounds, not solutions): 1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free(). Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this. How about this: http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/ Or you can wrap-up something similar along the same lines. Thanks for your and other's hints on reducing garbage collected memory, but I find it hard to isolate larger blocks of memory for manual management. Most of the structs and classes are rather small. Then clearly you need a custom allocator/allocation scheme. Most likely a mark/release or a free list kind of thing. Like say TempAlloc by David. As standard allocator design is still in motion you'd have to do your own thing ATM. Parsers and lexers are notable examples where doing custom allocation pays off nicely. I'm rather unhappy to sell D with the hint Go back to manual memory management if you need more than 64MB of memory and want your application to be responsive. I totally understand this sentiment, and unless GC improves by an order of magnitude it is not going to work well with large to medium-scale apps. Particularly long running ones, I once had been running VisualD for about 16 hours straight (back in the days of GSOC 2011) ;) -- Dmitry Olshansky
D and Heterogeneous Computing
Greetings! As someone with a research interest in software abstractions for image processing, the D programming language appears to offer unsurpassed language features for constructing beautiful and efficient programs. With that said, what would really get me to abandon C++ is if D supported a heterogenous programming model. My personal inclination would be something closer to OpenACC than anything else I've seen available. Though only in the sense that I like the idea of writing code once and being able to compile/run/debug it with or without automatic vectorization/kernelization. Presumably we could achieve more elegant syntax with tighter integration into the language. Has anyone been working on anything like this? Is this something the community would be interested in seeing? What should the solution look like? One path forward could be a patch to the compiler to generate and execute OpenCL kernels for appropriately marked-up D code. While I'm new the the D language, I'd be happy to work on a proof of concept of this if it is something the community thinks would be valuable and I could get specific feedback about the right way to approach it.
Re: DIP16: Transparently substitute module with package
On 2012-04-07 02:25, Jonathan M Davis wrote: On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote: I feel like most people will still import the main package module, and not the submodules. I don't think I ever wrote a piece of java code that didn't have: import java.io.*; Which is actually considered bad practice in Java, though a lot of people do like to do it. What's generally considered good practice is to explicitly import every module/class, and Eclipse likes to underline imports in red if you don't. import foo.bar.* Is used all over the place in SWT. -- /Jacob Carlborg
Re: custom attribute proposal (yeah, another one)
On 2012-04-07 05:29, Kapps wrote: I slightly prefer this function method over the struct method because: 1) No need to generate a custom struct for everything. Plenty of things are just a true or false, or a string. Saves a little bit of TypeInfo generation. But you still need to create a function. 2) The more important one: The possibility to eventually include an alias template parameter. This allows things like looking up whether the symbol with the attribute has other attributes applied, or determining type. This allows things like constraints, and can be a nice benefit. This can't be done for structs? -- /Jacob Carlborg
Re: Discussion on Go and D
On 4/6/2012 6:20 PM, deadalnix wrote: Le 06/04/2012 18:07, Andrei Alexandrescu a écrit : A few more samples of people's perception of the two languages: http://news.ycombinator.com/item?id=3805302 Andrei I did some measurement on that point for D lately : http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/ I studied the GC a bit more and noticed a possible issue: - memory allocations are aligned up to a power of 2 = page size - the memory area beyond the actually requested size is left untouched when allocating - when the memory is collected, it is also untouched - the marking of references during collection does not know the requested size, so it scans the full memory block Result: When a collected memory block is reused by a smaller allocation, there might still be false pointers in the unused area. When I clear this data, my first impression is that it has improved the situation, but not enough. I'll have to create some non-interactive test to verify. Rainer
Re: Discussion on Go and D
On 7 April 2012 19:04, Dmitry Olshansky dmitry.o...@gmail.com wrote: On 07.04.2012 18:43, Rainer Schuetze wrote: On 4/7/2012 8:24 AM, Dmitry Olshansky wrote: On 07.04.2012 2:08, Rainer Schuetze wrote: On 4/6/2012 8:01 PM, Walter Bright wrote: On 4/6/2012 10:37 AM, Rainer Schuetze wrote: I hope there is something wrong with my reasoning, and that you could give me some hints to avoid the memory bloat and the application stalls. A couple of things you can try (they are workarounds, not solutions): 1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free(). Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this. How about this: http://blog.thecybershadow.**net/2010/07/15/data-d-** unmanaged-memory-wrapper-for-**d/http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/ Or you can wrap-up something similar along the same lines. Thanks for your and other's hints on reducing garbage collected memory, but I find it hard to isolate larger blocks of memory for manual management. Most of the structs and classes are rather small. Then clearly you need a custom allocator/allocation scheme. Most likely a mark/release or a free list kind of thing. Like say TempAlloc by David. As standard allocator design is still in motion you'd have to do your own thing ATM. Parsers and lexers are notable examples where doing custom allocation pays off nicely. I'm rather unhappy to sell D with the hint Go back to manual memory management if you need more than 64MB of memory and want your application to be responsive. I totally understand this sentiment, and unless GC improves by an order of magnitude it is not going to work well with large to medium-scale apps. Particularly long running ones, I once had been running VisualD for about 16 hours straight (back in the days of GSOC 2011) ;) Yeeesss.. I run VisualD for days at a time, and it just leaks memory until my computer chokes and crashes. It hovers between 1gb and 2gb usage under 'normal' usage for me, on a relatively small project (only 20-ish files). I am now in the habit if killing and restarting it regularly, but that's clearly not a good sign for real-world D apps...
Re: custom attribute proposal (yeah, another one)
On 2012-04-07 13:32, Kapps wrote: This is not possible currently. The TypeInfo is required at runtime whether or not the type is used at compile-time, for reasons such as Object.factory. Object.factory can only create instances of classes. -- /Jacob Carlborg
Re: custom attribute proposal (yeah, another one)
On 2012-04-07 16:30, Piotr Szturmaj wrote: Jacob Carlborg wrote: On 2012-04-07 00:40, Piotr Szturmaj wrote: Ok, but it needs more work in the compiler, comparing to identifier search and remembering expression tuple of a symbol. Also, I just found a major drawback of this approach: consider parameterless attributes like @NotNull. What would you return from function named NotNull()? void ? I know that's the answer, but how would you store void, and get it from __traits(getAttributes) ? The compiler would only store that the attribute is attached to the declaration. In this case only hasAttribute is of interest. -- /Jacob Carlborg
Re: Discussion on Go and D
On 2012-04-07 17:41, Manu wrote: On 7 April 2012 17:03, Jacob Carlborg d...@me.com mailto:d...@me.com wrote: On 2012-04-06 19:37, Rainer Schuetze wrote: GC issues like this are currently blocking development of Visual D (a Win32 project): when just adding spaces to a file, parsing the new file every other second often needs 10 or more parsings until an equal amount of memory is collected compared to the allocated memory. AFAICT Visual D just keeps a reference to the root of the most recent AST of a source file. What's even worse: when the allocated memory gets larger (say 200MB), the garbage collection itself takes more than a second stalling the application, which is a real pain if it happens while you are typing source text (it does happen quite often). Can you pause the GC when the user is typing? When you're finished with the processing you can start it again. There's a bit of a problem there though, when you're coding, when are you NOT typing? :) I don't ever stop and sit there patiently for a few seconds for no reason. It depends. I can do quite a lot of coding without typing. But yeah, I type a lot. It would basically turn on and off the GC, probably not good. -- /Jacob Carlborg
Re: Discussion on Go and D
On 07.04.2012 20:51, Manu wrote: [snip] I totally understand this sentiment, and unless GC improves by an order of magnitude it is not going to work well with large to medium-scale apps. Particularly long running ones, I once had been running VisualD for about 16 hours straight (back in the days of GSOC 2011) ;) Yeeesss.. I run VisualD for days at a time, and it just leaks memory until my computer chokes and crashes. It hovers between 1gb and 2gb usage under 'normal' usage for me, on a relatively small project (only 20-ish files). I am now in the habit if killing and restarting it regularly, but that's clearly not a good sign for real-world D apps... I just turned off a couple of cool things like highlighting syntax errors. Seems to be fast and fluid afterwards. -- Dmitry Olshansky
Re: Discussion on Go and D
On 4/7/2012 7:43 AM, Rainer Schuetze wrote: I'm rather unhappy to sell D with the hint Go back to manual memory management if you need more than 64MB of memory and want your application to be responsive. Sure, that's why I said it was a workaround, not a solution.
Re: DIP16: Transparently substitute module with package
On Saturday, April 07, 2012 18:45:15 Jacob Carlborg wrote: On 2012-04-07 02:25, Jonathan M Davis wrote: On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote: I feel like most people will still import the main package module, and not the submodules. I don't think I ever wrote a piece of java code that didn't have: import java.io.*; Which is actually considered bad practice in Java, though a lot of people do like to do it. What's generally considered good practice is to explicitly import every module/class, and Eclipse likes to underline imports in red if you don't. import foo.bar.* Is used all over the place in SWT. Like I said, some people do like to do it, but Eclipse doesn't like you to, and there are quite a few Java folks who argue that it's bad practice. I forget what the reasons were (maybe increased buld times due to pulling in more symbols or more issues with symbol conflicts - I don't recall), but I wasn't particularly convinced when I heard them. Regardless though, there's a definite contingent against importing with * in the Java community, and it was my understanding that that contigent was the majority of that community, but I don't know. - Jonathan M Davis
openssl example for D
Prior work:
1. Download OpenSSL D interface from
https://github.com/D-Programming-Deimos/openssl and extract
2. install openssl-static (or other package inclued
/usr/lib/libssl.a and /usr/lib/libcrypto.a)
3. before compile, put your source file into
D-Programming-Deimos-openssl-xxx
or use -I/xxx/D-Programming-Deimos-openssl-xxx with compiler.
Example code:
//Server
import std.stdio;
import std.socket;
import std.algorithm;
import core.thread;
import deimos.openssl.ssl;
import deimos.openssl.err;
static import std.c.stdio;
const ListenAddress = 127.0.0.1;
const ListenPort = 10443;
const RecvSize = 1024;
const CertFile = server_cert.pem;
const KeyFile = server_key.pem;
class SSLConnection : Thread{
Socket r;
char[RecvSize] buf;
uint len;
SSL_CTX *ctx;
SSL *ssl;
this(Socket r, SSL_CTX *ctx){
this.r = r;
this.ctx = ctx;
this.isDaemon(true);
super(run);
}
void run(){
writeln(new connection from , r.remoteAddress().toString());
ssl = SSL_new(ctx);
SSL_set_fd(ssl, r.handle());
sslAssert(SSL_accept(ssl) != -1);
while (r.isAlive()){
len = SSL_read(ssl, cast(void*) buf, RecvSize);
if (len = 0){
break;
}
writeln(ssl read , len, bytes: , buf[0 .. len]);
SSL_write(ssl, cast(const void*) buf[0 .. len], len);
}
if (r.isAlive()){
writeln(close connection from ,
r.remoteAddress().toString());
}
SSL_free(ssl);
r.close();
}
}
void sslAssert(bool ret){
if (!ret){
ERR_print_errors_fp(std.c.stdio.stderr);
throw new Exception(SSL_ERROR);
}
}
void initSSL(){
SSL_library_init();
OpenSSL_add_all_algorithms();
SSL_load_error_strings();
}
SSL_CTX *getCTX(string certfile, string keyfile){
SSL_CTX *ctx = SSL_CTX_new(SSLv3_server_method());
sslAssert(!(ctx is null));
sslAssert(SSL_CTX_use_certificate_file(ctx, cast(const char*)
certfile, SSL_FILETYPE_PEM) 0);
sslAssert(SSL_CTX_use_PrivateKey_file(ctx, cast(const char*)
keyfile, SSL_FILETYPE_PEM) 0);
sslAssert(SSL_CTX_check_private_key(ctx) 0);
return ctx;
}
int main(string[] args){
initSSL();
SSL_CTX *ctx = getCTX(CertFile, KeyFile);
Socket s = new TcpSocket();
s.setOption(SocketOptionLevel.SOCKET, SocketOption.REUSEADDR,
true);
s.bind(new InternetAddress(ListenAddress, ListenPort));
s.listen(10);
writef(listen %s:%d\n, ListenAddress, ListenPort);
while (s.isAlive()){
Thread ss = new SSLConnection(s.accept(), ctx);
ss.start();
}
s.close();
SSL_CTX_free(ctx);
return 0;
}
//Client
import std.stdio;
import std.socket;
import std.algorithm;
import std.string;
import core.thread;
import core.memory;
import deimos.openssl.ssl;
import deimos.openssl.err;
import deimos.openssl.sha;
static import std.c.stdio;
const ConnectAddress = 127.0.0.1;
const ConnectPort = 10443;
const BufSize = 1024;
const msg = test message;
void sslAssert(bool ret){
if (!ret){
ERR_print_errors_fp(std.c.stdio.stderr);
throw new Exception(SSL_ERROR);
}
}
string getSha256(string input){
char[SHA256_DIGEST_LENGTH] hash;
string hash_hex;
SHA256_CTX *sha256 = new SHA256_CTX;
SHA256_Init(sha256);
SHA256_Update(sha256, cast(const void*) input, input.length);
SHA256_Final(cast(ubyte*) hash, sha256);
for(int i=0; iSHA256_DIGEST_LENGTH; i++){
hash_hex ~= format(%02x, hash[i]);
}
return hash_hex;
}
string getCertInfo(X509 *cert){
char buf[BufSize];
uint len;
string data;
BIO* strio = BIO_new(BIO_s_mem());
X509_print(strio, cert);
//BIO_ctrl(strio, BIO_C_FILE_SEEK, 0, null);
while (true){
len = BIO_gets(strio, cast(char*) buf, BufSize);
if (len = 0){
break;
}
data ~= buf[0 .. len];
}
BIO_free(strio);
return data;
}
bool verifyCert(X509 *cert){
X509_print_fp(std.c.stdio.stdout, cert);
/* PEM_read... are broken
//std.c.stdio.FILE *fp = std.c.stdio.fopen(cast(char*) CertFile,
rb);
BIO* lcertio = BIO_new_file(cast(char*) server_cert.pem, rb);
X509* lcert = PEM_read_bio_X509(lcertio, null, null, null);
writeln(X509_verify(lcert, X509_get_pubkey(cert)));
X509_free(lcert);
Re: openssl example for D
On Saturday, 7 April 2012 at 17:55:29 UTC, lzzll wrote: 2. PEM_read... are broken (ld error), the example use sha256 to verify certificate. What exactly are the error messages you get? David
Re: openssl example for D
What exactly are the error messages you get? dmd test_client.d -w -L-lssl -L-lcrypto -L-ldl ./test_client test_client.o: In function `_D11test_client10verifyCertFPS6deimos7openssl4x5097x509_stZb': test_client.d:(.text._D11test_client10verifyCertFPS6deimos7openssl4x5097x509_stZb+0x2e): undefined reference to `_D6deimos7openssl3pem65__T14DECLARE_PEM_rwVAyaa4_58353039TS6deimos7openssl4x5097x509_stZ67__T16DECLARE_PEM_readVAyaa4_58353039TS6deimos7openssl4x5097x509_stZ71__T20DECLARE_PEM_read_bioVAyaa4_58353039TS6deimos7openssl4x5097x509_stZ17PEM_read_bio_X509UNbPS6deimos7openssl3bio6bio_stPPS6deimos7openssl4x5097x509_stPUPaiiPvZiPvZPS6deimos7openssl4x5097x509_st' collect2: ld returned 1 exit status --- errorlevel 1 And sorry for a memory leak problem, add X509_free(cert); before return true; //or return (cert_hash == some_const_value) to verify cert can fix.
Re: Discussion on Go and D
On Apr 7, 2012, at 9:04 AM, Dmitry Olshansky dmitry.o...@gmail.com wrote: I totally understand this sentiment, and unless GC improves by an order of magnitude it is not going to work well with large to medium-scale apps. Particularly long running ones, I once had been running VisualD for about 16 hours straight (back in the days of GSOC 2011) ;) I keep all my apps open pretty much indefinitely, and restart when things slow down too much. Safari, for example, tends to leak memory and needs to be restarted every few days on the outside.
Re: Discussion on Go and D
On Apr 7, 2012, at 9:45 AM, Rainer Schuetze r.sagita...@gmx.de wrote: On 4/6/2012 6:20 PM, deadalnix wrote: Le 06/04/2012 18:07, Andrei Alexandrescu a écrit : A few more samples of people's perception of the two languages: http://news.ycombinator.com/item?id=3805302 Andrei I did some measurement on that point for D lately : http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/ I studied the GC a bit more and noticed a possible issue: - memory allocations are aligned up to a power of 2 = page size - the memory area beyond the actually requested size is left untouched when allocating - when the memory is collected, it is also untouched - the marking of references during collection does not know the requested size, so it scans the full memory block Result: When a collected memory block is reused by a smaller allocation, there might still be false pointers in the unused area. When I clear this data, my first impression is that it has improved the situation, but not enough. I'll have to create some non-interactive test to verify. The unused area is typically zeroed out on allocation. Check GC.mallocNoSync.
Re: D and Heterogeneous Computing
On Sat, 07 Apr 2012 11:38:15 -0500, Josh Klontz josh.klo...@gmail.com wrote: Greetings! As someone with a research interest in software abstractions for image processing, the D programming language appears to offer unsurpassed language features for constructing beautiful and efficient programs. With that said, what would really get me to abandon C++ is if D supported a heterogenous programming model. My personal inclination would be something closer to OpenACC than anything else I've seen available. Though only in the sense that I like the idea of writing code once and being able to compile/run/debug it with or without automatic vectorization/kernelization. Presumably we could achieve more elegant syntax with tighter integration into the language. Has anyone been working on anything like this? Is this something the community would be interested in seeing? What should the solution look like? One path forward could be a patch to the compiler to generate and execute OpenCL kernels for appropriately marked-up D code. While I'm new the the D language, I'd be happy to work on a proof of concept of this if it is something the community thinks would be valuable and I could get specific feedback about the right way to approach it. I've been using D with CUDA via a high-level wrapper around the driver API. It works very nicely, but it doesn't address the language integration issues. Might I recommend looking into hooking up LDC to the PTX LLVM back-end. That would seem much faster than writing your own back-end.
Re: Discussion on Go and D
On 4/7/2012 8:26 PM, Sean Kelly wrote: On Apr 7, 2012, at 9:45 AM, Rainer Schuetzer.sagita...@gmx.de wrote: On 4/6/2012 6:20 PM, deadalnix wrote: Le 06/04/2012 18:07, Andrei Alexandrescu a écrit : A few more samples of people's perception of the two languages: http://news.ycombinator.com/item?id=3805302 Andrei I did some measurement on that point for D lately : http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/ I studied the GC a bit more and noticed a possible issue: - memory allocations are aligned up to a power of 2= page size - the memory area beyond the actually requested size is left untouched when allocating - when the memory is collected, it is also untouched - the marking of references during collection does not know the requested size, so it scans the full memory block Result: When a collected memory block is reused by a smaller allocation, there might still be false pointers in the unused area. When I clear this data, my first impression is that it has improved the situation, but not enough. I'll have to create some non-interactive test to verify. The unused area is typically zeroed out on allocation. Check GC.mallocNoSync.. That's where I added it ;-) But in fact it is in malloc/calloc and a few more places, though I haven't checked whether all the other calls of mallocNoSync handle the situation correctly. Almost all calls are to malloc anyway, so my patch doesn't change anything.
Re: uploading with curl
Answers below:
On Friday, 6 April 2012 at 13:24:03 UTC, Gleb wrote:
Hello guys,
I'm trying to use curl library to satisfy my file transfer needs
under Windows 7. I've spent all the day and the most of
functionality I have already tried works like a charm. But I
have
a few issues with upload function.
First of all, if I try to use something like:
auto client = FTP(192.168.110.58);
or:
upload!FTP(file.zip, 192.168.110.58);
curl wrapper does not understand we are trying to use
ftp-protocol and uses http instead, returning something like:
?xml version=1.0 encoding=iso-8859-1?
!DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0
Transitional//EN
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;
html xmlns=http://www.w3.org/1999/xhtml; xml:lang=en
lang=en
head
title400 - Bad Request/title
/head
body
h1400 - Bad Request/h1
/body
/html
Not a big deal, I'll use ftp://xx.xx.xx.xx; format everywhere
below.
This is not how it should be. Github url with fixes below...
Here is the code I'm trying to use to upload a local file to the
ftp-host with an authentication:
auto client = FTP();
client.setAuthentication(login, pass);
upload!FTP(file.zip, ftp://192.168.110.58/file.zip;,
client);
This will pass the authentication but won't upload the file.
Then I decided to take a look to the code of std.net.curl.upload
function and use a low-level API the same way to find the
solution. Here is what I got:
auto f = new std.stream.BufferedFile(file.zip,
FileMode.In);
scope (exit) f.close();
auto client = FTP(ftp://192.168.110.58;);
client.verbose(true);
client.setAuthentication(login, pass);
client.onSend = (void[] data)
{
return f.read(cast(ubyte[])data);
};
client.contentLength = cast(size_t)f.size;
client.perform();
It's basically the same as upload function. This authenticates
correctly, gets directory listing and then nothing happens:
* Connection #0 to host 192.168.110.58 left intact
QUIT
221 Goodbye.
* Closing connection #0
And it looks correct for me, why should it upload any file!?
I can reproduce this and included the fix for it.
So I decided to replace the last line of the code with the
following:
client.addCommand(epsv);
client.addCommand(stor file.zip);
client.perform();
Here are the results:
epsv
229 Entering Extended Passive Mode (|||12761|)
stor file.zip
* FTP response timeout
* Connection #0 to host 192.168.110.58 left intact
* Timeout was reached
QUIT
* server response timeout
* Closing connection #0
std.net.curl.CurlTimeoutException@std\net\curl.d():
6B2BD8C6 on handle 166CB60
This way the file was created on the server, but it's empty. It
looks like client.onSend statements are never executed.
Unfortunately, I didn't managed to find out why, it's somewhere
in object private RefCounted!Impl p (curl.d:1956), but I
didn't
find where it is.
So, what am I doing wrong? Does std.net.curl.upload work for you
correctly? How do I upload a file to the ftp-host with
authentication? What is RefCounted!Impl p and where do I find
it's p.curl.perform() method?
RefCount!Impl p is a reference counted struct Impl defined above
the declaration in the source file.
The p.curl is an instance of the Curl struct defined later in the
source file. That struct has a perform() method.
P.S.: Firewall is not the case, other ftp clients and
std.net.curl.download function work fine, rules for the program
are created (just in case).
Thank you in advance, any advice is really appreciated.
You have identified a couple of bugs. A corrected version of
curl.d is located at
https://github.com/jcd/phobos/blob/curlfixes/std/net/curl.d
I've created a pull requests to get it upstream.
/Jonas
Re: uploading with curl
On 4/7/12 3:14 PM, Jonas Drewsen wrote: I've created a pull requests to get it upstream. Merged: https://github.com/D-Programming-Language/phobos/pull/528 Andrei
TickDuration.to's second template parameter
Whenever I use TickDuration.to, I need to add the pesky second argument, e.g. TickDuration.to!(nsecs, uint). Would a default make sense there? Also, double doesn't work, although I sometimes could use that. Thoughts? Destroyers? Andrei
Re: Discussion on Go and D
On 04/06/2012 02:01 PM, Walter Bright wrote: On 4/6/2012 10:37 AM, Rainer Schuetze wrote: I hope there is something wrong with my reasoning, and that you could give me some hints to avoid the memory bloat and the application stalls. A couple of things you can try (they are workarounds, not solutions): 1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free(). I keep reading that 'delete' is going to go away. Is this even future-proof, or is code written in this fashion going to suffer a reckoning later on? As an aside: I think this point generalizes to avoid using the GC in cases where it is easy to do so. D is very good at this due to having expressive structs, scope keyword, array slices, and a number of other features that synergize to track ownership/lifetime of memory without GC help. 2. Null out pointers references when you are done with them. This helps reduce unwanted pinning of unused gc memory. 3. Minimize use of global memory, as that is a major source of source of roots.
Re: Discussion on Go and D
On 4/7/12 4:26 PM, Chad J wrote: I keep reading that 'delete' is going to go away. Is this even future-proof, or is code written in this fashion going to suffer a reckoning later on? Biggest confusions in the history of humankind: 1. Pyramids have been built by aliens; 2. Flying with a device heavier than air is impossible; 3. The ability to dispose of memory will disappear along with the delete keyword. Andrei
Re: Discussion on Go and D
On 04/07/2012 04:43 PM, Rainer Schuetze wrote: On 4/7/2012 8:24 AM, Dmitry Olshansky wrote: On 07.04.2012 2:08, Rainer Schuetze wrote: On 4/6/2012 8:01 PM, Walter Bright wrote: On 4/6/2012 10:37 AM, Rainer Schuetze wrote: I hope there is something wrong with my reasoning, and that you could give me some hints to avoid the memory bloat and the application stalls. A couple of things you can try (they are workarounds, not solutions): 1. Actively delete memory you no longer need, rather than relying on the gc to catch it. Yes, this is as unsafe as using C's free(). Actually, having to deal with lifetime issues myself takes away the biggest plus of the GC, so I am a bit reluctant to do this. How about this: http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/ Or you can wrap-up something similar along the same lines. Thanks for your and other's hints on reducing garbage collected memory, but I find it hard to isolate larger blocks of memory for manual management. Most of the structs and classes are rather small. As you apparently just re-parse the whole source and throw the old AST away, wouldn't it be rather simple? You could just create a region allocator and free all the memory at once after the re-parse. I'm rather unhappy to sell D with the hint Go back to manual memory management if you need more than 64MB of memory and want your application to be responsive. I think it is actually awesome that manual memory management is possible.
Re: Precise GC
On 4/7/12 9:59 PM, Walter Bright wrote: On 4/7/2012 7:58 PM, Chad J wrote: Hey, that sounds awesome. I think I geeked out a bit. Would this make it any easier to reference count types that can be statically proven to have no cyclical references? It has nothing to do with reference counting that I can think of. Nevertheless good food for thought. This is all very cool. Andrei
Re: Discussion on Go and D
On 04/07/2012 05:42 PM, Andrei Alexandrescu wrote: On 4/7/12 4:26 PM, Chad J wrote: I keep reading that 'delete' is going to go away. Is this even future-proof, or is code written in this fashion going to suffer a reckoning later on? Biggest confusions in the history of humankind: 1. Pyramids have been built by aliens; 2. Flying with a device heavier than air is impossible; 3. The ability to dispose of memory will disappear along with the delete keyword. Andrei Oh. Whoops. Thanks!
More ddoc complaints
I have a pull request up to remove the big misfeature of embedded html in ddoc, and it is pending action, from me, to answer some of Walter's concerns. But, I was updating the documentation for some of my code (using std.ddoc and my improveddoc post-processor to make it prettier and remove Javascript), and I hit pain. http://arsdnet.net/web.d/std_dom.html#Form.addValueArray It is extremely difficult to document a HTML library when your HTML examples are mis-interpreted as markup! Also, ddoc should outdent the code examples: http://arsdnet.net/web.d/std_cgi.html#Cgi.request The examples are indented in my source to line up with the declarations, but this indentation doesn't make sense in the output!
a pretty exciting result for parallel D lang rmd following defrag by name
These are measured times to unzip and then delete a 2GB folder in
Win7. Both are using the parallel rmd to remove the directory on
a regular hard drive. The first measurement is for an unzip of
the archive. The second is remove of the folder when no defrag
has been done. The third is unzip of the same archive.
Following it, I used a myDefrag script to sort the LCN positions
of all the files in the folder based on the full path name. They
describe this sort by name script on their website. Following
that I ran the rmd D program to remove the folder, and it took
only 3.7 secs ( vs 197 secs the first time). I thought I must
have done something wrong so I repeated the whole thing, and zipd
up the folder before deleting it and also looked at its
properties and poked around in it. Same 3.7 second delete.
I'll have to analyze what is happening, but this is a huge
improvement. If it is just the sequential LCN order of the
operations, it may be that I can just pre-sort the delete
operations by the file lcn number and get similar results. It
also makes a case for creating a zip and unzip implementations
that preserve the sort by filepath order.
G:\uzp tz.zip tz
unzipping: .\tz.zip
finished! time: 87066 ms
G:\rmd tz
removing: .\tz
finished! time:197182 ms
G:\uzp tzip.zip tz
unzipping: .\tzip.zip
finished! time: 86015 ms
G:\rmd tz
removing: .\tz
finished! time:3654 ms
Below is the simple sortByName defrag script that I ran prior to
the deletion.
# MyDefrag v4.0 default script: Sort By Name
#
# This is an example script.
Title('Sort By Name tz')
Description('
Sort all the files in G:\tz by name on all the selected disk(s).
')
WriteLogfile(MyDefrag.log,LogHeader)
VolumeSelect
Name(g:)
VolumeActions
AppendLogfile(MyDefrag.log,LogBefore)
FileSelect
DirectoryName(tz)
FileActions
SortByName(Ascending)
FileEnd
AppendLogfile(MyDefrag.log,LogAfter)
VolumeEnd
AppendLogfile(MyDefrag.log,LogFooter)
Re: Precise GC
On 4/7/12 8:56 PM, Walter Bright wrote: [snip] I think this is an exciting idea, it will enable us to get a precise gc by enabling people to work on it in parallel rather than serially waiting for me. I'm also very excited about this design, and will make time to help with the library part of the implementation. Maybe we can get a GSoC project on that. We already have a related proposal (lock-free GC). Andrei
Precise GC
Of course, many of us have been thinking about this for a looong time, and what is the best way to go about it. The usual technique is for the compiler to emit some sort of table for each TypeInfo giving the layout of the object, i.e. where the pointers are. The general problem with these is the table is non-trivial, as it will require things like iterated data blocks, etc. It has to be compressed to save space, and the gc then has to execute a fair amount of code to decode it. It also requires some significant work on the compiler end, leading of course to complexity, rigidity, development bottlenecks, and the usual bugs. An alternative Andrei and I have been talking about is to put in the TypeInfo a pointer to a function. That function will contain customized code to mark the pointers in an instance of that type. That custom code will be generated by a template defined by the library. All the compiler has to do is stupidly instantiate the template for the type, and insert an address to the generated function. The compiler need know NOTHING about how the marking works. Even better, as ctRegex has demonstrated, the custom generated code can be very, very fast compared with a runtime table-driven approach. (The slow part will be calling the function indirectly.) And best of all, the design is pushed out of the compiler into the library, so various schemes can be tried out without needing compiler work. I think this is an exciting idea, it will enable us to get a precise gc by enabling people to work on it in parallel rather than serially waiting for me.
Re: Precise GC
On 4/7/2012 7:58 PM, Chad J wrote: Hey, that sounds awesome. I think I geeked out a bit. Would this make it any easier to reference count types that can be statically proven to have no cyclical references? It has nothing to do with reference counting that I can think of.
Re: DIP16: Transparently substitute module with package
On Saturday, 7 April 2012 at 17:44:20 UTC, Jonathan M Davis wrote Like I said, some people do like to do it, but Eclipse doesn't like you to, and there are quite a few Java folks who argue that it's bad practice. I forget what the reasons were (maybe increased buld times due to pulling in more symbols or more issues with symbol conflicts - I don't recall), but I wasn't particularly convinced when I heard them. Regardless though, there's a definite contingent against importing with * in the Java community, and it was my understanding that that contigent was the majority of that community, but I don't know. - Jonathan M Davis http://stackoverflow.com/questions/147454/why-is-using-a-wild-card-with-a-java-import-statement-bad Their reasoning sounds more due to various packages reinventing things or being poorly split than actual flaws with package imports themselves. Besides, D already addresses the issues of indicating which one you want, without the horribly long package names that Java has. Seems to me their examples are more like 'import std.*' than 'import std.datetime.*'.
Re: Precise GC
Hey, that sounds awesome. I think I geeked out a bit. Would this make it any easier to reference count types that can be statically proven to have no cyclical references?
Re: Precise GC
On 4/7/12 9:49 PM, Andrei Alexandrescu wrote: On 4/7/12 8:56 PM, Walter Bright wrote: [snip] I think this is an exciting idea, it will enable us to get a precise gc by enabling people to work on it in parallel rather than serially waiting for me. I'm also very excited about this design, and will make time to help with the library part of the implementation. Maybe we can get a GSoC project on that. We already have a related proposal (lock-free GC). BTW one exciting thing about both this and the nascent attributes design is they integrate wonderful with language's generic and generative capabilities. Andrei
Re: Precise GC
That sounds cool, perhaps people can have customizable GC for specific applications? Looking forward to D having a precise GC
Re: TickDuration.to's second template parameter
On Saturday, 7 April 2012 at 20:59:48 UTC, Andrei Alexandrescu wrote: Whenever I use TickDuration.to, I need to add the pesky second argument, e.g. TickDuration.to!(nsecs, uint). Would a default make sense there? Also, double doesn't work, although I sometimes could use that. Thoughts? Destroyers? Andrei Isn't this what the TickDuration.msecs / seconds / etc properties are for?
std.benchmark ready for review. Manager sought after
Hello, I finally found the time to complete std.benchmark. I got to a very simple API design, starting where I like it: one line of code. Code is in the form of a pull request at https://github.com/D-Programming-Language/phobos/pull/529. (There's some noise in there caused by my git n00biness). Documentation is at http://erdani.com/d/web/phobos-prerelease/std_benchmark.html. If reasonable and at all possible, I'd like to bump the priority of this proposal. Clearly D's user base is highly interested in efficiency, and many of the upcoming libraries have efficiency a virtual part of their design. So we should get std.benchmark in soon and require that new addition come with benchmarks for their essential functionality. My vision is that in the future Phobos will have a significant benchmarks battery, which will help improving Phobos and porting to new platforms. Andrei
Re: Discussion on Go and D
On Apr 7, 2012, at 12:37 PM, Rainer Schuetze r.sagita...@gmx.de wrote: On 4/7/2012 8:26 PM, Sean Kelly wrote: On Apr 7, 2012, at 9:45 AM, Rainer Schuetzer.sagita...@gmx.de wrote: On 4/6/2012 6:20 PM, deadalnix wrote: Le 06/04/2012 18:07, Andrei Alexandrescu a écrit : A few more samples of people's perception of the two languages: http://news.ycombinator.com/item?id=3805302 Andrei I did some measurement on that point for D lately : http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/ I studied the GC a bit more and noticed a possible issue: - memory allocations are aligned up to a power of 2= page size - the memory area beyond the actually requested size is left untouched when allocating - when the memory is collected, it is also untouched - the marking of references during collection does not know the requested size, so it scans the full memory block Result: When a collected memory block is reused by a smaller allocation, there might still be false pointers in the unused area. When I clear this data, my first impression is that it has improved the situation, but not enough. I'll have to create some non-interactive test to verify. The unused area is typically zeroed out on allocation. Check GC.mallocNoSync.. That's where I added it ;-) But in fact it is in malloc/calloc and a few more places, though I haven't checked whether all the other calls of mallocNoSync handle the situation correctly. Almost all calls are to malloc anyway, so my patch doesn't change anything. Is it not already there? Maybe it was removed in all the GC optimizations? I certainly remember a memset(0) in there.
Re: TickDuration.to's second template parameter
On Saturday, April 07, 2012 15:59:57 Andrei Alexandrescu wrote: Whenever I use TickDuration.to, I need to add the pesky second argument, e.g. TickDuration.to!(nsecs, uint). Would a default make sense there? Well TickDuration.nsecs is a wrapper for TickDuration.to!(nsecs, long), TickDuration.msecs is a wrapper for TickDuration.to!(msecs, long), etc. So, that's basically how defaults were added. I question that it makes sense to add defaults to the to function itself - though having long chosen as the default doesn't really help you, since you'll either have to be explicit like you have been or cast using the default version. auto value = td.to!(nsecs, uint)(); or auto value = cast(uint)td.nsecs; Also, double doesn't work, although I sometimes could use that. As for double not working, that sounds like a bug. It uses __traits(isFloating, T) if the type isn't an integral type and therefore should work with floating point values. IIRC, SHOO originally had some other wrapper functions for defaulting to real, which I removed because it made it too confusing to keep the long vs real functions straight, but double and real should still work. It _does_ look like real is used with it in some of the unit tests, but I don't think double is. If it doesn't work though, that sounds like a bug (either in dmd or druntime, depending on why it doesn't work). - Jonathan M Davis
Re: TickDuration.to's second template parameter
On Saturday, April 07, 2012 15:59:57 Andrei Alexandrescu wrote: Whenever I use TickDuration.to, I need to add the pesky second argument, e.g. TickDuration.to!(nsecs, uint). Would a default make sense there? Also, double doesn't work, although I sometimes could use that. Thoughts? Destroyers? That's odd. I was sure that I responded to this earlier, but my post seems to have disappeared. So, I'll try again. TickDuration has the seconds, msecs, usecs, and nsecs functions which effectively gives you to with a default of long for each of the units supported by TickDuration. So, that's the closest that it has to default arguments. I don't think that it would really make sense to give to default arguments to to on top of that. So, if you really want uint rather than long, you'd do either auto value = td.to!(nsecs, uint)(); or auto value = cast(uint)td.nsecs; As for double, it should work. SHOO had had functions similar to seconds, msecs, usecs, and nsecs which defaulted to real, but I removed them a while back, because it made it too confusing to keep them all straight. But to should work with integral values with a size of at least 4 bytes as well as all floating point types. The unit tests do test real, but they don't test double at all. So, it's possible that there's a bug which prevents it from working, but both float and double should work. If they don't, it's a bug which needs to be reported and fixed. - Jonathan M Davis
Re: std.benchmark ready for review. Manager sought after
On Sunday, 8 April 2012 at 03:25:16 UTC, Andrei Alexandrescu wrote: Hello, I finally found the time to complete std.benchmark. I got to a very simple API design, starting where I like it: one line of code. Nice, some comments: 1) Is it possible to do something about the kilonanoseconds, and print the two-letter SI time abbreviation instead (μs/ns/ms/s/...)? 2) The meaning of epoch in this context is unfamiliar to me: Measurement is done in epochs. For each function benchmarked, the smallest time is taken over all epochs. 3) benchmark_relative_file read should be replaced with a language construct. E.g. a function call like relativeBenchmark(file read), or an enum value like getopt's. 4) There is a TODO in benchmarkSuspend's example.
Re: TickDuration.to's second template parameter
On Sunday, 8 April 2012 at 04:09:45 UTC, Jonathan M Davis wrote: As for double, it should work. SHOO had had functions similar to seconds, msecs, usecs, and nsecs which defaulted to real, but I removed them a while back, because it made it too confusing to keep them all straight. But to should work with integral values with a size of at least 4 bytes as well as all floating point types. The unit tests do test real, but they don't test double at all. So, it's possible that there's a bug which prevents it from working, but both float and double should work. If they don't, it's a bug which needs to be reported and fixed. - Jonathan M Davis That's actually something I've been wondering about for a while. I have never once (once I knew that they returned fixed point numbers and fixed the bugs that caused) used the msecs/seconds/etc properties, because you almost never want such low precision. It would be much nicer to be able to just write .seconds instead of .to!(float, seconds), but returning a ulong prevents me being able to do that. Is there any reason those properties don't return float or double? It's probably too late to do anything about that at this point though even if it was desired.
Re: TickDuration.to's second template parameter
On Sunday, April 08, 2012 06:45:20 Kapps wrote: On Sunday, 8 April 2012 at 04:09:45 UTC, Jonathan M Davis wrote: As for double, it should work. SHOO had had functions similar to seconds, msecs, usecs, and nsecs which defaulted to real, but I removed them a while back, because it made it too confusing to keep them all straight. But to should work with integral values with a size of at least 4 bytes as well as all floating point types. The unit tests do test real, but they don't test double at all. So, it's possible that there's a bug which prevents it from working, but both float and double should work. If they don't, it's a bug which needs to be reported and fixed. - Jonathan M Davis That's actually something I've been wondering about for a while. I have never once (once I knew that they returned fixed point numbers and fixed the bugs that caused) used the msecs/seconds/etc properties, because you almost never want such low precision. It would be much nicer to be able to just write .seconds instead of .to!(float, seconds), but returning a ulong prevents me being able to do that. Is there any reason those properties don't return float or double? It's probably too late to do anything about that at this point though even if it was desired. There were properties which did both, but they were confusing, because it wasn't obvious which used long and which used real. Using long matched everything else in core.time and std.datetime, and none of the rest used floating point values for anything, so that's what was kept. But you should be able to use to!(float, seconds). If that doesn't work, it's a bug. float, double, and real should all work with TickDuration's to. - Jonathan M Davis
Re: std.benchmark ready for review. Manager sought after
On 4/7/12 11:45 PM, Vladimir Panteleev wrote: On Sunday, 8 April 2012 at 03:25:16 UTC, Andrei Alexandrescu wrote: Hello, I finally found the time to complete std.benchmark. I got to a very simple API design, starting where I like it: one line of code. Nice, some comments: 1) Is it possible to do something about the kilonanoseconds, and print the two-letter SI time abbreviation instead (μs/ns/ms/s/...)? Cool. 2) The meaning of epoch in this context is unfamiliar to me: Measurement is done in epochs. For each function benchmarked, the smallest time is taken over all epochs. To me too :o). 3) benchmark_relative_file read should be replaced with a language construct. E.g. a function call like relativeBenchmark(file read), or an enum value like getopt's. No can do. Need a function name-based convention so we can automate scheduleForBenchmarking. 4) There is a TODO in benchmarkSuspend's example. Will destroy. Andrei
Small Buffer Optimization for string and friends
Walter and I discussed today about using the small string optimization in string and other arrays of immutable small objects. On 64 bit machines, string occupies 16 bytes. We could use the first byte as discriminator, which means that all strings under 16 chars need no memory allocation at all. It turns out statistically a lot of strings are small. According to a variety of systems we use at Facebook, the small buffer optimization is king - it just works great in all cases. In D that means better speed, better locality, and less garbage. For this to happen, we need to start an effort of migrating built-in arrays into runtime, essentially making them templates that the compiler lowers to. So I have two questions: 1. What happened to the new hash project? We need to take that to completion. 2. Is anyone willing to start the effort of migrating built-in slices into templates? Thanks, Andrei
Re: Up to date documentation on D implementation.
On 07.04.2012 8:51, ReneSac wrote: On Friday, 6 April 2012 at 01:33:10 UTC, Mike Parker wrote: DMD runs just fine on 64-bit Windows. Then why 32 bit Windows (Win32) operating system, such as Windows XP is put as a requirement? This should be corrected: http://dlang.org/dmd-windows.html Anyway, in the mean time I have setup GDC using the latest binaries, and it is working well. The only thing I noticed is that a simple Hello World took several seconds to compile, and ended up with 1.25MB (release, non-debug build)! how about strip it? + MinGW debug info is notoriously bloated (if it was debug build). And I thought that D was fast to compile... But then I discovered that switching to std.c.stdio made the compilation almost instantaneous, and the executable size a slightly more reasonable 408KB. It works, but that isn't really an option, as D strings aren't readily compatible with C strings... I know that the lower limiter in binary size is higher, due to the statically compiled runtime, but this bloat in the std lib for a single function worries me a bit. Is DMD better in this measurement, or is it a limitation of the current D libraries? This may be kinda important latter, as in compression benchmarks, the decompressor size is added in the compressed size to prevent cheating. I don't want a multi-megabyte executable size. -- Dmitry Olshansky
Re: parallel unzip in progress
On Wednesday, 4 April 2012 at 19:41:21 UTC, Jay Norwood wrote: The work-around was to convert all the file operations to use std.stream equivalents, and that worked well, but I see i the bug reports that even that was only working correctly on windows. So I'm on windows, and ok for me, but it would be too bad to limit use to Windows. Seems like stdio runtime support for File operations above 2GB would be a basic expectation for a system language these days. btw, I posted a fix to setTimes that enables it to update the timestamp on directories as well as regular files, along with the source code of this example. I also did some research on why ntfs is such a dog when doing delete operations on hard drives, as well as spending several hours looking at procmon logs, and have decided that the problem is primarily related to multiple accesses in the master file table file for the larger files. There is much discussion on the matter of the MFT getting fragmented on these larger drives, and a couple of interesting proposed tweaks in the second link. http://ixbtlabs.com/articles/ntfs/index3.html http://www.gilsmethod.com/speed-up-vista-with-these-simple-ntfs-tweaks The second link shows you how to reserve a larger area for MFT, and the link below looks like it might be able to clean out any files from the reserved MFT spaces. http://www.mydefrag.com/index.html
Re: Win GUI Single Exe - Newbie
On 2012-04-06 17:37, Jesse Phillips wrote: I can't recommend whether D should or shouldn't be used based on your description. You can build stand alone GUI apps with DFL, and probably DWT. QtD and GtkD likely need their libraries shipped around. Documentation is pretty bad for DFL, but DWT has all the SWT docs and translation isn't bad. Building stand alone executables with DWT works great. DWT doesn't depend on any third party libraries, only on the system libraries. -- /Jacob Carlborg
Re: D1, D2, Tango, Phobos, Std, Runtime
On 2012-04-06 17:43, H. S. Teoh wrote: The whole Tango vs. Phobos debacle happened during D1 (which is a snapshot of an early version of D), allegedly due to some issues in Phobos. That issue no longer exists in D2, though. Phobos is now *the* standard library in D2. I don't think Tango is fully ported to D2 yet, although I heard someone was working on it, and IIRC there is a development snapshot of it available somewhere. Tango is fully ported to D2. It has some advantages over Phobos, like a faster XML parser, cryptographic related modules and some other modules that Phobos still doesn't have. -- /Jacob Carlborg
Re: SUL for Posix
On 05/04/2012 14:51, Jacob Carlborg wrote: snip http://dlang.org/phobos/std_getopt.html But it might not do what you want. Where is the code in std.getopt that has any relevance whatsoever to what smjg.libs.util.datetime or smjg.libs.util.commandline is for? Stewart.
GC-less tutorial?
Hi, are there any good tutorials for using D totally without GC, detailing some common pitfalls? One thing on my wishlist would be a way to detect accidental GC allocations resulting from side-effects of every-day operations... generating linking errors would be sufficient for detecting this...
Re: Up to date documentation on D implementation.
On Saturday, 7 April 2012 at 06:21:16 UTC, Dmitry Olshansky wrote: On 07.04.2012 8:51, ReneSac wrote: The only thing I noticed is that a simple Hello World took several seconds to compile, and ended up with 1.25MB (release, non-debug build)! how about strip it? + MinGW debug info is notoriously bloated (if it was debug build). I said it was a non-debug build. The debug build for the Hello World is 7.6MB.
Re: D1, D2, Tango, Phobos, Std, Runtime
On 04/07/2012 07:28 PM, Jacob Carlborg wrote: Tango is fully ported to D2. It has some advantages over Phobos, like a faster XML parser, cryptographic related modules and some other modules that Phobos still doesn't have. Tango is fully ported to D2? Really? You are talking about https://github.com/SiegeLord/Tango-D2, right?
Re: SUL for Posix
On 2012-04-07 14:36, Stewart Gordon wrote: On 05/04/2012 14:51, Jacob Carlborg wrote: snip http://dlang.org/phobos/std_getopt.html But it might not do what you want. Where is the code in std.getopt that has any relevance whatsoever to what smjg.libs.util.datetime or smjg.libs.util.commandline is for? Stewart. Both std.getopt and mjg.libs.util.commandline handle command line arguments? -- /Jacob Carlborg
Re: GC-less tutorial?
On 2012-04-07 17:25, Tove wrote: Hi, are there any good tutorials for using D totally without GC, detailing some common pitfalls? One thing on my wishlist would be a way to detect accidental GC allocations resulting from side-effects of every-day operations... generating linking errors would be sufficient for detecting this... The GC is plugable at link time, just remove it and you'll get linker errors when it's used. -- /Jacob Carlborg
Re: GC-less tutorial?
On 04/07/12 17:25, Tove wrote: Hi, are there any good tutorials for using D totally without GC, detailing some common pitfalls? One thing on my wishlist would be a way to detect accidental GC allocations resulting from side-effects of every-day operations... generating linking errors would be sufficient for detecting this... One problem with this is that the runtime wants to alloc some things before your code gets to run - so you can't just remove the functions from the build. You're left with either modifying the runtime or somehow ignoring the initial allocations (which usually won't cause any problems). If using a custom RT would have been an option you wouldn't be asking these questions (as you could just omit the symbols), so... If your platform supports gdb, try creating a text file trapallocs containing: b gc_malloc b gc_qalloc b gc_calloc b gc_realloc b gc_extend then run gdb -x ./trapallocs -ex run --args ./your_app The debugger will stop at the first allocation and you can then use bt to check what's going on, then c to skip to the next alloc. The first few will come from the runtime and various module ctors, but anything after that will be caused by your code, directly or indirectly. You can also trap just the array ops with eg: b _d_arraycatT b _d_arraycatnT b _d_arrayappendT b _d_arrayappendcTp b _d_newarrayT etc Once the runtime becomes a shared library simpler solutions will be possible, but, until then, it doesn't get much better than this. You need some way to determine which allocations are legal and which are not; doing this w/o a custom runtime is probably not worth the effort... And, yes, the initial runtime allocations could (and should) me made to use a different path; some of them shouldn't happen at all. For example std.datetime alone causes two allocs via _d_newclass, in every D app that imports std.stdio... artur
License for D setup
Greetings, I am getting interested in D. Pls, what License is for D setup. Is it free? Also, what operating systems can D run on? Win7, Mac,? Thanks...Vernon
Re: License for D setup
On Saturday, 7 April 2012 at 17:48:14 UTC, vmars316 wrote: Greetings, I am getting interested in D. Pls, what License is for D setup. Is it free? Also, what operating systems can D run on? Win7, Mac,? Thanks...Vernon Runs on Windows (x86), Linux (x86, 86_64), OS X (x86, x86_64) and FreeBSD (x86, x86_64). The compiler frontend is dual licensed under GPL/Artistic, the backend under a single-user non-redistribution source-available license. The standard and runtime libraries are Boost licensed.
Re: SUL for Posix
On 07/04/2012 17:54, Jacob Carlborg wrote: snip Both std.getopt and mjg.libs.util.commandline handle command line arguments? What's that to do with anything? If the code I need to finish smjg.libs.util.commandline is somewhere in std.getopt, please tell me where exactly it is. If it isn't, then why did you refer me to it? That's like telling someone who's writing a bigint library and struggling to implement multiplication to just look in std.math. After all, they both handle numbers. Stewart.
Re: License for D setup
q66 Wrote: On Saturday, 7 April 2012 at 17:48:14 UTC, vmars316 wrote: Greetings, I am getting interested in D. Runs on Windows (x86), Linux (x86, 86_64), OS X (x86, x86_64) and FreeBSD (x86, x86_64). The compiler frontend is dual licensed under GPL/Artistic, the backend under a single-user non-redistribution source-available license. The standard and runtime libraries are Boost licensed. Sounds good. Is there an up-todate Getting started with D programming? What to download, and where to install it. Also, is there a Visual GUI IDE? I have codeBlocks and GCC installed. Can D use wxWidgets? Evidently, C++ can use wxWidgets, but C can not. Which is why I am here. Thanks for your helps...vm
Re: License for D setup
On Sat, 07 Apr 2012 20:37:16 +0200, vmars316 vmars...@live.com wrote: Sounds good. Is there an up-todate Getting started with D programming? What to download, and where to install it. Also, is there a Visual GUI IDE? I have codeBlocks and GCC installed. Can D use wxWidgets? Evidently, C++ can use wxWidgets, but C can not. Which is why I am here. Thanks for your helps...vm http://dlang.org has a lot of information. Download: http://dlang.org/download.html Tutorial: http://www.informit.com/articles/article.aspx?p=1381876 Editors: http://prowiki.org/wiki4d/wiki.cgi?EditorSupport GUI libs: http://prowiki.org/wiki4d/wiki.cgi?GuiLibraries
