DVM - D Version Manager 0.2.0
I just released a new version of DVM, 0.2.0. For installation instructions see: https://bitbucket.org/doob/dvm Changelog: Version 0.2.0 New/Change Features * 64bit version now available on Linux * It's now possible to update an already existing DVM installation * Added an option for installing 32bit compilers, useful on 64bit platforms * Added support for the new structure of the DMD zip, appeared in version 1.068 and 2.053 * Added a current wrapper which points to the current compiler * Added an option for specifying a default compiler * Better compatible between different shells * Added support for installing Tango * Added support for installing 64bit compilers (default on 64bit platforms) * The fetch/install command now shows progress when downloading. Thanks to jdrewsen. * Added support for the new structure of the DMD zip, appeared in version 1.067 and 2.052. * Added a changelog. Bugs Fixed * RDMD now has executable permission * Exit if the DVM executable cannot be found * Always remove the temp path * Don't use exit in the DVM shell script * Added dmd.conf patch for druntime as well. * Fixed: DMD2 was incorrectly handled. * Bump version number. Sorry, still no version for Windows. I've seen another application that does the same but for Ruby, on Windows, so now I know it should be possible at least. -- /Jacob Carlborg
Goldie Parsing System v0.5 - Speed
Goldie Parsing System v0.5 is now out. This version focuses mainly on speed
improvements.
== Links: ==
Homepage and Documentation:
http://www.semitwist.com/goldie/
Prepackaged Downloads:
http://www.dsource.org/projects/goldie/browser/downloads
== New in v0.5: ==
- Improved lexing/parsing speed by about 5x-6x.
- Small additional speedup lexing languages with large character sets
(such as Unicode).
- GRMC: Grammar Compiler: Supports {All Valid} character set.
- GRMC: Grammar Compiler: Complex grammars are compiled to CGT up to
about 4x-8x faster.
- GRMC: Grammar Compiler: Verbose (-v) flag shows each step and amount
of time taken.
- Parse Anything: No more unhandled exception when parsing a source with
an error.
- Fixed to work with DMD 2.053 (still works with 2.052, too).
There are still more optimizations than can be done, but I felt this was
enough to warrant a new release.
Re: DVM - D Version Manager 0.2.0
Jacob Carlborg d...@me.com wrote in message news:iquopl$1l9u$1...@digitalmars.com... I just released a new version of DVM, 0.2.0. For installation instructions see: https://bitbucket.org/doob/dvm Changelog: Version 0.2.0 New/Change Features * 64bit version now available on Linux * It's now possible to update an already existing DVM installation * Added an option for installing 32bit compilers, useful on 64bit platforms * Added support for the new structure of the DMD zip, appeared in version 1.068 and 2.053 * Added a current wrapper which points to the current compiler * Added an option for specifying a default compiler * Better compatible between different shells * Added support for installing Tango * Added support for installing 64bit compilers (default on 64bit platforms) * The fetch/install command now shows progress when downloading. Thanks to jdrewsen. * Added support for the new structure of the DMD zip, appeared in version 1.067 and 2.052. * Added a changelog. Bugs Fixed * RDMD now has executable permission * Exit if the DVM executable cannot be found * Always remove the temp path * Don't use exit in the DVM shell script * Added dmd.conf patch for druntime as well. * Fixed: DMD2 was incorrectly handled. * Bump version number. Sorry, still no version for Windows. I've seen another application that does the same but for Ruby, on Windows, so now I know it should be possible at least. Sounds cool, but dvm-0.2.0-linux-32 is just giving me Illegal instruction on Kubuntu 10.04 x86-32. And I don't see any instructions for how to build it anywhere in the source tree or on the homepage.
Re: Curl wrapper
On Mon, 16 May 2011 17:07:57 -0400, jdrewsen jdrew...@nospam.com wrote:
Hi,
I've been working on a wrapper for the etc.c.curl module. It is now
pretty stable and I would very much like some feedback on the API.
http://freeze.steamwinter.com/D/web/phobos/etc_curl.html
BTW I use template mixins which doesn't seem to get included in the
generated docs. Is there any way I can make this work?
/Jonas
I think the API looks good with regard to the core functionality it
provides, but the documentation is sub-par, an certain structs should be
defined inside the Http class. I've listed a bunch of comments I had while
reading the documentation below.
One minor question regarding the actual implementation: do you define a
pragma(lib, ...) to ease setup/tool chain pains, or do you intend
something different?
Regarding the First example:
* Use string instead of const(char)[]
* Don't waste vertical space with empty comment lines like this
//
// This comment style
//
* Comment lines should be grammatically correct, i.e. '// GET using async
range' should be '// GET using an asynchronous range' or '// Perform a GET
using an asynchronous range', etc
* Don't use 'l' as a variable. It looks like a 1. Also, using 'line'
would make the meaning clearer
* Example code should not exceed 80 chars in width. People's browser
setups vary, and wrapped lines are bad.
* You can drop the 'delegate size_t' from the PUT example by making the
return 0u, 0U or size_t.init.
* The last example is missing a description. (Also, it goes over 80 chars)
The license section should include a mention of libcurl's license, etc in
addition to the wrapper's license. In general, for non-Boost libraries
like this, should we include the license/copyright as a string/tuple enum?
(i.e. for use in about boxes / command line options) Should said
license/copyright be register to a global repository upon module import?
(i.e. for auto-magical about boxes, etc)
'Do not use the same instance of this class in two threads simultanously.'
First, you need to spell check all your documentation (i.e. simultaneously
vs simultanously). Second, this should be assumed by most programmers to
be true of all non-shared classes. Third, if you do want to leave this
line in, use something short and sweet, like 'Http is not thread safe',
instead.
You need to look at/think about the order in which things are declared.
Having things like HttpMethod declared long after Http doesn't make any
sense. Also, it looks like you're not using '///ditto' for HttpMethod.
On that note, why is HttpMethod a free-standing enum and not declared
inside http? Http.Method, Http.Result, etc. make a lot more sense.
The setTimeCondition documentation seems a bit confusing.
I'd recommend stream-lining the convenience function documentation. For
example, http.head only requires a /// Returns: An HttpResult containing
the url's headers. Alternatively, you could use ditto and merge the
documentation and examples for head/get/etc into a single block. Also,
headAsync seems to document a data argument, but not take one, while post
has an undocumented parameter, etc. In general, you might want to consider
removing some of the redundant/self-obvious parameter documentation
blocks. On second thought, use ditto, slim down the docs and move this set
of functions either to the beginning or end of Http. It would make sense
to put these right after the definition of Http.Method/method. You also
might want to consider grouping all the xxxAsync functions together,
instead of interleaving them.
Check the grammar in http.postData
http.onReceiveHeader has issues with it's 'See' link. It's example is too
wide. You don't need really need the parameter documentation, particularly
if you improve the example, i.e.:
string[string] headerInformation;
with(auto http = new Http(http://www.google.com;)) {
onReceiveHeader = (string key, string value) {
if ( value.length = 10 )
headerInformation[key.idup] = value.idup;
};
onReceive = (ubyte[] data) { };
perform;
}
Also, an tab/indent should be 4 spaces not 5 and no example should start
indented. Remember to comment like this:
/** My regular comment
* Example:
---
// Example code
---
* Other Stuff:
*/
You need to fix/complete onSend's documentation.
contentLength(size_t len); = contentLength(size_t length);
'Perform http request' = 'Perform an http request'
AuthMethod/CurlAuth need to be part of docs.
Why is setAuthenticationMethod not authenticationMethod or authentication?
(And why is AuthMethod not AuthenticationMethod?)
Similarly, why is setTimeCondition not timeCondition and CurlTimeCond not
Http.TimeCondition?
Why is followLocation(int maxRedirs) not named maxRedirections?
HttpResult.text should have huge warning flags with regard to use and/or
have its design re-thought. Is
Re: Curl wrapper
On 17/05/11 02.43, dsimcha wrote: == Quote from jdrewsen (jdrew...@nospam.com)'s article Hi, I've been working on a wrapper for the etc.c.curl module. It is now pretty stable and I would very much like some feedback on the API. http://freeze.steamwinter.com/D/web/phobos/etc_curl.html BTW I use template mixins which doesn't seem to get included in the generated docs. Is there any way I can make this work? /Jonas This looks very good and I'd definitely be in favor of including it in the next release. However, there are a few small issues: 1. In the docs for Http, I don't think you need to say Do not use the same instance in two threads simultaneously. I think this is pretty much common sense. Same with the Ftp class. I included it because the libcurl handle used in a HTTP instance is not thread safe itself. What if I instantiated a shared instance of this class. Wouldn't that make the comment relevant in that access to the internal libcurl handle is not protected by mutexes? 2. A short overview of the asynch stuff at the top of the file would be nice. I'm a little confused about what it's for and when to use it instead of the regular synchronous functions. Does this allow you to process the beginning of a request while the end is still being received? Exactly. I'll add the missing overview. 3. I don't think you need to say how all the convenience functions are implemented, i.e. Internally this is implemented using an instance of the Http class. This is both likely to be assumed and an irrelevant implementation detail. You're right. Overall, nice work. Thank you for the feedback!
Re: Implementing std.log
On 2011-05-16 01:04, Andrei Alexandrescu wrote: On 05/15/2011 10:22 AM, Jacob Carlborg wrote: On 2011-05-14 22:37, Andrei Alexandrescu wrote: I want to nail usability before extension. Extensibility can be provided via an indirection inside FileLogger (which should indeed receive a more appropriate name). Andrei The obvious solution would be to have the logging functions returning an interface, Logger, but interfaces seem to be banned from Phobos. Ok, I see now, since templates are used this can be a problem. Hmm, templates are used EVERYWHERE in Phobos. No need to be ironic. Interfaces are the obvious solution, and what I meant when I said indirection (i.e. via a reference type, either interface or class). That being said, I think your design has a number of issues that I'll get into soon. Andrei It wasn't my intention to be ironic. I missed indirection and just read something about inside FileLogger. -- /Jacob Carlborg
Re: Implementing std.log
On 2011-05-16 01:27, Jonathan M Davis wrote: On 2011-05-15 16:01, Andrei Alexandrescu wrote: On 05/15/2011 10:26 AM, Jacob Carlborg wrote: On 2011-05-15 00:33, Andrei Alexandrescu wrote: On 05/14/2011 05:17 PM, Jonathan M Davis wrote: On 2011-05-14 14:56, Michel Fortin wrote: On 2011-05-14 17:31:30 -0400, Jonathan M Davisjmdavisp...@gmx.com said: So, I do think that knowing which thread is logging what could be very important for some programs, but I don't think that separating the log files is necessarily a good idea. If you did, you'd lose timing information (unless the time is at the beginning of every log line (which could also be useful), but then you'd have to read the times and compare them to see what happened before what). So, I'd be all for some options and extra information which could be added to each log line which would help debugging, but I don't think that thread-local logs is a great idea. I'd even go further and question whether it makes sense to have info, warning, and errors be written to separate files. I'd definitely vote for them all to be in the same file, but I don't generally see much benefit in having multiple log files. I like having them all in one place where you can see what happened in what order. Having them in separate log files is just going to make it harder to figure out what happened, and I think that it would become tempting (for me at least) to just log everything at exactly the same level so that they ended up in the same file. The info log contains log messages for all levels. Generally I'd want to stray from glog's major design decision as little as possible. It's an approach validated by years of experience in heavy-duty applications. Also, people coming from glog and used to its features would find it unpleasant that we decided to do things a different way without solid reasons. Andrei Why not? It doesn't say anywhere in the documentation, as far as I can see, that it's based on glog. That's beside the point as the documentation could and should be changed. I've said repeatedly it's based on glog, and I think it's a good thing. I think that, in general, that is a poor argument: Because library A does it like that we also have to do it like that. What about all people coming from library B, C, D and so on, it will be unpleasant for those as well because we didn't follow their particular library design. It's a good argument if library A is considered better by the person making the argument than B, C, and D. One thing is that glog has most desirable qualities I am looking for in a logging library: allows compile-time and run-time enabling/disabling, it's relatively small, it's simple, it's effective, and is widely used. I'm sure there are other libraries that score better on either dimension, but not on all. There are two departures I chose to make from glog: (a) I added the critical log which throws an exceptions (Google wouldn't need one because they don't use exceptions), and (b) I replaced the clunky LOG_IF_EVERY_N etc. with the flexible when(), every(), after() etc. I think these are well justified choices. I think that it makes good sense to start with an API based on an existing API which is known and heavily used and then adjust it as appropriate to our needs. In doing so, we might as well leave it the same as the original where we don't need to change it, since there is some gain for those familiar with the original. However, the real question then is where it does and doesn't make sense to change it. If enough people agree with Andrei on those decisions, then it will likely become std.log. However, if there's enough disagreement, and we can't reach an appropriate compromise on such changes, then it likely won't become std.log. Hopefully we can come a solid design based on glog with whatever changes we actually need without having to drastically change what it does in comparison to glog. That would be ideal. However, I also think that if it really looks like the general consensus is against something about how glog does things, we shouldn't be married to how glog works. So, overall, I agree with Andrei. The devil is the details, however. - Jonathan M Davis I agree with Jonathan, if glog is good enough we can follow its design but if it isn't, we shouldn't hesitate to make some design changes. The approach I take when designing an API for a library I'm writing I look at the API of several other libraries that does the same thing. Then I try to take the best parts and the parts I like of each library and combine them into something that make sense. -- /Jacob Carlborg
Re: should int/short/byte implicitly cast to dchar/wchar/char?
Walter Bright newshou...@digitalmars.com wrote in message
news:iqsu2b$1aqs$1...@digitalmars.com...
On 5/16/2011 9:09 PM, Jonathan M Davis wrote:
On 2011-05-16 21:04, Don wrote:
It's not easy. Simply disallowing it would make code like this illegal:
char c = '0' + n;
Well, forcing the n to be cast to char or byte would do it, but that _is_
kind
of ugly.
char c = '0' + cast(char)n;
Definitely a good point though.
It's THE reason why such casts are not allowed. Pascal requires such
casts, and I always hated it for that.
The casts are not only annoying, they can cause bugs when the types of the
rest of the expression are changed in maintenance.
There isn't any simple answer to mixed integer type arithmetic, although
there are many obvious answers that have subtly pernicious behavior.
I was going to suggest the rule:
char + numeric type == char (and likewise for wchar/dchar), provided that
the value range propogation works out, and if it doesn't then you do: char c
= cast(char)('0' + n)
Is that one of the obvious answers that has subtly pernicious behavior? If
so, I'm curious what the problem with it is.
Re: Curl wrapper
On 2011-05-16 23:07, jdrewsen wrote: Hi, I've been working on a wrapper for the etc.c.curl module. It is now pretty stable and I would very much like some feedback on the API. http://freeze.steamwinter.com/D/web/phobos/etc_curl.html BTW I use template mixins which doesn't seem to get included in the generated docs. Is there any way I can make this work? /Jonas I like the API, seems simple. -- /Jacob Carlborg
Re: Curl wrapper
On 17/05/11 06.43, Andrei Alexandrescu wrote: On 05/16/2011 04:07 PM, jdrewsen wrote: Hi, I've been working on a wrapper for the etc.c.curl module. It is now pretty stable and I would very much like some feedback on the API. http://freeze.steamwinter.com/D/web/phobos/etc_curl.html BTW I use template mixins which doesn't seem to get included in the generated docs. Is there any way I can make this work? /Jonas Now that's one sight for sore eyes. Nice work. A few comments and nits: 1. Http can't be a class. Network resources are the poster child of expensive resources that are NOT memory and don't work well with garbage collection. RAII is the poster child of deterministic resource management. So... I see what you mean. Will fix. 2. Furthermore I see customization via delegates (onReceive etc). That should be a good proxy for customization instead of using inheritance. The code actually defined the Http/Ftp classes as final. But somehow it is not reflected in the generated docs. Anyway, I will change it to struct and inheritance is not an issue anyway. 3. Data on wire is ubyte[], not void[]. Data received is ubyte[] and a convenience method for conversion to string is provided. Data send is void[] since all arrays implicitely converts to void[]. This makes e.g [1,2,3,4,5] works as input. Convenience methods are provided for chars in order to set HTTP headers correctly. For onSend callback it is simply void[] in order to use it as an out parameter. For Http.post/put it is const(void)[] For Http.postAsync/putAsync it is immutable(void)[] because internally I need to send the data to another thread using message passing. Hopes this makes sense? 4. Shouldn't onSend take a ref to its argument in case it wants to expand it? It is not allowed to expand it. The callback originates from libcurl which determines the size of the array. 5. Why HttpMethod and not Http.Method? I had the thought myself but decided on the former. By looking at the replies to my post it seems that it should be changed. I'll do that. 6. Do not use the same instance of this class in two threads simultanously. - remove. A D program isn't supposed to do that unless you offer shared methods. Ok. So even though I declare: shared Http http = new Http(); I cannot have multiple threads call http.perform without that method being marked as shared as well? If thats the case I'll remove the comment. 7. HttpStatusLine - Http.StatusLine. One up for encapsulation. 8. HttpMethod - Http.Method. Ditto. 9. this(in const(char)[] url); - this(string url); I assume you store a copy of the URL inside by calling e.g. to!string. If the client has a string, you refuse to tap into that nice information and share the string safely and efficiently, and make a copy of it for no good reason. If the client doesn't have a string, no worry, the compiler will tell her she needs to do a copy. I do not store a copy but calls libcurl which in turn does a copy by itself. So I believe the current signature is appropriate? 10. addHeader etc. Same comment. Use string whenever you want to keep a copy anyway. At 9. the string is copied by libcurl itself. 11. setTimeCondition - use core.Duration for time representation throughout. Will fix. 12. AsyncHttpResult - Http.AsyncResult :o) 13. HttpResult - Http.Result 14. Isn't the max redirect configurable via a parameter? Yes. It is called Http.followLocation from libcurls followLocation option. I will rename it to maxRedirections for clearity. 15. Typo: Callbacks is not supported... ok. 16. Documentation should point to descriptions of the HTTP methods wrapped (e.g. post, del etc). Do you mean point to the relevant RFC? 17. There should be an example with login/password. ok. 18. See onReceiveHeaderstd.curl.Curl.onReceiveHeader and others look like doc macros gone wrong. 19. chuncked - chunked 20. Max allowed redirs. -1 for infinite. - use uint and uint.max for infinite. Will fix. 21. short isRunning() - what's the semantics of the short? This is a bug. It returns an internal bool value. 22. byLine/byChunk should not expose a string or generally data that can be shared safely by the client. That's because it would need to create a new string with each iteration, which is very inefficient. Instead, they should expose char[]/ubyte[] just like File.byLine does, and reuse the buffer with each call. Essentially byLine/byChunk should be near-zero-overhead means to transfer and inspect arbitrarily large streams. byLine/byChunk is currenly only available for the async methods which is implemented using message passing. This means that for passing the message a copy has to be made because a message is by-value or immutable data. Is there another way for me to pass the message without doing a copy... some kind of move semantic? 23. Ftp should have get in-memory and streaming byLine/byChunk, not only get to file. Definitely. I wanted
Re: Curl wrapper
On 17/05/11 07.23, Vladimir Panteleev wrote: On Tue, 17 May 2011 07:43:06 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: 3. Data on wire is ubyte[], not void[]. I was the one who suggested using void[]. If data on wire is ubyte[], why is data on disk void[] (according to std.file.read/write)? I believe we decided on ubyte[] for reading and void[] for write in an earlier thread. And std.file should be fixed.
Re: Implementing std.log
On 2011-05-16 02:05, Andrei Alexandrescu wrote: Thanks for your work. I think there's an important distinction to be made. There are two APIs being discussed. One is the client interface and the other is the extensibility interface. Jose looked into both: he provided a client interface that has formatting, levels, enabling, and such, and an extensibility interface that essentially is a simple output stream. My library explores the client interface and leaves the extensibility interface as an obvious piece of work that needs little agreement and minimal design effort. Finally, your library keeps the client interface to a minimum and focuses almost exclusively on the extensibility interface. In doing so, it makes few choices that I disagree with. Allow me to share some specific feedback. Note that my suggestion was just a simple and incomplete suggestion on how the API could look like. I only provided info, warning and error methods as examples, I'm not saying the API should only have these three levels. 1. It becomes dynamic too early. First, in order to disable logging during compilation, a barrier of aliased types is needed. The alias would choose either a null type that does nothing (see StaticNullLogger at https://github.com/andralex/phobos/blob/master/std/log.d), or a log that actually does something. I think your design can be adjusted to do that with relative ease. I didn't consider disabling logging during compilation at all. If people like the basic idea with my API then, of course, I would flesh it out and add support compile time disabling of the logging. 2. It becomes dynamic too early from a different perspective. The interface for extensibility is identical with the interface for use and as a consequence leaves too broad functionality unimplemented. A better design is to handle a bunch of decisions (is the log dynamically enabled? every N calls? every N seconds? before N seconds have passed? after N seconds have passed?) in a lightweight front end, i.e. _before_ arguments have been evaluated (this is crucial!). Also, the front end should take care of formatting minutiae, leaving the dynamic back-end the simple task of streaming text where it's supposed to be going. I think Jose's design did the right thing there by doing enabled checks and formatting in the front-end, leaving only transport to the back-end. 3. Another consequence is that extensibility points have the signature: Logger log(string file, long line, ...); which leaves the heavyweight formatting task to the antiquated ... interface (no static types left, meaning e.g. no structs defining toString can be logged). Even assuming that's not a problem, implementors of this function are left with a fair amount of work to do. The problem with variadic template methods are that they're not virtual. 4. The Logger class contains info, warning, and error Level objects. Each Level object in turn contains a reference to a Logger. This design leaves some odd shrapnel behind, e.g. one can get to the critical log from the info log. I thought it was a good idea that you could chain calls like this: log.info(foo).error(bar); 5. I've seen the notion of a default logging level elsewhere but I think it's inadequate for the info/warning/error/critical/fatal approach. It is adequate for verbosity levels, i.e. vlog and friends. For info/warning/etc. the decision of the appropriate level belongs to the caller. Ok. Just a note, nobody is forcing you to use the default logging level. 6. My initial design also contained a global log battery containing the info, warning, etc. objects as members. I found that design stilted because it provides absolutely no benefit besides a marginal log.info vs. logInfo, has no encapsulation advantage, is adverse to extensibility, and worse of all perpetuates poor object oriented design. It's best to call a spade a spade, so there should be five global objects because that's in keep with reality. Andrei I got the impression that you change log.info to logInfo just because you wanted user defined loggers to look the same. My suggestion shows that's not necessary. Also I tried to minimize the use of global variables. -- /Jacob Carlborg
Resolution of core.time.Duration...
...why it is in hnsec? I know that this resolution is used in Win32 API (file time), but since TickDuration may be 1 ns resolution, wouldn't it be better to make Duration to be stored with maximum (defined so far) resolution? Especially because Duration may not hold long intervals ( months) - so there is no problem with overflow. Thread.sleep() accepts Duration (or hnsec) as an argument, while system resolution is higher, and on some systems it is even possible that it can sleep less than 100ns. SysTime is also kept in hnsecs, while resolution of system time (on Linux at least) is 1ns. Sure, in case of SysTime it is all bound to overflow, but it depends how value is stored - if we split seconds and nanoseconds, it will be fine. Additionally, when accepting long values as an argument for duration it is more logically to use SI units :) /Alexander
Re: Resolution of core.time.Duration...
On 2011-05-17 03:15, Alexander wrote: ...why it is in hnsec? I know that this resolution is used in Win32 API (file time), but since TickDuration may be 1 ns resolution, wouldn't it be better to make Duration to be stored with maximum (defined so far) resolution? Especially because Duration may not hold long intervals ( months) - so there is no problem with overflow. Thread.sleep() accepts Duration (or hnsec) as an argument, while system resolution is higher, and on some systems it is even possible that it can sleep less than 100ns. SysTime is also kept in hnsecs, while resolution of system time (on Linux at least) is 1ns. Sure, in case of SysTime it is all bound to overflow, but it depends how value is stored - if we split seconds and nanoseconds, it will be fine. Additionally, when accepting long values as an argument for duration it is more logically to use SI units :) /Alexander hnsecs is the highest resolution that you can hold in a 64-bit integer and still have it cover a reasonable range. And remember that a Duration is what's used as the result of subtracting one SysTime from another, so making it more precise would be bad. As it is, it can't actually hold the difference of SysTime.max - SysTime.min. Because it's signed, it can only hold half that range. Also, the system clock definitely does not reach hnsecs on any system that I've seen. It generally maxes out at microseconds or a bit better than that. So, in most cases, using a precision greater than hnsecs doesn't gain you anything. I'd love to have greater precision, but it would require something larger than a 64-bit integer, which we don't have. Or it would require splitting up the seconds and sub-seconds which makes the types take up more space and makes the math that much worse. During the development and review process, it was decided that hnsecs was ideal. Sure, higher precision might be nice, but we don't have anything bigger than a 64-bit integer, and odds are that you couldn't really take advantage of the higher precision anyway. - Jonathan M Davis
Re: Curl wrapper
On 17/05/11 08.03, Robert Jacques wrote:
On Mon, 16 May 2011 17:07:57 -0400, jdrewsen jdrew...@nospam.com wrote:
Hi,
I've been working on a wrapper for the etc.c.curl module. It is now
pretty stable and I would very much like some feedback on the API.
http://freeze.steamwinter.com/D/web/phobos/etc_curl.html
BTW I use template mixins which doesn't seem to get included in the
generated docs. Is there any way I can make this work?
/Jonas
I think the API looks good with regard to the core functionality it
provides, but the documentation is sub-par, an certain structs should be
defined inside the Http class. I've listed a bunch of comments I had
while reading the documentation below.
One minor question regarding the actual implementation: do you define a
pragma(lib, ...) to ease setup/tool chain pains, or do you intend
something different?
I define a pragma yes.
Regarding the First example:
* Use string instead of const(char)[]
Please see my reply to Andrei
* Don't waste vertical space with empty comment lines like this
ok
//
// This comment style
//
* Comment lines should be grammatically correct, i.e. '// GET using
async range' should be '// GET using an asynchronous range' or '//
Perform a GET using an asynchronous range', etc
ok
* Don't use 'l' as a variable. It looks like a 1. Also, using 'line'
would make the meaning clearer
ok
* Example code should not exceed 80 chars in width. People's browser
setups vary, and wrapped lines are bad.
ok
* You can drop the 'delegate size_t' from the PUT example by making the
return 0u, 0U or size_t.init.
nice. will do.
* The last example is missing a description. (Also, it goes over 80 chars)
ok
The license section should include a mention of libcurl's license, etc
in addition to the wrapper's license. In general, for non-Boost
libraries like this, should we include the license/copyright as a
string/tuple enum? (i.e. for use in about boxes / command line options)
Should said license/copyright be register to a global repository upon
module import? (i.e. for auto-magical about boxes, etc)
I'll mention what license libcurl is under. Until a standard for a
special string/tuple has been decided upon I'll leave this out.
'Do not use the same instance of this class in two threads
simultanously.' First, you need to spell check all your documentation
(i.e. simultaneously vs simultanously). Second, this should be assumed
by most programmers to be true of all non-shared classes. Third, if you
do want to leave this line in, use something short and sweet, like 'Http
is not thread safe', instead.
I'll do a spell check. See reply to Andrei on the other issue.
You need to look at/think about the order in which things are declared.
Having things like HttpMethod declared long after Http doesn't make any
sense. Also, it looks like you're not using '///ditto' for HttpMethod.
I don't know what ditto does. I'll try to look at the ddoc documentation.
On that note, why is HttpMethod a free-standing enum and not declared
inside http? Http.Method, Http.Result, etc. make a lot more sense.
See reply to Andrei.
The setTimeCondition documentation seems a bit confusing.
I'll try to improve.
I'd recommend stream-lining the convenience function documentation. For
example, http.head only requires a /// Returns: An HttpResult containing
the url's headers. Alternatively, you could use ditto and merge the
documentation and examples for head/get/etc into a single block. Also,
headAsync seems to document a data argument, but not take one, while
post has an undocumented parameter, etc. In general, you might want to
consider removing some of the redundant/self-obvious parameter
documentation blocks. On second thought, use ditto, slim down the docs
and move this set of functions either to the beginning or end of Http.
It would make sense to put these right after the definition of
Http.Method/method. You also might want to consider grouping all the
xxxAsync functions together, instead of interleaving them.
As mentioned I need to figure out what ditto does.
headAsync comment does not match its signature. I'll fix.
I've noticed that trying out examples live in the docs may be activated
at some point. If that is the case I think it is nice to have running
examples of each method.
Check the grammar in http.postData
ok
http.onReceiveHeader has issues with it's 'See' link. It's example is
too wide. You don't need really need the parameter documentation,
particularly if you improve the example, i.e.:
string[string] headerInformation;
with(auto http = new Http(http://www.google.com;)) {
onReceiveHeader = (string key, string value) {
if ( value.length = 10 )
headerInformation[key.idup] = value.idup;
};
onReceive = (ubyte[] data) { };
perform;
}
I see your point. The question is how we would like phobos docs to be:
1, consistent with all parameters documented for each method and each
method explained and exemplified
2, minimal with examples and
Re: Resolution of core.time.Duration...
On 17.05.2011 12:34, Jonathan M Davis wrote: Also, the system clock definitely does not reach hnsecs on any system that I've seen. Duration (unlike TickDuration) is not tied to system clock only, AFAIK (at least, this is not mentioned in the documentation). Using rdtsc it is possible to obtain even more precision than 1ns. Also, Duration can be used to store intervals which are not really related to system clock at all. Or it would require splitting up the seconds and sub-seconds which makes the types take up more space and makes the math that much worse. OK, though I don't think that 4 or 8 bytes more will really make any noticeable change (many Phobos/druntime structures are over-sized anyway, IMHO) - time values are not used in big arrays, usually. As to math - for most operations involving calendar computations only seconds are relevant, and math itself takes literally nanoseconds to execute - so only functions which actually require better precision may use more complex math. What could be done, though - is to hide the internal format (hnsecs), i.e. do not expose it to functions like Thread.sleep() with ulong argument. /Alexander
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On Tue, 17 May 2011 00:04:07 -0400, Don nos...@nospam.com wrote:
Nick Sabalausky wrote:
Steven Schveighoffer schvei...@yahoo.com wrote in message
news:op.vvk48tn9eav7ka@localhost.localdomain...
(int/short/byte can sometimes implicitly cast to dchar/wchar/char)
What do you think?
Ick! Kill it!
It's not easy. Simply disallowing it would make code like this illegal:
char c = '0' + n;
don't you mean dchar c = '0' + n?
And I wouldn't mind if dchar + int = dchar. I also wouldn't mind
requiring a cast when range propagation proves the dchar could be invalid.
In any case, I think:
string s = hello;
s ~= 123;
looks like it should result in:
assert(s == hello123);
Especially anyone coming from any other language. They certainly would
not expect it to be hello{
To me, dchar is the more specialized type, it's like a typedef of uint --
you should be able to implicitly cast to uint/int, but not to dchar.
It's kind of like a pointer, you can do this:
int n;
int* ptr;
ptr += n;
but you can't do this:
ptr = n;
-Steve
Re: Resolution of core.time.Duration...
On Tue, 17 May 2011 06:15:54 -0400, Alexander aldem+dm...@nk7.net wrote: ...why it is in hnsec? I know that this resolution is used in Win32 API (file time), but since TickDuration may be 1 ns resolution, wouldn't it be better to make Duration to be stored with maximum (defined so far) resolution? if you use hnsecs, then you get a range of SysTime of -30k to 30k years. That might seem overkill, but consider that even going to dnsecs (10 nanoseconds) reduces your range to -3k to +3k years. The problem is that nobody is likely to care about that extra 10 intervals, but losing out on 27,000 years x 2 is pretty significant. It seems like a no-brainer to me. straight nanoseconds are not possible, because we couldn't even represent our current date with it. Plus, we have good precedence, both Microsoft and Tango use that tick duration. It's a natural conclusion. Especially because Duration may not hold long intervals ( months) - so there is no problem with overflow. A Duration is the result of subtracting two SysTime's, which uses hnsecs as its tick, so yeah, there is a problem with overflow if you use a finer resolution. Thread.sleep() accepts Duration (or hnsec) as an argument, while system resolution is higher, and on some systems it is even possible that it can sleep less than 100ns. The minimum sleep time for a thread is one clock period. If your OS is context switching more than once per 100ns, your OS is going to be doing nothing but context switching. Processors just aren't fast enough to deal with that (and likely won't ever be). 100 ns is a reasonable resolution for that. Real time applications may require more precise timing, but you would likely need a separate API for that. SysTime is also kept in hnsecs, while resolution of system time (on Linux at least) is 1ns. Sure, in case of SysTime it is all bound to overflow, but it depends how value is stored - if we split seconds and nanoseconds, it will be fine. Again, the resolution of the *structure* may be nsecs, but the actual intervals you have access to is about every 4ms on linux ( see http://en.wikipedia.org/wiki/Jiffy_(time) ). If it makes you feel better to use higher resolution timing, the facilities are there, just use the C system calls. Additionally, when accepting long values as an argument for duration it is more logically to use SI units :) I agree that accepting a long as an alternative to Duration, it makes sense to use a more normal tick resolution. The chances of someone wanting to have their process sleep for more than 300 years (e.g. for nanosecond resolution) is pretty small. This might be a worthwhile change. I'm not sure how much code this might affect, though. It would be plenty disturbing if your code started sleeping for 100ms instead of the 10s you thought you requested. What might be a good path is to disable those functions that accept a long for a few releases, then re-instate them with a new meaning. -Steve
Re: Resolution of core.time.Duration...
On 17.05.2011 15:44, Alexander wrote: On 17.05.2011 12:34, Jonathan M Davis wrote: Also, the system clock definitely does not reach hnsecs on any system that I've seen. Duration (unlike TickDuration) is not tied to system clock only, AFAIK (at least, this is not mentioned in the documentation). Using rdtsc it is possible to obtain even more precision than 1ns. Actually, rdtsc is limited and useful mostly for performance measures, and has it's fare share of caveats: http://msdn.microsoft.com/en-us/library/ee417693.aspx or more practical: http://www.strchr.com/performance_measurements_with_rdtsc (also there are processors with dynamic frequency...) And like you said there are no APIs that use such precision. Also, Duration can be used to store intervals which are not really related to system clock at all. Or it would require splitting up the seconds and sub-seconds which makes the types take up more space and makes the math that much worse. OK, though I don't think that 4 or 8 bytes more will really make any noticeable change (many Phobos/druntime structures are over-sized anyway, IMHO) - time values are not used in big arrays, usually. As to math - for most operations involving calendar computations only seconds are relevant, and math itself takes literally nanoseconds to execute - so only functions which actually require better precision may use more complex math. The gain of using these fat integers is obscure so far. Unless, again, we are talking about small scale CPU performance measures. What could be done, though - is to hide the internal format (hnsecs), i.e. do not expose it to functions like Thread.sleep() with ulong argument. Right. /Alexander -- Dmitry Olshansky
Re: Resolution of core.time.Duration...
Am 17.05.2011 15:25, schrieb Steven Schveighoffer: I agree that accepting a long as an alternative to Duration, it makes sense to use a more normal tick resolution. The chances of someone wanting to have their process sleep for more than 300 years (e.g. for nanosecond resolution) is pretty small. This might be a worthwhile change. I'm not sure how much code this might affect, though. It would be plenty disturbing if your code started sleeping for 100ms instead of the 10s you thought you requested. What might be a good path is to disable those functions that accept a long for a few releases, then re-instate them with a new meaning. Or just add nanoSleep() or something like that. Cheers, - Daniel
Re: Resolution of core.time.Duration...
On 17.05.2011 15:25, Steven Schveighoffer wrote: if you use hnsecs, then you get a range of SysTime of -30k to 30k years. That might seem overkill, but consider that even going to dnsecs (10 nanoseconds) reduces your range to -3k to +3k years. The problem is that nobody is likely to care about that extra 10 intervals, but losing out on 27,000 years x 2 is pretty significant. It seems like a no-brainer to me. Well, if you put it this way, it seems reasonable. Plus, we have good precedence, both Microsoft and Tango use that tick duration. It's a natural conclusion. Linux/Posix are using ns (clock_gettime(), nanosleep() etc - timespec) - I guess there is a reason for this. Again, the resolution of the *structure* may be nsecs, but the actual intervals you have access to is about every 4ms on linux ( see http://en.wikipedia.org/wiki/Jiffy_(time) ). Not really. Take a difference from two consecutive get_clock() calls on Linux, and you will see that it is far below 1 µs (depends on CPU, though). Not all systems use timer interrupt for timekeeping. I agree that accepting a long as an alternative to Duration, it makes sense to use a more normal tick resolution. The chances of someone wanting to have their process sleep for more than 300 years (e.g. for nanosecond resolution) is pretty small. This might be a worthwhile change. Well, this is, actually, the whole reason of my post :) While 100ns resolution seems reasonable (after your explanations), accepting 100ns intervals as values directly seems not really good idea. /Alexander
Re: Generators in D
Piotr Szturmaj wrote:
Hi,
I've written some proof of concept code of generator pattern, example:
void genSquares(out int result, int from, int to)
{
foreach (x; from .. to + 1)
{
yield!result(x * x);
}
}
void main(string[] argv)
{
foreach (sqr; generator(genSquares, 10, 20))
writeln(sqr);
}
posted on github:
https://github.com/pszturmaj/dgenerators
Thanks to Sean Kelly who wrote Fiber code.
Piotr
Well, I've just found this:
http://www.mail-archive.com/digitalmars-d@puremagic.com/msg30204.html.
It seems someone did it before ;)
In regards to serializable fibers, I've found some interesting potential
usage - session handling in HTTP server. Fiber code could represent
session and could be saved to disk in the middle of execution, just like
sessions are saved in PHP. This is used to offload memory when handling
great number of sessions.
If serialization will be clever enough, these fibers could even be
shared across different servers! This is really a requirement in load
balanced environments.
Piotr
Re: Curl wrapper
On 5/17/11 12:23 AM, Vladimir Panteleev wrote: On Tue, 17 May 2011 07:43:06 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: 3. Data on wire is ubyte[], not void[]. I was the one who suggested using void[]. If data on wire is ubyte[], why is data on disk void[] (according to std.file.read/write)? Bug in std.file. Andrei
Re: Curl wrapper
Thanks for the response! A few more answers and comments within (everything deleted counts as sounds great). On 5/17/11 3:50 AM, Jonas Drewsen wrote: 3. Data on wire is ubyte[], not void[]. Data received is ubyte[] and a convenience method for conversion to string is provided. Data send is void[] since all arrays implicitely converts to void[]. This makes e.g [1,2,3,4,5] works as input. Convenience methods are provided for chars in order to set HTTP headers correctly. For onSend callback it is simply void[] in order to use it as an out parameter. For Http.post/put it is const(void)[] For Http.postAsync/putAsync it is immutable(void)[] because internally I need to send the data to another thread using message passing. Hopes this makes sense? I think it's best to systematically use void[] when sending data out (i.e. you start with structured data in memory and you let it implicitly decay as you pass it on the wire), and ubyte[] when receiving data (i.e. you start with unstructured data and force the user to explicitly build structure on it). As noted the current File API is not to be considered an example to follow. 6. Do not use the same instance of this class in two threads simultanously. - remove. A D program isn't supposed to do that unless you offer shared methods. Ok. So even though I declare: shared Http http = new Http(); I cannot have multiple threads call http.perform without that method being marked as shared as well? If thats the case I'll remove the comment. Yes, shared limits access to only shared fields and methods. 9. this(in const(char)[] url); - this(string url); I assume you store a copy of the URL inside by calling e.g. to!string. If the client has a string, you refuse to tap into that nice information and share the string safely and efficiently, and make a copy of it for no good reason. If the client doesn't have a string, no worry, the compiler will tell her she needs to do a copy. I do not store a copy but calls libcurl which in turn does a copy by itself. So I believe the current signature is appropriate? Got it, thanks. I sense a little inefficiency here that we can't quite get rid of - the user passes a string, we must convert it to a zero-terminated char* (one allocation) and then libcurl copies it again (two allocations). Oh well. 14. Isn't the max redirect configurable via a parameter? Yes. It is called Http.followLocation from libcurls followLocation option. I will rename it to maxRedirections for clearity. You mention this about many higher-level functions: Maximum redirections are set to 10. Then I'm thinking, if it's an important parameter, make it a defaulted parameter for the function. 16. Documentation should point to descriptions of the HTTP methods wrapped (e.g. post, del etc). Do you mean point to the relevant RFC? Yah, or a good tutorial. (I didn't know DEL existed!) 22. byLine/byChunk should not expose a string or generally data that can be shared safely by the client. That's because it would need to create a new string with each iteration, which is very inefficient. Instead, they should expose char[]/ubyte[] just like File.byLine does, and reuse the buffer with each call. Essentially byLine/byChunk should be near-zero-overhead means to transfer and inspect arbitrarily large streams. byLine/byChunk is currenly only available for the async methods which is implemented using message passing. This means that for passing the message a copy has to be made because a message is by-value or immutable data. Is there another way for me to pass the message without doing a copy... some kind of move semantic? A library element is planned but not on the short list yet. You can work around that by internally doing a cast. As long as you observe the commitment that buffers are not accessed simultaneously by more than one thread you're well within defined behavior. One more suggestion - you may want to also provide classic boring synchronous read/write methods that take a user-provided buffer. I'm sure people could use such e.g. when they want to stream data inside their own threads, or even in the main thread if they don't mind blocking. 24. The shutdown mechanism should be handled properly. Shutting down libcurl would have all pending transfers instantly throw a special exception. Without a shutdown API, applications won't be able to implement e.g. a Quit button. If you're only using this API this should be handled. But I see that it is not documented. Yah, in the future we need to put an onShutdown protocol inside core, similar to atexit() in C. Each library that may block would plant a hook. That way an application will be able to exit quickly and gracefully even if it has blocked threads. Andrei
Re: Resolution of core.time.Duration...
On Tue, 17 May 2011 09:42:27 -0400, Daniel Gibson metalcae...@gmail.com wrote: Am 17.05.2011 15:25, schrieb Steven Schveighoffer: I agree that accepting a long as an alternative to Duration, it makes sense to use a more normal tick resolution. The chances of someone wanting to have their process sleep for more than 300 years (e.g. for nanosecond resolution) is pretty small. This might be a worthwhile change. I'm not sure how much code this might affect, though. It would be plenty disturbing if your code started sleeping for 100ms instead of the 10s you thought you requested. What might be a good path is to disable those functions that accept a long for a few releases, then re-instate them with a new meaning. Or just add nanoSleep() or something like that. Probably a good idea, and deprecate Thread.sleep(long). It's more self-documenting. -Steve
Re: Resolution of core.time.Duration...
On Tue, 17 May 2011 09:55:49 -0400, Alexander aldem+dm...@nk7.net wrote: On 17.05.2011 15:25, Steven Schveighoffer wrote: Plus, we have good precedence, both Microsoft and Tango use that tick duration. It's a natural conclusion. Linux/Posix are using ns (clock_gettime(), nanosleep() etc - timespec) - I guess there is a reason for this. I just read the man page on time (page 7) and indeed, there is support for sub-jiffy timing. And Duration will support that notion quite well. If you need sub-hnsec timing, you can call these functions directly. The structures have used nanoseconds for over 10 years (I think gettimeofday used it back in the 90s!), so the reason for using it was likely for future compatibility (clearly nanosecond timing wasn't possible back then). It looks like the future is now, so it's good to have that resolution. I still maintain that for things like sleeping, it is pointless to sleep less than 100ns since it likely takes longer than that to do a context switch, and your sleep time is only guaranteed to be *greater* than what you requested. The only time this is useful is for very specific real-time applications. I could be wrong, there have obviously been improvements to the timing mechanisms of Linux since I last looked at it. As for measuring time, yes, it would be good to use a higher precision timer. And in fact, std.datetime.StopWatch does just that. http://www.digitalmars.com/d/2.0/phobos/std_datetime.html#StopWatch The core.time.Duration type is for low-level timing facilities, such as waiting on a condition or sleeping a thread. It is also beneficial to re-use the structure in std.datetime for timing facilities that are generically useful for most cases. This allows one to fluently do date calculations, timing calculations and pass them directly to low level facilities. There will always be those cases where the time resolution is not enough, and for those cases, you will need to use a more specialized API. Consider that Java, C#, and many other languages get by just fine with coarser resolution timing. IMO dealing with times less than 1ms is pretty specialized. Again, the resolution of the *structure* may be nsecs, but the actual intervals you have access to is about every 4ms on linux ( see http://en.wikipedia.org/wiki/Jiffy_(time) ). Not really. Take a difference from two consecutive get_clock() calls on Linux, and you will see that it is far below 1 µs (depends on CPU, though). Not all systems use timer interrupt for timekeeping. I stand corrected, I was not aware of these timing facilities. I agree that accepting a long as an alternative to Duration, it makes sense to use a more normal tick resolution. The chances of someone wanting to have their process sleep for more than 300 years (e.g. for nanosecond resolution) is pretty small. This might be a worthwhile change. Well, this is, actually, the whole reason of my post :) While 100ns resolution seems reasonable (after your explanations), accepting 100ns intervals as values directly seems not really good idea. Yes, hnsecs is not a typical concept one has to deal with. -Steve
Re: Curl wrapper
On Tue, 17 May 2011 07:24:16 -0400, Jonas Drewsen jdrew...@nospam.com
wrote:
On 17/05/11 08.03, Robert Jacques wrote:
On Mon, 16 May 2011 17:07:57 -0400, jdrewsen jdrew...@nospam.com
wrote:
Hi,
I've been working on a wrapper for the etc.c.curl module. It is now
pretty stable and I would very much like some feedback on the API.
http://freeze.steamwinter.com/D/web/phobos/etc_curl.html
BTW I use template mixins which doesn't seem to get included in the
generated docs. Is there any way I can make this work?
/Jonas
[snip]
I don't know what ditto does. I'll try to look at the ddoc documentation.
If a documentation comment for a declaration consists only of the
identifier ditto then the documentation comment for the previous
declaration at the same declaration scope is applied to this declaration
as well.
[snip]
http.onReceiveHeader has issues with it's 'See' link. It's example is
too wide. You don't need really need the parameter documentation,
particularly if you improve the example, i.e.:
string[string] headerInformation;
with(auto http = new Http(http://www.google.com;)) {
onReceiveHeader = (string key, string value) {
if ( value.length = 10 )
headerInformation[key.idup] = value.idup;
};
onReceive = (ubyte[] data) { };
perform;
}
I see your point. The question is how we would like phobos docs to be:
1, consistent with all parameters documented for each method and each
method explained and exemplified
2, minimal with examples and explanations where absolutely needed.
or somewhere in between. The current style if the curl wrapper is closer
to 1 than 2.
Anything official about this?
Official? I don't know. But looking over std.algorithm, Param: is only
used 3 times, for each of the find variants, in order to clarify what
haystack and needle are and in one case make the limitations/requirements
on haystack/needle explicit. Looking over the rest of phobos, I see a mix
of styles, depending on the author, and some things (like datetime) seem
to do both.
[snip]
CurlTimeCond is not calld Http.TimeCondition because it comes from a
etc.c.curl. I could alias it in Http ofcourse.
I think where it's defined is less of an issue than where the
documentation is.
[snip]
HttpResult.text should have huge warning flags with regard to use and/or
have its design re-thought. Is there some reason you're not at least
caching the result and throwing errors on invalid uses of content, etc.
Calling text multiple times should be okay. And calling content after
text should be either logically valid, or throw.
It is mostly based on storage concerns. Initially the data is stored as
ubyte[] and can be returned by content(). When calling text() to get a
string it will be decoded and returned. I now have a coule of options:
1, to store the string in the result for future calls to text(). I can
do that and keep the original data for future calls to content(). This
will double the amount of memory needed.
2, generated the string on each call to text(). This way no copy of the
data is kept in HttpResult.
3, generated and store the string and then throw away the original data.
Again no copy is kept in HttpResult.
I've chosen the 2. version since I guessed that once you have called
text() it is more likely that you'll call text again and not content().
Hmm... I take it that content and text are going to be large enough, and
result long lived enough to warrant not having the array cached inside
result. In that case you might want to rename text to toString or decode.
'text' feels like it should be cheaply reusable, while 'toString'/'decode'
doesn't. (this is a verb/noun thing) Also, 'toString' tends to compose
better with the rest of D while 'decode' is similar to the std.utf
functions. Also, if memory re-use is a concern, a 'toStringInPlace' or
'decodeInPlace' with the old behavior might be appropriate, although I
might recommend setting content to the utf-8 string and updating the
encodingSchemeName as appropriate.
Re: Resolution of core.time.Duration...
On 17.05.2011 16:45, Steven Schveighoffer wrote: The structures have used nanoseconds for over 10 years (I think gettimeofday used it back in the 90s!), so the reason for using it was likely for future compatibility (clearly nanosecond timing wasn't possible back then). It looks like the future is now, so it's good to have that resolution. As to gettimeofday() - it is using timeval, which has 1µs resolution - still quite good for most applications. As for measuring time, yes, it would be good to use a higher precision timer. And in fact, std.datetime.StopWatch does just that. Just in case - StopWatch is used in benchmarking functions while measuring wall-clock time, and this may produce incorrect results on busy systems when benchmarking CPU-intensive code. /Alexander
Re: Generators in D
On 5/17/11 9:37 AM, Piotr Szturmaj wrote:
Piotr Szturmaj wrote:
Hi,
I've written some proof of concept code of generator pattern, example:
void genSquares(out int result, int from, int to)
{
foreach (x; from .. to + 1)
{
yield!result(x * x);
}
}
void main(string[] argv)
{
foreach (sqr; generator(genSquares, 10, 20))
writeln(sqr);
}
posted on github:
https://github.com/pszturmaj/dgenerators
Thanks to Sean Kelly who wrote Fiber code.
Piotr
Well, I've just found this:
http://www.mail-archive.com/digitalmars-d@puremagic.com/msg30204.html.
It seems someone did it before ;)
That's why it's important to follow through with a formal submission to
Phobos.
In regards to serializable fibers, I've found some interesting potential
usage - session handling in HTTP server. Fiber code could represent
session and could be saved to disk in the middle of execution, just like
sessions are saved in PHP. This is used to offload memory when handling
great number of sessions.
If serialization will be clever enough, these fibers could even be
shared across different servers! This is really a requirement in load
balanced environments.
Sounds very interesting!
Andrei
Re: Curl wrapper
On Tue, 17 May 2011 17:45:00 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/17/11 12:23 AM, Vladimir Panteleev wrote: On Tue, 17 May 2011 07:43:06 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: 3. Data on wire is ubyte[], not void[]. I was the one who suggested using void[]. If data on wire is ubyte[], why is data on disk void[] (according to std.file.read/write)? Bug in std.file. I remember suggesting a long time ago that all types that do not have pointers be implicitly convertible to ubyte[]. That way, write functions can safely use ubyte[] without casts (and even provide a bit of safety against accidentally writing pointers to disk or network). Are you still against the idea? -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: Curl wrapper
On 5/17/11 12:39 PM, Vladimir Panteleev wrote: On Tue, 17 May 2011 17:45:00 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/17/11 12:23 AM, Vladimir Panteleev wrote: On Tue, 17 May 2011 07:43:06 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: 3. Data on wire is ubyte[], not void[]. I was the one who suggested using void[]. If data on wire is ubyte[], why is data on disk void[] (according to std.file.read/write)? Bug in std.file. I remember suggesting a long time ago that all types that do not have pointers be implicitly convertible to ubyte[]. That way, write functions can safely use ubyte[] without casts (and even provide a bit of safety against accidentally writing pointers to disk or network). Are you still against the idea? Yes. The problem is that converting a type implicitly to ubyte[] allows anyone to write raw bytes into an object, making a hash of any type system guarantees without a cast in sight. That is, granted, memory-safe (because there are no indirections), but is not recommendable. Andrei
Re: Curl wrapper
On Tue, 17 May 2011 20:43:11 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/17/11 12:39 PM, Vladimir Panteleev wrote: On Tue, 17 May 2011 17:45:00 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/17/11 12:23 AM, Vladimir Panteleev wrote: On Tue, 17 May 2011 07:43:06 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: 3. Data on wire is ubyte[], not void[]. I was the one who suggested using void[]. If data on wire is ubyte[], why is data on disk void[] (according to std.file.read/write)? Bug in std.file. I remember suggesting a long time ago that all types that do not have pointers be implicitly convertible to ubyte[]. That way, write functions can safely use ubyte[] without casts (and even provide a bit of safety against accidentally writing pointers to disk or network). Are you still against the idea? Yes. The problem is that converting a type implicitly to ubyte[] allows anyone to write raw bytes into an object, making a hash of any type system guarantees without a cast in sight. That is, granted, memory-safe (because there are no indirections), but is not recommendable. Ah, of course. const(ubyte)[] ? :) -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: Resolution of core.time.Duration...
On Tue, 17 May 2011 09:42:27 -0400, Daniel Gibson metalcae...@gmail.com wrote: Am 17.05.2011 15:25, schrieb Steven Schveighoffer: I agree that accepting a long as an alternative to Duration, it makes sense to use a more normal tick resolution. The chances of someone wanting to have their process sleep for more than 300 years (e.g. for nanosecond resolution) is pretty small. This might be a worthwhile change. I'm not sure how much code this might affect, though. It would be plenty disturbing if your code started sleeping for 100ms instead of the 10s you thought you requested. What might be a good path is to disable those functions that accept a long for a few releases, then re-instate them with a new meaning. Or just add nanoSleep() or something like that. Probably a good idea, and deprecate Thread.sleep(long). It's more self-documenting. I very much support the idea of deprecating all of the functions in druntime and phobos which take naked time values. And then if a function that sleeps at nanosecond resolution is considered to be of real value (which I question), then we can add that as something like nanoSleep. I believe that at present, most - if not all - of the function in druntime in Phobos which take a naked time value (aside from std.date) also take a Duration now, but I don't think that any of the old versions which take naked values are scheduled for deprecation. If there is a valid reason for wanting to keep any of them around, then we should consider it, but I'm definitely inclined to get rid of them in favor of the greater safety and maintainability of functions which take a Duration. - Jonathan M Davis
Re: Resolution of core.time.Duration...
On 17.05.2011 16:45, Steven Schveighoffer wrote: The structures have used nanoseconds for over 10 years (I think gettimeofday used it back in the 90s!), so the reason for using it was likely for future compatibility (clearly nanosecond timing wasn't possible back then). It looks like the future is now, so it's good to have that resolution. As to gettimeofday() - it is using timeval, which has 1µs resolution - still quite good for most applications. As for measuring time, yes, it would be good to use a higher precision timer. And in fact, std.datetime.StopWatch does just that. Just in case - StopWatch is used in benchmarking functions while measuring wall-clock time, and this may produce incorrect results on busy systems when benchmarking CPU-intensive code. StopWatch uses a mononotonic clock. - Jonathan M Davis
Re: Curl wrapper
On 5/17/11 1:00 PM, Vladimir Panteleev wrote: On Tue, 17 May 2011 20:43:11 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/17/11 12:39 PM, Vladimir Panteleev wrote: On Tue, 17 May 2011 17:45:00 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/17/11 12:23 AM, Vladimir Panteleev wrote: On Tue, 17 May 2011 07:43:06 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: 3. Data on wire is ubyte[], not void[]. I was the one who suggested using void[]. If data on wire is ubyte[], why is data on disk void[] (according to std.file.read/write)? Bug in std.file. I remember suggesting a long time ago that all types that do not have pointers be implicitly convertible to ubyte[]. That way, write functions can safely use ubyte[] without casts (and even provide a bit of safety against accidentally writing pointers to disk or network). Are you still against the idea? Yes. The problem is that converting a type implicitly to ubyte[] allows anyone to write raw bytes into an object, making a hash of any type system guarantees without a cast in sight. That is, granted, memory-safe (because there are no indirections), but is not recommendable. Ah, of course. const(ubyte)[] ? :) Yah, const(ubyte)[] is much more reasonable. We may as well add a function representation() that returns it for any object. Andrei
Re: Curl wrapper
On 5/17/11, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Yah, const(ubyte)[] is much more reasonable. We may as well add a function representation() that returns it for any object. Would this mean that e.g. you could save the state of an object to disk via repr(), and then retrieve the state at a later point?
Re: Curl wrapper
On May 17, 11 22:45, Andrei Alexandrescu wrote: On 5/17/11 12:23 AM, Vladimir Panteleev wrote: On Tue, 17 May 2011 07:43:06 +0300, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: 3. Data on wire is ubyte[], not void[]. I was the one who suggested using void[]. If data on wire is ubyte[], why is data on disk void[] (according to std.file.read/write)? Bug in std.file. Andrei I suppose that applies to std.mmfile too? MmFile.opSlice() currently returns a void[].
Re: Curl wrapper
On 5/17/11 1:35 PM, Andrej Mitrovic wrote: On 5/17/11, Andrei Alexandrescuseewebsiteforem...@erdani.org wrote: Yah, const(ubyte)[] is much more reasonable. We may as well add a function representation() that returns it for any object. Would this mean that e.g. you could save the state of an object to disk via repr(), and then retrieve the state at a later point? If it embeds no pointers, sure. Otherwise, a more involved serialization mechanism must be used. Andrei
Re: Generators in D
If serialization will be clever enough, these fibers could even be shared
across different servers! This is really a requirement in load balanced
environments.
A very old research paper (I think it was for the amoeba project)
wrote a long time ago (I don't remember the wording) that is usually a
bad idea to migrate already running tasks (process, thread, fiber,
etc). Leading to the observation that it is probably best to do
offloading/load-balancing before the work is started. That is how most
modern scale-able architectures work. This is just an observation
since you haven't exactly said how to use Fiber to save Http sessions.
On Tue, May 17, 2011 at 11:37 AM, Piotr Szturmaj bncr...@jadamspam.pl wrote:
Piotr Szturmaj wrote:
Hi,
I've written some proof of concept code of generator pattern, example:
void genSquares(out int result, int from, int to)
{
foreach (x; from .. to + 1)
{
yield!result(x * x);
}
}
void main(string[] argv)
{
foreach (sqr; generator(genSquares, 10, 20))
writeln(sqr);
}
posted on github:
https://github.com/pszturmaj/dgenerators
Thanks to Sean Kelly who wrote Fiber code.
Piotr
Well, I've just found this:
http://www.mail-archive.com/digitalmars-d@puremagic.com/msg30204.html. It
seems someone did it before ;)
In regards to serializable fibers, I've found some interesting potential
usage - session handling in HTTP server. Fiber code could represent session
and could be saved to disk in the middle of execution, just like sessions
are saved in PHP. This is used to offload memory when handling great number
of sessions.
If serialization will be clever enough, these fibers could even be shared
across different servers! This is really a requirement in load balanced
environments.
Piotr
Re: Generators in D
On May 11, 2011, at 3:09 PM, Piotr Szturmaj wrote: %u wrote: On 05/10/2011 07:06 PM, Piotr Szturmaj wrote: I forgot to ask. Any comments or suggestions? This is **beast**. Just one thing: Would it work correctly if I was using fibers in my own code? I don't know. If fibers may call other fibers then it should work. They can. The only issue to be aware of is that non-shared static data is thread-local, not fiber-local, so some code may exhibit unexpected behavior if not written with fibers in mind.
Re: Generators in D
On May 11, 2011, at 3:06 PM, Piotr Szturmaj wrote: Andrei Alexandrescu wrote: * can you implement save() to make this a forward range (e.g. by creating a new fiber that has its own state)? It is not that simple I guess. Fiber class doesn't give any opportunity to save current stack and I think it might be wrong to do this, because of GC issues. But I don't certainly know. I think that author of the Fiber class could answer this better than me. If Fiber will expose some additional API, then I would certainly implement save(). Cloning a Fiber would essentially produce a shallow copy--if the Fiber's stack contains references to data elsewhere on the same stack, the clone's stack would reference the original's. So while this could be done, I don't think it's advisable.
Re: Generators in D
On May 17, 2011, at 7:37 AM, Piotr Szturmaj wrote: In regards to serializable fibers, I've found some interesting potential usage - session handling in HTTP server. Fiber code could represent session and could be saved to disk in the middle of execution, just like sessions are saved in PHP. This is used to offload memory when handling great number of sessions. If serialization will be clever enough, these fibers could even be shared across different servers! This is really a requirement in load balanced environments. The trick is serializing the map of dynamic objects referenced by the Fiber's stack. And more generally, dealing with any pointers on the stack at all. It's the same problem as pointers in a memory-mapped file. That said, you really shouldn't have to serialize Fibers just to offload memory. That's what the virtual memory manager is for.
Re: Resolution of core.time.Duration...
On May 17, 2011, at 11:06 AM, Jonathan M Davis wrote: I very much support the idea of deprecating all of the functions in druntime and phobos which take naked time values. And then if a function that sleeps at nanosecond resolution is considered to be of real value (which I question), then we can add that as something like nanoSleep. That was the plan. I just wanted a few releases with the new Duration routines before deprecating the old ones. Now sounds like a good time.
Re: Resolution of core.time.Duration...
On 2011-05-17 12:34, Sean Kelly wrote: On May 17, 2011, at 11:06 AM, Jonathan M Davis wrote: I very much support the idea of deprecating all of the functions in druntime and phobos which take naked time values. And then if a function that sleeps at nanosecond resolution is considered to be of real value (which I question), then we can add that as something like nanoSleep. That was the plan. I just wanted a few releases with the new Duration routines before deprecating the old ones. Now sounds like a good time. As I understand it, deprecation is supposed to be a 3 stage process. 1. Mark the item to be deprecated as scheduled to be deprecated in the documentation, and if possible, give it a pragma which says it as well (though that generally involves turning the function in question into a template function if it isn't already so that the pragma will only kick in if the function is actually used). 2. Mark it with @deprecated so that -d is required to use it. 3. Fully remove it from the code. None of those functions have actually had #1 happen to them yet, so for the most part, people have likely kept on using them as they have been. So, they should probably be marked as scheduled for deprecation for at least a release before they're actually deprecated. Unfortunately, we haven't actually decided on how long each phase of deprecation is supposed to be. It's been brought up a time or two, but no decision has ever been made. I keep intending to bring it up again, since the last couple of releases have seen several items enter phase 1, but I keep forgetting. - Jonathan M Davis
Re: Builtin regex (Was: How to complex switch?)
On 13/05/2011 06:10, KennyTM~ wrote: On May 13, 11 12:14, Ary Manzana wrote: snip I didn't use regex a lot before I started using Ruby. The thing is, in Ruby it's so easy to use regex that I just started using them a lot more than before. Of course, ruby has built-in operators for matching regexs, so maybe that should also be added to the language (it's the =~ operator, but in D it should be a different one.) IIRC it was once there, but very soon removed in the 0.x era (can't find that changelog). There was a builtin regexp feature added in 0.147 - I think it was a ~~ or =~ operator with a string either side. I'm not sure whether it broke too much existing code or was just thought not right as a language builtin, but it was dropped again in 0.148. You can't distinguish between division and regex literal in the parser with this syntax. And that's half the reason such a syntax has never been implemented in D. See: - http://www.digitalmars.com/d/2.0/faq.html#regexp_literals - http://www.digitalmars.com/d/2.0/regular-expression.html Which ignores the idea of using a different notation for regexp literals. Though it's probably not worth much debate when the idea of a built-in regexp type has been rejected as language bloat. I think D is going the right way on the whole by leaving regexps to a library. Though this does limit such possibilities as optimised regexp switches. Stewart.
Re: Implementing std.log
On 5/17/11 4:02 AM, Jacob Carlborg wrote: On 2011-05-16 02:05, Andrei Alexandrescu wrote: Thanks for your work. I think there's an important distinction to be made. There are two APIs being discussed. One is the client interface and the other is the extensibility interface. Jose looked into both: he provided a client interface that has formatting, levels, enabling, and such, and an extensibility interface that essentially is a simple output stream. My library explores the client interface and leaves the extensibility interface as an obvious piece of work that needs little agreement and minimal design effort. Finally, your library keeps the client interface to a minimum and focuses almost exclusively on the extensibility interface. In doing so, it makes few choices that I disagree with. Allow me to share some specific feedback. Note that my suggestion was just a simple and incomplete suggestion on how the API could look like. I only provided info, warning and error methods as examples, I'm not saying the API should only have these three levels. [snip] I thought about this some more and I understand I sounded unfair. There is a lot of merit and there are a lot of good ideas in your code (and of course Jose's), which I didn't mention for the simple but cold reason that negative feedback is more informative. But neglecting the merits is a mistake as well. I'll incorporate some of the ideas you suggested in the next pass through std.log. Thanks, Andrei
Re: Curl wrapper
Please see comments below.
Den 17-05-2011 16:42, Andrei Alexandrescu skrev:
Thanks for the response! A few more answers and comments within
(everything deleted counts as sounds great).
On 5/17/11 3:50 AM, Jonas Drewsen wrote:
14. Isn't the max redirect configurable via a parameter?
Yes. It is called Http.followLocation from libcurls followLocation
option. I will rename it to maxRedirections for clearity.
You mention this about many higher-level functions: Maximum
redirections are set to 10. Then I'm thinking, if it's an important
parameter, make it a defaulted parameter for the function.
I guess I should just remove that comment because other defaults has
been selected as well e.g. timeouts.
16. Documentation should point to descriptions of the HTTP methods
wrapped (e.g. post, del etc).
Do you mean point to the relevant RFC?
Yah, or a good tutorial. (I didn't know DEL existed!)
Well DEL is actually called DELETE in the HTTP RFC. But delete is a
reserved word i D so I used del() instead. delete() wouldn't work in
following code in think:
with(auto http = ...) {
delete(...);
}
22. byLine/byChunk should not expose a string or generally data that can
be shared safely by the client. That's because it would need to create a
new string with each iteration, which is very inefficient. Instead, they
should expose char[]/ubyte[] just like File.byLine does, and reuse the
buffer with each call. Essentially byLine/byChunk should be
near-zero-overhead means to transfer and inspect arbitrarily large
streams.
byLine/byChunk is currenly only available for the async methods which is
implemented using message passing. This means that for passing the
message a copy has to be made because a message is by-value or immutable
data. Is there another way for me to pass the message without doing a
copy... some kind of move semantic?
A library element is planned but not on the short list yet. You can work
around that by internally doing a cast. As long as you observe the
commitment that buffers are not accessed simultaneously by more than one
thread you're well within defined behavior.
ok nice to know.
One more suggestion - you may want to also provide classic boring
synchronous read/write methods that take a user-provided buffer. I'm
sure people could use such e.g. when they want to stream data inside
their own threads, or even in the main thread if they don't mind blocking.
This would mean hooking into libcurl and selecting on its sockets.
Totally doable. But I would rather stop here feature wise and wrap this
thing up. I would like to focus my efforts on a candidate for
std.net.event/reactor/proactor module.
Anyone who wants to implement this read/write in the curl wrapper are
ofcourse very welcome.
24. The shutdown mechanism should be handled properly. Shutting down
libcurl would have all pending transfers instantly throw a special
exception. Without a shutdown API, applications won't be able to
implement e.g. a Quit button.
If you're only using this API this should be handled. But I see that it
is not documented.
Yah, in the future we need to put an onShutdown protocol inside core,
similar to atexit() in C. Each library that may block would plant a
hook. That way an application will be able to exit quickly and
gracefully even if it has blocked threads.
That would be neat.
/Jonas
Re: Curl wrapper
Den 17-05-2011 16:57, Robert Jacques skrev:
On Tue, 17 May 2011 07:24:16 -0400, Jonas Drewsen jdrew...@nospam.com
wrote:
On 17/05/11 08.03, Robert Jacques wrote:
On Mon, 16 May 2011 17:07:57 -0400, jdrewsen jdrew...@nospam.com
wrote:
Hi,
I've been working on a wrapper for the etc.c.curl module. It is now
pretty stable and I would very much like some feedback on the API.
http://freeze.steamwinter.com/D/web/phobos/etc_curl.html
BTW I use template mixins which doesn't seem to get included in the
generated docs. Is there any way I can make this work?
/Jonas
[snip]
I don't know what ditto does. I'll try to look at the ddoc documentation.
If a documentation comment for a declaration consists only of the
identifier ditto then the documentation comment for the previous
declaration at the same declaration scope is applied to this declaration
as well.
[snip]
http.onReceiveHeader has issues with it's 'See' link. It's example is
too wide. You don't need really need the parameter documentation,
particularly if you improve the example, i.e.:
string[string] headerInformation;
with(auto http = new Http(http://www.google.com;)) {
onReceiveHeader = (string key, string value) {
if ( value.length = 10 )
headerInformation[key.idup] = value.idup;
};
onReceive = (ubyte[] data) { };
perform;
}
I see your point. The question is how we would like phobos docs to be:
1, consistent with all parameters documented for each method and each
method explained and exemplified
2, minimal with examples and explanations where absolutely needed.
or somewhere in between. The current style if the curl wrapper is
closer to 1 than 2.
Anything official about this?
Official? I don't know. But looking over std.algorithm, Param: is only
used 3 times, for each of the find variants, in order to clarify what
haystack and needle are and in one case make the
limitations/requirements on haystack/needle explicit. Looking over the
rest of phobos, I see a mix of styles, depending on the author, and some
things (like datetime) seem to do both.
[snip]
CurlTimeCond is not calld Http.TimeCondition because it comes from a
etc.c.curl. I could alias it in Http ofcourse.
I think where it's defined is less of an issue than where the
documentation is.
[snip]
HttpResult.text should have huge warning flags with regard to use and/or
have its design re-thought. Is there some reason you're not at least
caching the result and throwing errors on invalid uses of content, etc.
Calling text multiple times should be okay. And calling content after
text should be either logically valid, or throw.
It is mostly based on storage concerns. Initially the data is stored
as ubyte[] and can be returned by content(). When calling text() to
get a string it will be decoded and returned. I now have a coule of
options:
1, to store the string in the result for future calls to text(). I can
do that and keep the original data for future calls to content(). This
will double the amount of memory needed.
2, generated the string on each call to text(). This way no copy of
the data is kept in HttpResult.
3, generated and store the string and then throw away the original
data. Again no copy is kept in HttpResult.
I've chosen the 2. version since I guessed that once you have called
text() it is more likely that you'll call text again and not content().
Hmm... I take it that content and text are going to be large enough, and
result long lived enough to warrant not having the array cached inside
result. In that case you might want to rename text to toString or
decode. 'text' feels like it should be cheaply reusable, while
'toString'/'decode' doesn't. (this is a verb/noun thing) Also,
'toString' tends to compose better with the rest of D while 'decode' is
similar to the std.utf functions. Also, if memory re-use is a concern, a
'toStringInPlace' or 'decodeInPlace' with the old behavior might be
appropriate, although I might recommend setting content to the utf-8
string and updating the encodingSchemeName as appropriate.
I think I'll go for the toString() version.
/Jonas
Re: Builtin regex (Was: How to complex switch?)
Stewart Gordon: I think D is going the right way on the whole by leaving regexps to a library. Though this does limit such possibilities as optimised regexp switches. Leaving regexps to a library is an acceptable choice for D, or maybe even the best choice. In D even associative arrays are not so important as builtins, you just need literals for associative arrays. On the other hand my experience shows me some other things are better as builtins (or partial built-ins), as tuples. I'd like unpacking syntax sugar for tuples, to be used inside foreach too, and as recently emerged in D.learn, some better cast for tuples (to allow conversion of slightly differently typed tuples, and structural typing). Tuples are a general language construct, useful in many situations for many different purposes, they are _much_ more commonly useful than regex. I use some regex most Python scripts that processes lot of text, but I use tuples every 4-5 lines of code or less :-) And indeed, in Python too regexes are not built-in, while tuples are. Bye, bearophile
Re: How to complex switch?
On 12/05/2011 13:21, Matthew Ong wrote: @dennis luehring snip I do not think that this is syntactic sugar. Compare this 2 block of code. Which is easier to read? snip Features to improve the readability of code are the very definition of syntactic sugar. Indeed, Prolog programmers rely on syntactic sugar a lot. Could you be thinking of syntactic saccharin? Stewart.
Re: How to complex switch?
Am 17.05.2011 22:44, schrieb Stewart Gordon: On 12/05/2011 13:21, Matthew Ong wrote: @dennis luehring snip I do not think that this is syntactic sugar. Compare this 2 block of code. Which is easier to read? snip Features to improve the readability of code are the very definition of syntactic sugar. Indeed, Prolog programmers rely on syntactic sugar a lot. Could you be thinking of syntactic saccharin? Sweet syntax with a disgusting bitter aftertaste? ;) Stewart.
Re: Builtin regex (Was: How to complex switch?)
Stewart Gordon: I think D is going the right way on the whole by leaving regexps to a library. Though this does limit such possibilities as optimised regexp switches. Leaving regexps to a library is an acceptable choice for D, or maybe even the best choice. In D even associative arrays are not so important as builtins, you just need literals for associative arrays. On the other hand my experience shows me some other things are better as builtins (or partial built-ins), as tuples. I'd like unpacking syntax sugar for tuples, to be used inside foreach too, and as recently emerged in D.learn, some better cast for tuples (to allow conversion of slightly differently typed tuples, and structural typing). Tuples are a general language construct, useful in many situations for many different purposes, they are _much_ more commonly useful than regex. I use some regex most Python scripts that processes lot of text, but I use tuples every 4-5 lines of code or less :-) And indeed, in Python too regexes are not built-in, while tuples are. Bye, bearophile Tuple literals would indeed be very nice. This _would_ mean that tuples get entirely built-in (having syntactic sugar for phobos functionality in the language seems like a very bad design to me, it would at least have to be copied/moved to druntime). However, tuple literals break the comma operator... But who would write something like ? auto t=(1,2,3); anyway in serious code? (where , means comma operator). Therefore Tuple literals could be introduced by a minor change in the grammar, where uses of the comma operator like return a=2,b=3; would still work normally. What would be the type of such a tuple literal? (T,T,T,...)? I don't expect something like this to be implemented soon though. Timon
Re: Generators in D
Sean Kelly wrote: On May 11, 2011, at 3:06 PM, Piotr Szturmaj wrote: Andrei Alexandrescu wrote: * can you implement save() to make this a forward range (e.g. by creating a new fiber that has its own state)? It is not that simple I guess. Fiber class doesn't give any opportunity to save current stack and I think it might be wrong to do this, because of GC issues. But I don't certainly know. I think that author of the Fiber class could answer this better than me. If Fiber will expose some additional API, then I would certainly implement save(). Cloning a Fiber would essentially produce a shallow copy--if the Fiber's stack contains references to data elsewhere on the same stack, the clone's stack would reference the original's. So while this could be done, I don't think it's advisable. But couldn't Fiber's stack be scanned for references to itself and readjusted?
Re: Generators in D
Sean Kelly wrote: On May 17, 2011, at 7:37 AM, Piotr Szturmaj wrote: In regards to serializable fibers, I've found some interesting potential usage - session handling in HTTP server. Fiber code could represent session and could be saved to disk in the middle of execution, just like sessions are saved in PHP. This is used to offload memory when handling great number of sessions. If serialization will be clever enough, these fibers could even be shared across different servers! This is really a requirement in load balanced environments. The trick is serializing the map of dynamic objects referenced by the Fiber's stack. And more generally, dealing with any pointers on the stack at all. It's the same problem as pointers in a memory-mapped file. That said, you really shouldn't have to serialize Fibers just to offload memory. That's what the virtual memory manager is for. While I fully agree on your opinion about virtual memory manager, I really meant something different. Offloading memory is not the sole reason. I didn't state that before, but I thought of fibers which do not reference any external data. They should be self-contained in terms of referenced memory. All external data should be passed as arguments to fiber function. Fiber should make a copy of all passed data. Other ideas involve persistent objects that should always be there. Upon deserialization Fiber should be reconnected to these object -- it will just get fresh references. Offloading memory is useful in case of sessions that may be unused for a long time. This include web sessions but not only them. Great examples are online games Second Life and EVE Online. SecondLife required that code be suspended at any point in time and that its entire state be serializable into a format suitable for storage into a database. Serialized state could then be restored at a different point in time or on a different computer (for example while moving from node to node). EVE Online uses stackless python in similar manner. So, I find it very interesting, especially if it would be possible to use with D :-) Here are some interesting links I found, about serializing fibers/coroutines/continuations/etc.: http://stackoverflow.com/questions/734638/language-that-supports-serializing-coroutines http://mikehadlow.blogspot.com/2011/04/serializing-continuations.html http://tirania.org/blog/archive/2009/Apr-09.html Piotr
Re: Resolution of core.time.Duration...
On 17.05.2011 20:06, Jonathan M Davis wrote: StopWatch uses a mononotonic clock. Monotonic clock is not CPU-usage-bound - it is in sync with wall-time, so problem on busy systems remains. /Alexander
Re: Builtin regex (Was: How to complex switch?)
Timon Gehr: Tuple literals would indeed be very nice. I have asked for tuple unpacking syntax (and other things like some support from the type system). Tuple literals are less needed. (having syntactic sugar for phobos functionality in the language seems like a very bad design to me, I don't see it as a bad design. Why do you see it as a bad thing? However, tuple literals break the comma operator... If something like a banana bracket syntax (|...|) is required (or some other kind of thing), then I think it doesn't clash with C usages of comma operator. What would be the type of such a tuple literal? (T,T,T,...)? I presume Tuple!(int, int, int) I don't expect something like this to be implemented soon though. Stuff like this requires months of thinking, and just few days to be implemented by Walter :-) Bye, bearophile
Re: Resolution of core.time.Duration...
On 17.05.2011 20:06, Jonathan M Davis wrote: StopWatch uses a mononotonic clock. Monotonic clock is not CPU-usage-bound - it is in sync with wall-time, so problem on busy systems remains. A monotonic clock is as good as you're going to get for accurate stopwatch functionality. The system cannot possibly do any better than that. Context switching can always get in the way. Increasing precision doesn't help that. And since StopWatch uses TickDuration, it has the highest precision that the the sytem has anyway (whatever its clock tick is) rather than hnsecs. I don't understand what your concern is. StopWatch is using the highest precision, most accurate clock that it can, and there's no way to stop issues with context switching. What more do you expect it to do? You're talking like there's something wrong with std.datetime's benchmarking functionality, and as far as I know, it has the best that is possible as far as timing accuracy goes. - Jonathan M Davis
Re: Generators in D
On May 17, 2011, at 2:37 PM, Piotr Szturmaj wrote: Sean Kelly wrote: On May 11, 2011, at 3:06 PM, Piotr Szturmaj wrote: Andrei Alexandrescu wrote: * can you implement save() to make this a forward range (e.g. by creating a new fiber that has its own state)? It is not that simple I guess. Fiber class doesn't give any opportunity to save current stack and I think it might be wrong to do this, because of GC issues. But I don't certainly know. I think that author of the Fiber class could answer this better than me. If Fiber will expose some additional API, then I would certainly implement save(). Cloning a Fiber would essentially produce a shallow copy--if the Fiber's stack contains references to data elsewhere on the same stack, the clone's stack would reference the original's. So while this could be done, I don't think it's advisable. But couldn't Fiber's stack be scanned for references to itself and readjusted? Without type information, there's no way to be sure that something is actually a reference, so corruption could occur.
Accessing Private Fields from Outside
Is there any (hacky) way of accessing a private field from outside a data type? (The equivalent of reflection in managed languages.) I'm trying to write a piece of marshaling code that needs access to a data type's fields, but can't access them because it's not allowed to. :(
keywords objconst and objimmutable vs. const(Object) ref
Greetings all. First time poster here. By way of introduction, my name is Christopher. I've been fascinated with D for a few years now. My main programming language has been Python, but I have used and studied many programming languages to some extent. I love to design things, like houses, pipe organs, and even computer languages. What I like about D is that it seems to strike the right balance of allowing fairly high level programming with static typing yet still permitting access on several progressively deeper levels to the nuts and bolts of the computer. I also love D's module system and its dynamic arrays, to name just a few things. Now down to business. I want to talk about const and immutable. There are 5,256 posts in this list containing the text const so I just this topic is discussed often. We all know that object references are really implemented as pointers to the object's data structures on the heap. When we say const Object a, we declare a constant variable pointing to a constant data structure. Sometimes we need to declare a modifiable variable pointing to a constant data structure. I propose the syntax objconst Object a to declare modifiable reference variable pointing to a const object data structure, and objimmutable Object A for modifiable reference variable pointing to immutable object data structure: objimmutable Object a = cast(immutable) new Object(); objconst Object b = new Object(); b = a; // b is modifiable // ILLEGAL: b references constant data structure b.someVariable = newValue; These could also be used in the parenthesized way: objconst(Object)a; objconst(Object)*[1]a_array; a = new Object(); a_array[0] = a; *a_array[0] = new Object(); // change a to a different object Now I understand that another syntax has been nominated to do this job which is the const(Object)ref syntax. I dislike that syntax because const(Object) means a constant variable referencing a constant data structure, and const(Object)ref LOOKS like it should be a reference to a variable of type const(Object) -- to me it suggests a variable referencing a constant variable referencing a constant data structure. const(Object)ref suggests TWO levels of pointers, but that is not the meaning being ascribed thereunto. I picked the keywords objconst and objimmutable to suggest that the object data structure (hence the obj prefix) is immutable or const. This was a compromise between brevity and descriptiveness. I suppose they could also go constobj and immutableobj. Another reason why the objconst/objimmutable syntax is better than const(Object)ref is that it requires fewer tokens to parse and to type -- this makes it easier to read: const(Object) ref a; // 5 tokens in the type objconst Object a; // only 2 tokens I submitted this idea to the bug tracking system as a feature request, but I also want to hear what people have to say about it on here. What do you think guys, Mr. Bright? And what do you like better--objconst and objimmutable or constobj and immutableobj? --Christopher
Re: Accessing Private Fields from Outside
On Tue, 17 May 2011 19:47:29 -0400, Mehrdad wfunct...@hotmail.com wrote:
Is there any (hacky) way of accessing a private field from outside a
data type? (The equivalent of reflection in managed languages.)
I'm trying to write a piece of marshaling code that needs access to a
data type's fields, but can't access them because it's not allowed to. :(
The short answer:
T.tupleof works for anything that you have the source to, but doesn't
support polymorphism out of the box.
The long answer (from my json library):
static Value from(T)(T value) {
static if(is(T == class) ) if( value is null ) return
Value(Null.init);
//...
static if(is(T == struct) || is(T == class)) {
Value[string] result;
foreach(i,v;value.tupleof) {
result[value.tupleof[i].stringof[value..length..$]] =
from(v);
}
return Value( result );
}
//..
}
P.S. Are you marshaling to/from an open format or something proprietary?
Re: keywords objconst and objimmutable vs. const(Object) ref
This is what Rebindable(T) is for: http://www.digitalmars.com/d/2.0/phobos/std_typecons.html
Re: Accessing Private Fields from Outside
Check out the __traits(allMembers, TYPE) system: http://dpldocs.info/traits
Re: keywords objconst and objimmutable vs. const(Object) ref
On 5/17/11 6:17 PM, %u wrote: This is what Rebindable(T) is for: http://www.digitalmars.com/d/2.0/phobos/std_typecons.html Thanks for the reference, but to me, rebindable feels like a hack. My opinion is that D deserves to have such an integral feature as mutable references to const object data structures baked into the language with a standard syntax, rather than added on using templates in an imported module. That a rebindable template can implement this functionality at all is a testament to the power and flexibility of D, but this really ought to be a core language feature.
Re: Accessing Private Fields from Outside
On 5/17/2011 5:13 PM, Robert Jacques wrote:
On Tue, 17 May 2011 19:47:29 -0400, Mehrdad wfunct...@hotmail.com
wrote:
Is there any (hacky) way of accessing a private field from outside a
data type? (The equivalent of reflection in managed languages.)
I'm trying to write a piece of marshaling code that needs access to a
data type's fields, but can't access them because it's not allowed
to. :(
The short answer:
T.tupleof works for anything that you have the source to, but doesn't
support polymorphism out of the box.
The long answer (from my json library):
static Value from(T)(T value) {
static if(is(T == class) ) if( value is null ) return
Value(Null.init);
//...
static if(is(T == struct) || is(T == class)) {
Value[string] result;
foreach(i,v;value.tupleof) {
result[value.tupleof[i].stringof[value..length..$]]
= from(v);
}
return Value( result );
}
//..
}
P.S. Are you marshaling to/from an open format or something proprietary?
T.tupleof has a problem though: It doesn't seem to let me actually
access the value; it just give me a tuple I can't do anything with. Your
example only gets the name of the field, but you never actually seem to
access it.
(P.S.: I'm trying to marshal from/to Windows data structures, so I guess
it's kinda both?)
Re: Accessing Private Fields from Outside
On 5/17/2011 5:18 PM, Adam D. Ruppe wrote: Check out the __traits(allMembers, TYPE) system: http://dpldocs.info/traits Thanks, but traits doesn't really let me read or write to the variable though. :(
Re: Accessing Private Fields from Outside
Thanks, but traits doesn't really let me read or write to the variable though. :( Use getMember there. foreach(member; __traits(allMembers, TYPE)) __traits(getMember, instance_of_type, member) = something; tupleof is probably better for this though, since it only includes actual data members; it excludes methods. You can write to tupleof by using an index.
Re: Accessing Private Fields from Outside
On Tue, 17 May 2011 20:44:24 -0400, Mehrdad wfunct...@hotmail.com wrote:
On 5/17/2011 5:13 PM, Robert Jacques wrote:
On Tue, 17 May 2011 19:47:29 -0400, Mehrdad wfunct...@hotmail.com
wrote:
Is there any (hacky) way of accessing a private field from outside a
data type? (The equivalent of reflection in managed languages.)
I'm trying to write a piece of marshaling code that needs access to a
data type's fields, but can't access them because it's not allowed to.
:(
The short answer:
T.tupleof works for anything that you have the source to, but doesn't
support polymorphism out of the box.
The long answer (from my json library):
static Value from(T)(T value) {
static if(is(T == class) ) if( value is null ) return
Value(Null.init);
//...
static if(is(T == struct) || is(T == class)) {
Value[string] result;
foreach(i,v;value.tupleof) {
result[value.tupleof[i].stringof[value..length..$]]
= from(v);
}
return Value( result );
}
//..
}
P.S. Are you marshaling to/from an open format or something proprietary?
T.tupleof has a problem though: It doesn't seem to let me actually
access the value; it just give me a tuple I can't do anything with. Your
example only gets the name of the field, but you never actually seem to
access it.
(P.S.: I'm trying to marshal from/to Windows data structures, so I guess
it's kinda both?)
I actually do both (name and value) in the above example.
Name: value.tupleof[i].stringof[value..length..$]
Value: v
You can also use value.tupleof[i] if all you want is to set a field.
Re: Accessing Private Fields from Outside
On Tue, 17 May 2011 20:52:02 -0400, Robert Jacques sandf...@jhu.edu
wrote:
On Tue, 17 May 2011 20:44:24 -0400, Mehrdad wfunct...@hotmail.com
wrote:
On 5/17/2011 5:13 PM, Robert Jacques wrote:
On Tue, 17 May 2011 19:47:29 -0400, Mehrdad wfunct...@hotmail.com
wrote:
Is there any (hacky) way of accessing a private field from outside a
data type? (The equivalent of reflection in managed languages.)
I'm trying to write a piece of marshaling code that needs access to a
data type's fields, but can't access them because it's not allowed
to. :(
The short answer:
T.tupleof works for anything that you have the source to, but doesn't
support polymorphism out of the box.
The long answer (from my json library):
static Value from(T)(T value) {
static if(is(T == class) ) if( value is null ) return
Value(Null.init);
//...
static if(is(T == struct) || is(T == class)) {
Value[string] result;
foreach(i,v;value.tupleof) {
result[value.tupleof[i].stringof[value..length..$]]
= from(v);
}
return Value( result );
}
//..
}
P.S. Are you marshaling to/from an open format or something
proprietary?
T.tupleof has a problem though: It doesn't seem to let me actually
access the value; it just give me a tuple I can't do anything with.
Your example only gets the name of the field, but you never actually
seem to access it.
(P.S.: I'm trying to marshal from/to Windows data structures, so I
guess it's kinda both?)
I actually do both (name and value) in the above example.
Name: value.tupleof[i].stringof[value..length..$]
Value: v
You can also use value.tupleof[i] if all you want is to set a field.
P.S. In fact, since foreach(i,ref v;value.tupleof) won't compile, you have
to use value.tupleof[i] to assign to a field.
Re: Accessing Private Fields from Outside
On 5/17/2011 5:52 PM, Robert Jacques wrote: On Tue, 17 May 2011 20:44:24 -0400, Mehrdad wfunct...@hotmail.com wrote: On 5/17/2011 5:13 PM, Robert Jacques wrote: T.tupleof has a problem though: It doesn't seem to let me actually access the value; it just give me a tuple I can't do anything with. Your example only gets the name of the field, but you never actually seem to access it. (P.S.: I'm trying to marshal from/to Windows data structures, so I guess it's kinda both?) I actually do both (name and value) in the above example. Name: value.tupleof[i].stringof[value..length..$] Value: v You can also use value.tupleof[i] if all you want is to set a field. Hm, my bad. Seems like the newest version of DMD allows this, though I could swear I couldn't do that before (it would say the member is inaccessible). :-) (And yeah, I already noticed that thing about foreach(ref), thanks.) On 5/17/2011 5:56 PM, Adam D. Ruppe wrote: Thanks, but traits doesn't really let me read or write to the variable though. :( Use getMember there. foreach(member; __traits(allMembers, TYPE)) __traits(getMember, instance_of_type, member) = something; tupleof is probably better for this though, since it only includes actual data members; it excludes methods. You can write to tupleof by using an index. Nope, getMember doesn't work, it gives me an inaccessible field error.
Re: keywords objconst and objimmutable vs. const(Object) ref
Christopher the Magnificent Wrote: Greetings all. First time poster here. By way of introduction, my name is Christopher. I've been fascinated with D for a few years now. My main programming language has been Python, but I have used and studied many programming languages to some extent. I love to design things, like houses, pipe organs, and even computer languages. Hello, and welcome. Now I understand that another syntax has been nominated to do this job which is the const(Object)ref syntax. I dislike that syntax because const(Object) means a constant variable referencing a constant data structure, and const(Object)ref LOOKS like it should be a reference to a variable of type const(Object) -- to me it suggests a variable referencing a constant variable referencing a constant data structure. const(Object)ref suggests TWO levels of pointers, but that is not the meaning being ascribed thereunto. Syntax has definitely been a major problem for this feature. Walter's stance has been that he has tried many times to get the semantics and syntax to work and has given up. michelf (Sorry don't know his real name) has created a branch which implements the syntax you are against. As this is a change Walter is skeptical of it will take time to review. Now as for your distaste, I believe the intent is to make a valid declaration: Object ref foobar; Meaning that you are exposing the reference or address itself. However I think there might be more to it when considering generic code which would suggest these should also be valid forms: void function(T)(const(T) ref thing); const(int) ref foo; const(myStruct) ref bar; const(int*) ref bared; But that would suggest D has a reference type, and it doesn't. So in any case, this isn't simple and I'm not really sure how well const(Object) ref stands up to this kind of question.
Re: keywords objconst and objimmutable vs. const(Object) ref
On 5/17/11 7:26 PM, Jesse Phillips wrote: Christopher the Magnificent Wrote: Greetings all. First time poster here. By way of introduction, my name is Christopher. I've been fascinated with D for a few years now. My main programming language has been Python, but I have used and studied many programming languages to some extent. I love to design things, like houses, pipe organs, and even computer languages. Hello, and welcome. Now I understand that another syntax has been nominated to do this job which is the const(Object)ref syntax. I dislike that syntax because const(Object) means a constant variable referencing a constant data structure, and const(Object)ref LOOKS like it should be a reference to a variable of type const(Object) -- to me it suggests a variable referencing a constant variable referencing a constant data structure. const(Object)ref suggests TWO levels of pointers, but that is not the meaning being ascribed thereunto. Syntax has definitely been a major problem for this feature. Walter's stance has been that he has tried many times to get the semantics and syntax to work and has given up. michelf (Sorry don't know his real name) has created a branch which implements the syntax you are against. As this is a change Walter is skeptical of it will take time to review. Now as for your distaste, I believe the intent is to make a valid declaration: Object ref foobar; Meaning that you are exposing the reference or address itself. However I think there might be more to it when considering generic code which would suggest these should also be valid forms: void function(T)(const(T) ref thing); const(int) ref foo; const(myStruct) ref bar; const(int*) ref bared; But that would suggest D has a reference type, and it doesn't. So in any case, this isn't simple and I'm not really sure how well const(Object) ref stands up to this kind of question. Thanks for the background, Jesse Phillips. Do you have any thoughts on my proposed objconst/objimmutable syntax, as to whether you think it is readable, intuitive, and just whether you like it or don't like it?
Re: keywords objconst and objimmutable vs. const(Object) ref
On 2011-05-17 20:00:55 -0400, Christopher the Magnificent ultimatemacfana...@gmail.com said: Now I understand that another syntax has been nominated to do this job which is the const(Object)ref syntax. I don't think there has been any official word on that. But many people have shown support for it. I dislike that syntax because const(Object) means a constant variable referencing a constant data structure, and const(Object)ref LOOKS like it should be a reference to a variable of type const(Object) -- to me it suggests a variable referencing a constant variable referencing a constant data structure. const(Object)ref suggests TWO levels of pointers, but that is not the meaning being ascribed thereunto. But ask yourself this: why does Object ref looks like a double reference to you? I'll guess an answer: because of previous knowledge of the language that says to you that Object is always implicitly a reference type. The basic idea behind the const(Object)ref proposal is to change the meaning of Object to become a shortcut for Object ref. In other word, the ref part is always there, and if you omit it, it's added implicitly for you. Try to look at thing from a this new perspective and it'll fit better. I picked the keywords objconst and objimmutable to suggest that the object data structure (hence the obj prefix) is immutable or const. This was a compromise between brevity and descriptiveness. I suppose they could also go constobj and immutableobj. Personally, I like better the feel of const(Object)ref than any of these keywords. For one thing, it doesn't require any new keyword. For another, it's perfectly in line with how you do it for pointers. Also, it'll work for 'shared' and 'inout' too (once 'inout' works properly). Working like pointers has other advantages too: it allow you to transpose your understanding of type deduction and type matching in templates and is-expressions freely between pointers and objects references. Making the syntax different adds a barrier to applying your experience of one to the other, this barrier simply does not exist with const(Object)ref because the syntax is similar enough to const(S)*. Another reason why the objconst/objimmutable syntax is better than const(Object)ref is that it requires fewer tokens to parse and to type -- this makes it easier to read: const(Object) ref a; // 5 tokens in the type objconst Object a; // only 2 tokens Easier to type, granted. Easier to read, I'm not so sure; mimicking the existing syntax for pointer makes things more coherent. Easier to parse, I'm not too sure about that either, but it's quite irrelevant anyway. What's difficult is implementing the corresponding semantics in the compiler. The main reason being that, unlike the common perception people have, it's only the compiler backend that deals with class types the same way it deals with pointers. Type attributes (const, immutable, shared, inout) are enforced in the frontend, and the frontend has no separate type for an object and an object reference, and thus no way to attach attributes to the reference separately from the main object type. As the entirety of the frontend is written with that assumption in mind, it's hard to overcome without rewriting a very big portion of the code. Now, I figured a way around that, one that Walter will hopefully accept once he find enough time to review my patch. If he thinks another syntax (such as yours) is more appropriate, it should a pretty trivial change. But my preference remains with const(Object)ref. Thanks for your input Christopher, and welcome to D. - - - For reference, here's my pull request for const(Object)ref: https://github.com/D-Programming-Language/dmd/pull/3. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: keywords objconst and objimmutable vs. const(Object) ref
So how do functions which take such a parameter look like?
void bar(ref Foo a, const(Foo) ref b) { }
or
void bar(ref Foo a, ref const(Foo) b) { }
Re: Article Review: Migrating from std.date to std.datetime
On 2011-05-14 21:16, Jonathan M Davis wrote: It recently came to my attention that an article on converting code from using std.date to using std.datetime would be of value, so I wrote one up. Since it's an article, and it's within the time period set by Walter for the article contest, I guess that it's in the article contest, but I wrote it because it needed writing rather than for the contest. It might as well be part of the contest though. In any case, I put it in my d-programming-language.org repository on github: https://github.com/jmdavis/d-programming-language.org/tree/article_datetim e Or if you just want to download it without building d-programming-language.org yourself, go here: http://is.gd/jFAmDy So, feel free to read it and give feeback on it. Hopefully it does its job fairly well. Writing it did give me some insight into a couple of tweaks that need to be made to std.datetime though. In particular, I really should come up with a better way to save time zones for those that care about restoring the _exact_ time zone that they had in the SysTime that they saved in a database or to disk or whatever. ISO is great for storing the exact time - including what the offset from UTC of the original time zone was - but it's not generally enough for restoring the exact time zone that you were dealing with in the first place. Most code won't care, but I really should find a way to do it. I knew about the problem, but it's definitely a pain to solve, so I left it alone. I should work on fixing that. In any case, the article is now available for you to read and review. A minor update with a few corrections: http://is.gd/GNELTZ Naturally, it's also up on my github account: https://github.com/jmdavis/d- programming-language.org/tree/article_datetime - Jonathan M Davis
Re: keywords objconst and objimmutable vs. const(Object) ref
Christopher the Magnificent Wrote: Thanks for the background, Jesse Phillips. Do you have any thoughts on my proposed objconst/objimmutable syntax, as to whether you think it is readable, intuitive, and just whether you like it or don't like it? I'm going to go withMichel's reply. The const(Object) ref is more consistent with pointers and the smashed keyword looks weird to me.
Re: How To Dynamic Web Rendering?
On 2011-05-16 15:43, Adam D. Ruppe wrote: Instead you move the view layer into the model or controller layer. How's that any different? Is that really what's happening? Any template has variables made available to it from the model. I'm just making them available at a higher level (pre-wrapped in semantic tags and grouped together). The actual layout and styling, sometimes even data formatting - the stuff I think of as the view - is still done entirely in the separate html and css files. I realize this isn't a perfect argument, but it's still worth it to me to have those other processing options available. I got the impression that you basically did all HTML with D methods, maybe I don't understand your work flow. -- /Jacob Carlborg
Re: Can we make auto return type inference a little more lax?
Andrej Mitrovic: I don't think the compiler can figure that one out unless tuples become a standard language feature, not a library one. But maybe I'm wrong. Turning tuples into a a _partial_ language feature will be a good thing [TM] for D. Tuples are useful in many situations, and they are easy to use. Bye, bearophile
Re: How To Dynamic Web Rendering?
Jacob Carlborg d...@me.com wrote in message news:iqt6kb$1nd1$1...@digitalmars.com... On 2011-05-16 15:43, Adam D. Ruppe wrote: Instead you move the view layer into the model or controller layer. How's that any different? Is that really what's happening? Any template has variables made available to it from the model. I'm just making them available at a higher level (pre-wrapped in semantic tags and grouped together). The actual layout and styling, sometimes even data formatting - the stuff I think of as the view - is still done entirely in the separate html and css files. I realize this isn't a perfect argument, but it's still worth it to me to have those other processing options available. I got the impression that you basically did all HTML with D methods, maybe I don't understand your work flow. AIUI: 1. It starts out as ordinary HTML with no special non-standard tags or anything. But it might use id and class attributes to mean certain things. 2. In D, that HTML is loaded and parsed into an HTML DOM. 3. In D, the data is filled into the HTML DOM. Sometimes this might involve a amall amount of adding HTML tags/attributes or other weird tweaking if necessary. 4. The new HTML DOM is written out as straight HTML. Basically just like DHTML, except on the server and using D.
Re: How To Dynamic Web Rendering?
Nick Sabalausky a@a.a wrote in message news:iqtarl$229l$1...@digitalmars.com... Jacob Carlborg d...@me.com wrote in message news:iqt6kb$1nd1$1...@digitalmars.com... On 2011-05-16 15:43, Adam D. Ruppe wrote: Instead you move the view layer into the model or controller layer. How's that any different? Is that really what's happening? Any template has variables made available to it from the model. I'm just making them available at a higher level (pre-wrapped in semantic tags and grouped together). The actual layout and styling, sometimes even data formatting - the stuff I think of as the view - is still done entirely in the separate html and css files. I realize this isn't a perfect argument, but it's still worth it to me to have those other processing options available. I got the impression that you basically did all HTML with D methods, maybe I don't understand your work flow. AIUI: 1. It starts out as ordinary HTML with no special non-standard tags or anything. But it might use id and class attributes to mean certain things. 2. In D, that HTML is loaded and parsed into an HTML DOM. 3. In D, the data is filled into the HTML DOM. Sometimes this might involve a amall amount of adding HTML tags/attributes or other weird tweaking if necessary. 4. The new HTML DOM is written out as straight HTML. Basically just like DHTML, except on the server and using D. And AIUI, it can also be thought of as being like an HTML templating engine, except instead of working with proprietary tags and text subsitution, it works with id/class attributes and an enhanced HTML DOM.
Re: How To Dynamic Web Rendering?
On 2011-05-17 10:11, Nick Sabalausky wrote: Jacob Carlborgd...@me.com wrote in message news:iqt6kb$1nd1$1...@digitalmars.com... On 2011-05-16 15:43, Adam D. Ruppe wrote: Instead you move the view layer into the model or controller layer. How's that any different? Is that really what's happening? Any template has variables made available to it from the model. I'm just making them available at a higher level (pre-wrapped in semantic tags and grouped together). The actual layout and styling, sometimes even data formatting - the stuff I think of as the view - is still done entirely in the separate html and css files. I realize this isn't a perfect argument, but it's still worth it to me to have those other processing options available. I got the impression that you basically did all HTML with D methods, maybe I don't understand your work flow. AIUI: 1. It starts out as ordinary HTML with no special non-standard tags or anything. But it might use id and class attributes to mean certain things. 2. In D, that HTML is loaded and parsed into an HTML DOM. 3. In D, the data is filled into the HTML DOM. Sometimes this might involve a amall amount of adding HTML tags/attributes or other weird tweaking if necessary. 4. The new HTML DOM is written out as straight HTML. Basically just like DHTML, except on the server and using D. Ok I see. -- /Jacob Carlborg
Re: How To Dynamic Web Rendering?
Jacob Carlborg wrote:
That would require a lot of otherwise (possibly) unnecessary
id/class attributes (if they aren't removed).
Not that much in the sites I've done.
Here's another real example, that new d web site discussed on the
newsgroup a while ago:
Dynamic page:
http://arsdnet.net/d-web-site/
The HTML template it manipulates with DOM:
http://arsdnet.net/d-web-site/index.html
There's only three dynamic elements on that page, so the template
is almost identical to the finished page.
There's three placeholders getting filled in:
div id=announcements /
div id=commits /
and form id=code-runner
And the relevant code:
form:
auto f = cast(Form) document.getElementById(code-runner);
f.setValue(code, `void main() { assert(0, Hello, world!); }`);
It fills the form, without caring about how the form is written.
This does name=code and sets value to that string given. It
doesn't care how the HTML is written - here, it's a textarea,
but this code remains the same if it's an input or select.
The view controls how the data is presented, the dom just tells
it what to present.
Announcements and commits are done with more dom calls, but still,
not much.
void addAnnouncements(Element holder) {
auto document = holder.parentDocument;
foreach(announcement; getAnnouncements()) {
auto h = holder.addChild(div);
auto date = std.date.parse(announcement.date);
h.addChild(span, formatDate(date));
h.appendText( - );
h.appendChild(new Link(announcement.url, announcement.subject));
}
}
Commits is a little longer since it has more data, but same idea.
Arguably, this crosses the line into presentation, especially
with that appendText call. (I guess it could do an #announcements
span:after in css, but that's not reliable) But, you can see
that the vast majority of html is far away from here.
The #announcements id used to fill it in is also used to style
the inside:
#announcements div { style each announcement }
#announcements div span { style the date }
#announcements div a { style the link }
(man, css nesting is such a good thing to have!)
Side note: obviously, you don't *have* to do web apps my way, even
with my libraries. cgi.d is template-agnostic. dom.d isn't going
to kill you if you ask it to add logic to templates or do string
replacement instead.
My way is just one possibility of many.
Re: How To Dynamic Web Rendering?
Adam D. Ruppe destructiona...@gmail.com wrote in message
news:iqua65$rm2$1...@digitalmars.com...
auto f = cast(Form) document.getElementById(code-runner);
f.setValue(code, `void main() { assert(0, Hello, world!); }`);
Minor suggestion:
There should be an overload of getElementById that's defined like this:
T getElementById(T)(string str) if(is(T:WhateverYourBaseNodeTypeIs))
{
auto node = cast(T) getElementById(str);
if(node)
return node;
else
throw new ElementNotFoundException(T.stringof, str);
}
class ElementNotFoundException : Exception
{
this(string type, string str)
{
super(Element of type '~type~' matching '~str~' not found.);
}
}
That way if it's missing, you get an accurate message instead of an awful
null pointer exception. Or maybe call it requireElementById.
Or, of course, if it's in a class and can't have templated members (I really
hate that limitation. Templated class members would be so much more useful
than being able to ship .obj's and .lib's without the original source),
then:
T getElementById(T)(WhateverYourBaseNodeTypeIs root, string str)
if(is(T:WhateverYourBaseNodeTypeIs))
{
...
}
And, of course, just use the original getElementById if you want to allow
the element to be optional.
Another separate thought:
What happens if you want to (or try to) put the same data in more than one
place? Maybe there should be a way to detect and error if there's more than
one of the same thing. Or maybe better yet, getElementById could return a
special collection that contains all matching elements. And it could expose
the same interface as a single element, but automatically applies it to all
matched elements. Of course, you couldn't use id= for that since that's
really not supposed to have duplicates. So you'd have to use class=
instead. Which opens the possibility, if you're not doing this already, for
the HTML importing (and maybe the DOM, too, or at least serializing the DOM
at the end) to detect and error on multiple instances of the same id=.
Re: How To Dynamic Web Rendering?
Nick Sabalausky wrote: There should be an overload of getElementById that's defined like this: Aye, I've been thinking about that for a while, but could never think of a name I like (yeah, petty reason). I've just trained myself to use assert to avoid the null pointers. Or maybe call it requireElementById. That's a pretty good idea. Or, of course, if it's in a class and can't have templated members It is a class, but the requireElementById (and it's counterpart, requireSelector*) could always be final. I can't think of a reason to override them anyway, especially since the underlying implementation in getElementById is still overridable anyway. (same reason why I'm ok with opDispatch - it just forwards to set/getAttribute) This is definitely good though, it has the best cost/benefit ratio of things on the todo list. I take your name! side note:: * After looking at haml yesterday, I'm tempted to do a nothrow requireSelector. Again, can't think of a name. But what it'd do is try to get the first element that matches the css syntax. If it's not there, it starts from the best path that does exist and creates the rest. So if you have a blank element html/html and you do: makeSelector(html); // that root node is returned since it exists makeSelector(div span.date); That doesn't exist, so it does a return document.root.addChild(div).addChild(span).className(date); Though, I don't think I have a use for it; it'd be harder to use than just writing the html yourself in the template file. :: end side note What happens if you want to (or try to) put the same data in more than one place? Use a class or a custom attribute instead of id, then loop through the collection: foreach(e; document.getElementsBySelector(div.announcements)) addAnnouncements(e); Sometimes I do that even when there's just one because the return value there is guaranteed to never include null. Which opens the possibility, if you're not doing this already, for the HTML importing (and maybe the DOM, too, or at least serializing the DOM at the end) to detect and error on multiple instances of the same id=. No, it doesn't check that right now. It treats id as just another attribute for the most part. (getElementById returns the first one it sees in the tree.) It always could, but I haven't felt it's worth the time yet.
Re: Can we make auto return type inference a little more lax?
On 17/05/2011 04:40, Andrej Mitrovic wrote:
auto foo()
{
if (1)
{
return [0, 0];
}
else
{
size_t one;
size_t two;
return [one, two];
}
}
void main(){ }
Error: mismatched function return type inference of
uint[] and int[]
Surely the compiler can figure out a common type for these two when literals
are involved.
I haven't tested, but make the first array [0u, 0u], that should work,
as a work around. Make a bug report for this (maybe a patch if you feel
up to it? :D).
--
Robert
http://octarineparrot.com/
Re: Can we make auto return type inference a little more lax?
Bug is in bugzilla: http://d.puremagic.com/issues/show_bug.cgi?id=6022 I'm not a compiler writer though, sorry. :)
Memory corruption rant.
I would love to post a bug with a minimum test case, so that I'd be able to identify if its my fault, the compiler, or libraries I'm using. But at this time I can't seem to reduce it to a reasonable (non-proprietary) portion of code. So for now I'll just rant. I combining much here. I have two DLL's being loaded. One is Lua, being accessed with LuaD, and the other is a DLL my company has produced. I've been able to get some nice interaction between the three. The company DLL will make function calls which are passed through D into Lua and the result back through D and all this great stuff. But I've been fighting with delegates and objects disappearing (I think LuaD might not be preserving anonymous delegates, which I can't reproduce in the small). I have some code that has this general flow: auto myObj = new Object(); // Create lua and company dll objects // call some functions for lua // call some functions for company dll // Other heavy processing with D only (takse 60 seconds) // make a call to company dll (takes 30 seconds) writeln(Im still ok) writeln(myObject is null); // Crash with access violation I can remove the last call to the company dll and everything is fine. And this has been working for some time until recently when I changed some stuff on the heavy D processing side (added a struct into the mix). I'm not even accessing myObject, I'm just asking if it has a value it shouldn't! In all other respects it has been a blast combining all this.
Re: Memory corruption rant.
Could it be a stack corruption?
Re: Memory corruption rant.
On Wed, 18 May 2011 02:18:22 +0300, Jesse Phillips jessekphillip...@gmail.com wrote: [snip memory corruption rant] I've nothing to contribute but by own anecdotal experience. So: are you doing *any* allocations or throwing exceptions in finalizers (destructors of objects on the heap)? If you are, don't do it. :P -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: Non-GC threads
On Sun, 15 May 2011 21:30:32 +0300, Piotr Szturmaj bncr...@jadamspam.pl wrote: What are the consequences of using non garbage collected threads in D? If you move pointers around while the GC is looking for them, the GC might miss them and free referenced memory. This will lead to memory corruption. The GC will also not be able to scan the stack and registers of threads it doesn't know about. I want to write a reliable communication protocol, but I don't want suspend this code by the GC. The GC might be much faster than you think it is. You should try benchmarking the GC with a typical memory load, the delay might be acceptable for your purposes. Is there any method to bind allocated memory to thread itself, so it will be freed after thread terminates, but not in the middle of execution? You can use malloc/free together with RAII. Or something hacky and platform-dependent like pthread_cleanup_push(). This is important because in the other case, GC could free memory referenced by non-GC thread. I can think of no perfect solution for your case. You'll either need to give up on using managed memory, or accept the periodical delays of the GC. -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: Memory corruption rant.
Trass3r Wrote: Could it be a stack corruption? That would make the most sense for the behavior I'm seeing, but I don't really know how to corrupt the stack without using ASM, and even then I'd probably fail.
Re: Memory corruption rant.
Vladimir Panteleev Wrote: On Wed, 18 May 2011 02:18:22 +0300, Jesse Phillips jessekphillip...@gmail.com wrote: [snip memory corruption rant] I've nothing to contribute but by own anecdotal experience. So: are you doing *any* allocations or throwing exceptions in finalizers (destructors of objects on the heap)? If you are, don't do it. :P -- Best regards, Vladimirmailto:vladi...@thecybershadow.net Well I do use allocations to get values out of dll calls, but I didn't want to negotiate ownership so I duplicated everything it gave me and promptly freeing it through the free function it provides, and I ignore ownership of the values I give it (so I wouldn't be surprised if I had memory leaks, but it's not like the DLL will tell me when it is done with the memory) As for the destructor thing, I'll have to check, I originally tried to use a reference counted struct and believe I may had thown something in a destructor there (think that one is safe?). But when I move to class I'm pretty sure I just gave up on cleaning up. I mean I'm done with it when I exit my program anyway. And thing thing is that it is consistent. In that a build that fails will always fail and a build the succeeded always continues to function. So not only should it not be destroying anything yet, it really seems to be how the code is generated. I actually had a print statement which printed some values of StopWatch, and I could consistently use that to get an access violation, and I just hoped I wouldn't have any more issues. What does D do when it runs out of stack space? (Not that it is likely I am)
[Issue 6024] New: Windows 2000 SP4 is not supported any more? And what is still supported?
http://d.puremagic.com/issues/show_bug.cgi?id=6024
Summary: Windows 2000 SP4 is not supported any more? And what
is still supported?
Product: D
Version: D2
Platform: Other
OS/Version: Windows
Status: NEW
Severity: blocker
Priority: P2
Component: druntime
AssignedTo: nob...@puremagic.com
ReportedBy: verylonglogin@gmail.com
--- Comment #0 from Denis verylonglogin@gmail.com 2011-05-17 00:39:19 PDT
---
Since dmd v2.053 RtlCaptureContext function (minimum supported client Windows
XP) is in file
runtime\src\core\sys\windows\stacktrace.d
So on Windows 2000 you have Entry point not found error when any D2 program
starts (e.g void main() { } program).
I'll understand, if Windows 2000 is not supported any more. I think it can be
supported, but if community doesn't want, so be it. But this undefined
behaviour is very bad. Please write supported operating systems somewhere in
plain view (e.g. in Requirements and Downloads section of
http://www.digitalmars.com/d/2.0/dmd-windows.html). And in write in changelog
... OS is not supported any more.
Since some previous version, you have no command line arguments on Windows 98,
but in std.file there are a lot of code aimed to support this OS. It's very
confusing. So, please, write what systems are supported. operating system,
such as Windows XP is sounds very obscure (and it is).
--
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
[Issue 6024] Windows 2000 SP4 is not supported any more? And what is still supported?
http://d.puremagic.com/issues/show_bug.cgi?id=6024 Vladimir Panteleev thecybersha...@gmail.com changed: What|Removed |Added CC||thecybersha...@gmail.com --- Comment #2 from Vladimir Panteleev thecybersha...@gmail.com 2011-05-17 01:23:33 PDT --- It looks like the RtlCaptureContext call can be replaced with some simple assembler. All it needs is the contents of the EIP, EBP and ESP registers. Some compilers offer intrinsics for special CPU registers, although I don't think DMD does. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 6000] Value range propagation does not work with modulus (%) with a constant
http://d.puremagic.com/issues/show_bug.cgi?id=6000
bearophile_h...@eml.cc changed:
What|Removed |Added
CC||bearophile_h...@eml.cc
--- Comment #1 from bearophile_h...@eml.cc 2011-05-17 02:51:59 PDT ---
A similar case:
void main() {
long n = 10520;
ubyte b = n % 10;
}
--
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
[Issue 6021] std.parallelism missing from dpl.org
http://d.puremagic.com/issues/show_bug.cgi?id=6021 David Simcha dsim...@yahoo.com changed: What|Removed |Added CC||dsim...@yahoo.com --- Comment #1 from David Simcha dsim...@yahoo.com 2011-05-17 05:36:47 PDT --- Same for std.net.isemail. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 6018] Multiple includes of std.parallelism causes DMD to segfault.
http://d.puremagic.com/issues/show_bug.cgi?id=6018 --- Comment #2 from Jonathan Crapuchettes jcrapuchet...@gmail.com 2011-05-17 09:18:52 PDT --- Created an attachment (id=978) DMD2 -v output I just tried to create a simple test case, but couldn't get it to break. I am including the output from DMD with -v. As you can see all of the files were compiled together. I did notice, after commenting out one file, that DMD always died just after outputting function this. This makes me wonder if it has some connection to auto varPool = taskPool.workerLocalStorage(new ThreadStore()); but that isn't based on actual evidence. I hope this helps a little. I would be happy to send you the files privately, but I can't post them as they belong to the company I work for. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 6024] Windows 2000 SP4 is not supported any more? And what is still supported?
http://d.puremagic.com/issues/show_bug.cgi?id=6024 Walter Bright bugzi...@digitalmars.com changed: What|Removed |Added CC||bugzi...@digitalmars.com --- Comment #3 from Walter Bright bugzi...@digitalmars.com 2011-05-17 12:11:54 PDT --- (In reply to comment #1) std.datetime won't work with anything prior to Win2k pro because some of the functions that it needs didn't exist before that. Which functions? -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 6024] Windows 2000 SP4 is not supported any more? And what is still supported?
http://d.puremagic.com/issues/show_bug.cgi?id=6024 --- Comment #4 from Walter Bright bugzi...@digitalmars.com 2011-05-17 12:12:48 PDT --- (In reply to comment #2) It looks like the RtlCaptureContext call can be replaced with some simple assembler. All it needs is the contents of the EIP, EBP and ESP registers. Some compilers offer intrinsics for special CPU registers, although I don't think DMD does. DMD has inline assembler, so this should be straightforward. What exactly is needed? -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
[Issue 6024] Windows 2000 SP4 is not supported any more? And what is still supported?
http://d.puremagic.com/issues/show_bug.cgi?id=6024 --- Comment #5 from Vladimir Panteleev thecybersha...@gmail.com 2011-05-17 12:21:10 PDT --- (In reply to comment #4) (In reply to comment #2) It looks like the RtlCaptureContext call can be replaced with some simple assembler. All it needs is the contents of the EIP, EBP and ESP registers. Some compilers offer intrinsics for special CPU registers, although I don't think DMD does. DMD has inline assembler, so this should be straightforward. What exactly is needed? The contents of the EIP, EBP and ESP registers. (I only mentioned intrinsics because they would be favorable to inline assembler.) StackWalk64 also accepts a CONTEXT structure, which, according to its documentation, is optional on x86, but may improve its behavior in certain situations. Thus, RtlCaptureContext should still be used when it is available. https://github.com/D-Programming-Language/druntime/blob/5098e60a74d5ff06aa808a5f02202f4762feac03/src/core/sys/windows/stacktrace.d#L288 -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email --- You are receiving this mail because: ---
