Re: Curl wrapper
On Mon, 16 May 2011 17:07:57 -0400, jdrewsen wrote:
Hi,
I've been working on a wrapper for the etc.c.curl module. It is now
pretty stable and I would very much like some feedback on the API.
http://freeze.steamwinter.com/D/web/phobos/etc_curl.html
BTW I use template mixins which doesn't seem to get included in the
generated docs. Is there any way I can make this work?
/Jonas
I think the API looks good with regard to the core functionality it
provides, but the documentation is sub-par, an certain structs should be
defined inside the Http class. I've listed a bunch of comments I had while
reading the documentation below.
One minor question regarding the actual implementation: do you define a
pragma(lib, ...) to ease setup/tool chain pains, or do you intend
something different?
Regarding the First example:
* Use string instead of const(char)[]
* Don't waste vertical space with empty comment lines like this
//
// This comment style
//
* Comment lines should be grammatically correct, i.e. '// GET using async
range' should be '// GET using an asynchronous range' or '// Perform a GET
using an asynchronous range', etc
* Don't use 'l' as a variable. It looks like a 1. Also, using 'line'
would make the meaning clearer
* Example code should not exceed 80 chars in width. People's browser
setups vary, and wrapped lines are bad.
* You can drop the 'delegate size_t' from the PUT example by making the
return 0u, 0U or size_t.init.
* The last example is missing a description. (Also, it goes over 80 chars)
The license section should include a mention of libcurl's license, etc in
addition to the wrapper's license. In general, for non-Boost libraries
like this, should we include the license/copyright as a string/tuple enum?
(i.e. for use in about boxes / command line options) Should said
license/copyright be register to a global repository upon module import?
(i.e. for auto-magical about boxes, etc)
'Do not use the same instance of this class in two threads simultanously.'
First, you need to spell check all your documentation (i.e. simultaneously
vs simultanously). Second, this should be assumed by most programmers to
be true of all non-shared classes. Third, if you do want to leave this
line in, use something short and sweet, like 'Http is not thread safe',
instead.
You need to look at/think about the order in which things are declared.
Having things like HttpMethod declared long after Http doesn't make any
sense. Also, it looks like you're not using '///ditto' for HttpMethod.
On that note, why is HttpMethod a free-standing enum and not declared
inside http? Http.Method, Http.Result, etc. make a lot more sense.
The setTimeCondition documentation seems a bit confusing.
I'd recommend stream-lining the convenience function documentation. For
example, http.head only requires a /// Returns: An HttpResult containing
the url's headers. Alternatively, you could use ditto and merge the
documentation and examples for head/get/etc into a single block. Also,
headAsync seems to document a data argument, but not take one, while post
has an undocumented parameter, etc. In general, you might want to consider
removing some of the redundant/self-obvious parameter documentation
blocks. On second thought, use ditto, slim down the docs and move this set
of functions either to the beginning or end of Http. It would make sense
to put these right after the definition of Http.Method/method. You also
might want to consider grouping all the xxxAsync functions together,
instead of interleaving them.
Check the grammar in http.postData
http.onReceiveHeader has issues with it's 'See' link. It's example is too
wide. You don't need really need the parameter documentation, particularly
if you improve the example, i.e.:
string[string] headerInformation;
with(auto http = new Http("http://www.google.com";)) {
onReceiveHeader = (string key, string value) {
if ( value.length <= 10 )
headerInformation[key.idup] = value.idup;
};
onReceive = (ubyte[] data) { };
perform;
}
Also, an tab/indent should be 4 spaces not 5 and no example should start
indented. Remember to comment like this:
/** My regular comment
* Example:
---
// Example code
---
* Other Stuff:
*/
You need to fix/complete onSend's documentation.
contentLength(size_t len); => contentLength(size_t length);
'Perform http request' => 'Perform an http request'
AuthMethod/CurlAuth need to be part of docs.
Why is setAuthenticationMethod not authenticationMethod or authentication?
(And why is AuthMethod not AuthenticationMethod?)
Similarly, why is setTimeCondition not timeCondition and CurlTimeCond not
Http.TimeCondition?
Why is followLocation(int maxRedirs) not named maxRedirections?
HttpResult.text should have huge warning flags with regard to use and/or
have its design re-thought. Is there some reason
Re: Curl wrapper
On Tue, 17 May 2011 07:43:06 +0300, Andrei Alexandrescu wrote: 3. Data on wire is ubyte[], not void[]. I was the one who suggested using void[]. If data on wire is ubyte[], why is data on disk void[] (according to std.file.read/write)? -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: Curl wrapper
On 05/16/2011 04:07 PM, jdrewsen wrote: Hi, I've been working on a wrapper for the etc.c.curl module. It is now pretty stable and I would very much like some feedback on the API. http://freeze.steamwinter.com/D/web/phobos/etc_curl.html BTW I use template mixins which doesn't seem to get included in the generated docs. Is there any way I can make this work? /Jonas Now that's one sight for sore eyes. Nice work. A few comments and nits: 1. Http can't be a class. Network resources are the poster child of expensive resources that are NOT memory and don't work well with garbage collection. RAII is the poster child of deterministic resource management. So... 2. Furthermore I see customization via delegates (onReceive etc). That should be a good proxy for customization instead of using inheritance. 3. Data on wire is ubyte[], not void[]. 4. Shouldn't onSend take a ref to its argument in case it wants to expand it? 5. Why HttpMethod and not Http.Method? 6. "Do not use the same instance of this class in two threads simultanously." -> remove. A D program isn't supposed to do that unless you offer shared methods. 7. HttpStatusLine -> Http.StatusLine. One up for encapsulation. 8. HttpMethod -> Http.Method. Ditto. 9. "this(in const(char)[] url);" -> "this(string url);" I assume you store a copy of the URL inside by calling e.g. to!string. If the client has a string, you refuse to tap into that nice information and share the string safely and efficiently, and make a copy of it for no good reason. If the client doesn't have a string, no worry, the compiler will tell her she needs to do a copy. 10. addHeader etc. Same comment. Use string whenever you want to keep a copy anyway. 11. setTimeCondition -> use core.Duration for time representation throughout. 12. AsyncHttpResult -> Http.AsyncResult :o) 13. HttpResult -> Http.Result 14. Isn't the max redirect configurable via a parameter? 15. Typo: "Callbacks is not supported..." 16. Documentation should point to descriptions of the HTTP methods wrapped (e.g. "post", "del" etc). 17. There should be an example with login/password. 18. See onReceiveHeader">std.curl.Curl.onReceiveHeader and others look like doc macros gone wrong. 19. "chuncked" -> "chunked" 20. "Max allowed redirs. -1 for infinite." -> use uint and uint.max for infinite. 21. "short isRunning()" -> what's the semantics of the short? 22. byLine/byChunk should not expose a string or generally data that can be shared safely by the client. That's because it would need to create a new string with each iteration, which is very inefficient. Instead, they should expose char[]/ubyte[] just like File.byLine does, and reuse the buffer with each call. Essentially byLine/byChunk should be near-zero-overhead means to transfer and inspect arbitrarily large streams. 23. Ftp should have get in-memory and streaming byLine/byChunk, not only get to file. 24. The shutdown mechanism should be handled properly. Shutting down libcurl would have all pending transfers instantly throw a special exception. Without a shutdown API, applications won't be able to implement e.g. a Quit button. Again, great work! Andrei
Re: Curl wrapper
Am 17.05.2011 02:43, schrieb dsimcha: == Quote from jdrewsen (jdrew...@nospam.com)'s article Hi, I've been working on a wrapper for the etc.c.curl module. It is now pretty stable and I would very much like some feedback on the API. http://freeze.steamwinter.com/D/web/phobos/etc_curl.html BTW I use template mixins which doesn't seem to get included in the generated docs. Is there any way I can make this work? /Jonas This looks very good and I'd definitely be in favor of including it in the next release. However, there are a few small issues: 1. In the docs for Http, I don't think you need to say "Do not use the same instance in two threads simultaneously." I think this is pretty much common sense. Same with the Ftp class. Maybe it's common sense, but it doesn't hurt to have that information. Someone will certainly try that and ask why it gives strange results ;) 2. A short overview of the asynch stuff at the top of the file would be nice. I'm a little confused about what it's for and when to use it instead of the regular synchronous functions. Does this allow you to process the beginning of a request while the end is still being received? 3. I don't think you need to say how all the convenience functions are implemented, i.e. "Internally this is implemented using an instance of the Http class." This is both likely to be assumed and an irrelevant implementation detail. Overall, nice work.
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On 5/16/2011 9:09 PM, Jonathan M Davis wrote: On 2011-05-16 21:04, Don wrote: It's not easy. Simply disallowing it would make code like this illegal: char c = '0' + n; Well, forcing the n to be cast to char or byte would do it, but that _is_ kind of ugly. char c = '0' + cast(char)n; Definitely a good point though. It's THE reason why such casts are not allowed. Pascal requires such casts, and I always hated it for that. The casts are not only annoying, they can cause bugs when the types of the rest of the expression are changed in maintenance. There isn't any simple answer to mixed integer type arithmetic, although there are many "obvious" answers that have subtly pernicious behavior.
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On May 17, 11 12:04, Don wrote: Nick Sabalausky wrote: "Steven Schveighoffer" wrote in message news:op.vvk48tn9eav7ka@localhost.localdomain... (int/short/byte can sometimes implicitly cast to dchar/wchar/char) What do you think? Ick! Kill it! It's not easy. Simply disallowing it would make code like this illegal: char c = '0' + n; It is already illegal. You cannot implicitly convert an 'int' to 'char' (Error: cannot implicitly convert expression (48 + n) of type int to char).
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On 2011-05-16 21:04, Don wrote: > Nick Sabalausky wrote: > > "Steven Schveighoffer" wrote in message > > news:op.vvk48tn9eav7ka@localhost.localdomain... > > (int/short/byte can sometimes implicitly cast to dchar/wchar/char) > > > >> What do you think? > > > > Ick! Kill it! > > It's not easy. Simply disallowing it would make code like this illegal: > > char c = '0' + n; Well, forcing the n to be cast to char or byte would do it, but that _is_ kind of ugly. char c = '0' + cast(char)n; Definitely a good point though. - Jonathan M Davis
Re: should int/short/byte implicitly cast to dchar/wchar/char?
Nick Sabalausky wrote: "Steven Schveighoffer" wrote in message news:op.vvk48tn9eav7ka@localhost.localdomain... (int/short/byte can sometimes implicitly cast to dchar/wchar/char) What do you think? Ick! Kill it! It's not easy. Simply disallowing it would make code like this illegal: char c = '0' + n;
Re: Curl wrapper
It looks nice. I plan to use this soon. I hope it can be in phobos soon.
Re: Curl wrapper
== Quote from jdrewsen (jdrew...@nospam.com)'s article > Hi, > I've been working on a wrapper for the etc.c.curl module. It is now > pretty stable and I would very much like some feedback on the API. > http://freeze.steamwinter.com/D/web/phobos/etc_curl.html > BTW I use template mixins which doesn't seem to get included in the > generated docs. Is there any way I can make this work? > /Jonas This looks very good and I'd definitely be in favor of including it in the next release. However, there are a few small issues: 1. In the docs for Http, I don't think you need to say "Do not use the same instance in two threads simultaneously." I think this is pretty much common sense. Same with the Ftp class. 2. A short overview of the asynch stuff at the top of the file would be nice. I'm a little confused about what it's for and when to use it instead of the regular synchronous functions. Does this allow you to process the beginning of a request while the end is still being received? 3. I don't think you need to say how all the convenience functions are implemented, i.e. "Internally this is implemented using an instance of the Http class." This is both likely to be assumed and an irrelevant implementation detail. Overall, nice work.
Re: Article Review: Migrating from std.date to std.datetime
> Jonathan M Davis wrote: > > On 2011-05-15 03:34, Rainer Schuetze wrote: > >> - my current uses of datetime are comparing file times and displaying > >> the file time. Much better than std.date, the times displayed are now > >> the same as shown by Explorer/dir most of the time, but some are off by > >> one hour. It seems this happens for times with a different daylight > >> saving. Is this a bug? Or do I need to call something else than > >> > >> writeln(timeLastModified(file).toSimpleString()); > >> > >> The ISO versions or conversion to DateTime produce the same output. My > >> local time is UTC+1+DST. > > > > I'd need more data to have any idea what's going on, but it sounds like a > > bug. > > I have investigated it a bit, and I'm currently thinking that Explorer, > dir and Total Commander are wrong, and std.datetime is right. > > What I did to test: > - create a new file, verify the dates printed are correct. fine in all > programs > - set the system time back to February > - create the file again, verify the dates printed are correct. fine in > all programs > - set the system time back to May > - now the displayed file time is the same when using std.datetime, but > an hour later with all other programs. > > Maybe it's an XP or NTFS issue, but I would expect the file time not to > change with the date it is displayed. Well, Microsoft made the mistake of making it so that Windows boxes keep their time in local time (which they probably can't fix and maintain backwards compatability, so we're likely stuck), and that can have some interesting consequences, since that means that the system clock actually gets changed when DST begins or ends. I would have thought that the OS and file system would handle that well enough for Explorer to display the time correctly, but I don't know. Given how hard it can be to catch DST-related bugs and the fact that people don't generally examine the times on their files closely enough to verify that their times aren't 1 hour off 6 months later, it wouldn't entirely surprise me if there were bugs with regards to that on XP. But I still would have thought that they would have caught it. - Jonathan M Davis
Re: reddit discussion about Go turns to D again
> "Mike Parker" wrote in message > news:iqrbht$1k1o$1...@digitalmars.com... > > > On 5/16/2011 12:06 PM, Andrei Alexandrescu wrote: > >> On 05/15/2011 10:04 PM, Daniel Gibson wrote: > >>> So you have to write > >>> std.parallel_algorithm.map() instead of map() all the time? > >> > >> alias std.parallel_algorithm p; > >> > >> > >> Andrei > > > > Or this, which I prefer to alias: > > > > import p = std.parallel_algorithm; > > What would be the difference between... > > alias std.parallel_algorithm p; > > ...and... > > import p = std.parallel_algorithm; > > ..? Without private on the alias, it'll affect any module which imports the module that you created the alias on, and thanks to http://d.puremagic.com/issues/show_bug.cgi?id=6013 it'll happen anyway. I would also expect that using std.parallel_algorithm.func would still work with the alias whereas it wouldn't with the import p. - Jonathan M Davis
Re: reddit discussion about Go turns to D again
On 5/16/11 5:03 PM, Nick Sabalausky wrote: "Mike Parker" wrote in message news:iqrbht$1k1o$1...@digitalmars.com... On 5/16/2011 12:06 PM, Andrei Alexandrescu wrote: On 05/15/2011 10:04 PM, Daniel Gibson wrote: So you have to write std.parallel_algorithm.map() instead of map() all the time? alias std.parallel_algorithm p; Andrei Or this, which I prefer to alias: import p = std.parallel_algorithm; What would be the difference between... alias std.parallel_algorithm p; ...and... import p = std.parallel_algorithm; ..? One line. Andrei
Re: Article Review: Migrating from std.date to std.datetime
Jonathan M Davis wrote: On 2011-05-15 03:34, Rainer Schuetze wrote: - my current uses of datetime are comparing file times and displaying the file time. Much better than std.date, the times displayed are now the same as shown by Explorer/dir most of the time, but some are off by one hour. It seems this happens for times with a different daylight saving. Is this a bug? Or do I need to call something else than writeln(timeLastModified(file).toSimpleString()); The ISO versions or conversion to DateTime produce the same output. My local time is UTC+1+DST. I'd need more data to have any idea what's going on, but it sounds like a bug. I have investigated it a bit, and I'm currently thinking that Explorer, dir and Total Commander are wrong, and std.datetime is right. What I did to test: - create a new file, verify the dates printed are correct. fine in all programs - set the system time back to February - create the file again, verify the dates printed are correct. fine in all programs - set the system time back to May - now the displayed file time is the same when using std.datetime, but an hour later with all other programs. Maybe it's an XP or NTFS issue, but I would expect the file time not to change with the date it is displayed.
Re: reddit discussion about Go turns to D again
"Mike Parker" wrote in message news:iqrbht$1k1o$1...@digitalmars.com... > On 5/16/2011 12:06 PM, Andrei Alexandrescu wrote: >> On 05/15/2011 10:04 PM, Daniel Gibson wrote: > >>> >>> So you have to write >>> std.parallel_algorithm.map() instead of map() all the time? >> >> alias std.parallel_algorithm p; >> >> >> Andrei > > Or this, which I prefer to alias: > > import p = std.parallel_algorithm; > What would be the difference between... alias std.parallel_algorithm p; ...and... import p = std.parallel_algorithm; ..?
Re: Arrays are sufficient for ArrayLists? Really??
On May 16, 2011, at 12:01 PM, Mehrdad wrote: > > Sean: >> Removing and then appending one element to an array won't cause any >> allocations to occur. > > How is that possible, though? (See the example in my response to Timon.) I wasn't thinking about how remove() worked. You'd have to adjust the length property of the original array before appending or it would think you were appending to a slice.
Curl wrapper
Hi, I've been working on a wrapper for the etc.c.curl module. It is now pretty stable and I would very much like some feedback on the API. http://freeze.steamwinter.com/D/web/phobos/etc_curl.html BTW I use template mixins which doesn't seem to get included in the generated docs. Is there any way I can make this work? /Jonas
Re: reddit discussion about Go turns to D again
"Matthew Ong" wrote in message
news:iqr858$1eqa$1...@digitalmars.com...
> On 5/15/2011 5:56 PM, Timon Gehr wrote:
>>
>> I think D has difficulties getting new users, although it is superior to
>> any
>> programming language I know in almost every way.
>>
> Yes. D has far more syntax and well above Java & I think even C#.
>
Oh absolutely. Just try to do this in C#:
T add(T)(T a, T b)
{
return a+b;
}
Not even *possible* to make templated arithmetic work in C#.
Re: should int/short/byte implicitly cast to dchar/wchar/char?
"Steven Schveighoffer" wrote in message news:op.vvk48tn9eav7ka@localhost.localdomain... > (int/short/byte can sometimes implicitly cast to dchar/wchar/char) > > What do you think? > Ick! Kill it!
Re: Implementing std.log
Andrei Alexandrescu wrote: > On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: > [snip] > > I updated my std.log draft. Added a lot of features including > formatted writing, delayed logging, and a variety of configuration > options. Replaced the redundant log.xyz with logXyz. The > implementation is getting close to reviewable form. > > Documentation: > > http://d-programming-language.org/phobos-prerelease/std_log.html > > Source: > > https://github.com/andralex/phobos > > Feedback welcome. I started to test it out. Some things to consider: * If I miss a minus on a program option that option will be silently missed. E.g. I wrote -minloglevel 0 instead of --minloglevel 0 and was quite puzzled that I didn't get all messages. * Rename TEST_TMPDIR to LOG_TMPDIR or LOG_DIR? * I believe printing a ulong as thread id is too wasteful and makes the output harder to read. * I would arrange the loggers in increasing order of importance in the documentation (and source), i.e. logInfo, logWarning, ... And at very last: vlog. In the same order as done in the synopsis. * I'm unsure whether the logging levels should better be implemented using enum instead of dchars[]. Because 'I', 'W', 'E', 'C', 'F' are used several times in the source. * Instead of logInfo, logWarning, logError etc. just naming them info, warn, error etc. may be an option. I think it's quite obvious that these are logging statements. No need for the prefix log? * Even though vlog is short maybe logVerbose is more consistent. But admittedly it's quite long. I'm trying to read linker map files to evaluate how much code is generated when using the strip_log_* version options. Is there some good documentation on how to read those files? I believe I will have to read some basic literature about linker and loaders. Jens
Re: Article Review: Migrating from std.date to std.datetime
> Fantastic work! I was lost in that std.datetime jungle for a while > now. It's a great read (your English is pretty fluent!). I would hope so. I'm a native of California. English is my native language. I'd be more worried about my French fluency (which is good but likely deteriorating since I haven't been to Europe recently) than my English fluency. And, of course, I'm not fluent in any other language, so I'd be complete rubbish at anything else. > Btw, I think there's a missing word in this sentence: "If what you're > doesn't need that extra boost of efficiency. (doing?). Yes. It sounds like it's the word doing that's missing. I'll fix it this evening. - Jonathan M Daviss
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 1:22 PM, Andrei Alexandrescu wrote: It does seem that all answers were ill-fitted to the way the problem has been ultimately formulated. Andrei Yeah, there's no clean answer to this that's of this form, so I'm just working around the problem. Thanks everybody for your input, I appreciate it! :)
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 23:22:21 +0300, Andrei Alexandrescu wrote: On 5/16/11 3:15 PM, Vladimir Panteleev wrote: On Mon, 16 May 2011 23:11:50 +0300, Andrei Alexandrescu wrote: I tested the function above. OP's intention was to create a version of removeAt which modifies the array in-place only when there are no other aliases towards the data. The initial approach was to attempt to check if the passed array is a slice of a larger array, but as it was discussed, it's flawed. The parameters of the problem have been re(de)fined several times during this discussion. Various responses focused on as many perceptions of what the request was. Some code snippets were wrong in the sense that they didn't work within their own specification. Those are different from responses that do work within their charter, hence my clarification. It does seem that all answers were ill-fitted to the way the problem has been ultimately formulated. I wasn't challenging your statement, sorry if I made it sound like that. -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 16:17:45 -0400, Mehrdad wrote: On 5/16/2011 1:15 PM, Vladimir Panteleev wrote: On Mon, 16 May 2011 23:11:50 +0300, Andrei Alexandrescu wrote: I tested the function above. OP's intention was to create a version of removeAt which modifies the array in-place only when there are no other aliases towards the data. The initial approach was to attempt to check if the passed array is a slice of a larger array, but as it was discussed, it's flawed. +1 Wait, I don't think the approach is flawed, just not correctly implemented (see my suggestion with capacity). If you have an array: [1, 2, 3, 4] and you want to remove the 2nd element, another alias to the original array would be: [1, 3, 4, 4] Are you trying to argue that the last 4 in that array is still valid data? If so, then yes, it's impossible to tell if there are other aliases to the same data. All you can tell is if the data ends at the valid data for that block (via the capacity property). I can't see any use case for continuing to use the original alias, because the data is obviously corrupt. Any alias that still references that last element would be corrupted. If you passed in the slice [0..3], you get an original aliased array of: [1, 3, 3, 4] Now, anything that references that third element is corrupt. But that fourth element is not corrupted, it wasn't part of the arguments to removeAt. So you can continue to refer to that element, and a properly implemented removeAt should avoid stomping on that element with a later append. -Steve
Re: Article Review: Migrating from std.date to std.datetime
Fantastic work! I was lost in that std.datetime jungle for a while now. It's a great read (your English is pretty fluent!). Btw, I think there's a missing word in this sentence: "If what you're doesn't need that extra boost of efficiency. (doing?).
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/11 3:15 PM, Vladimir Panteleev wrote: On Mon, 16 May 2011 23:11:50 +0300, Andrei Alexandrescu wrote: I tested the function above. OP's intention was to create a version of removeAt which modifies the array in-place only when there are no other aliases towards the data. The initial approach was to attempt to check if the passed array is a slice of a larger array, but as it was discussed, it's flawed. The parameters of the problem have been re(de)fined several times during this discussion. Various responses focused on as many perceptions of what the request was. Some code snippets were wrong in the sense that they didn't work within their own specification. Those are different from responses that do work within their charter, hence my clarification. It does seem that all answers were ill-fitted to the way the problem has been ultimately formulated. Andrei
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 16:14:33 -0400, Timon Gehr wrote:
Steven Schveighoffer wrote:
On Mon, 16 May 2011 15:52:23 -0400, Mehrdad
wrote:
On 5/16/2011 12:53 PM, Timon Gehr wrote:
In fact I even need to take that back. In order to work correctly,
the
function would have to iterate downwards. It _is_ indeed buggy, and I
should stop emitting opinions when I'm short on time...
Andrei
Whoops, you are right:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; retro(arr[1 .. index+1]))
item = arr[i - 1];
arr = arr[1 .. $];
}
Timon
Wouldn't that stomp on the super-slice of arr, though?
As was pointed out on SO, the problem is actually always there: if
someone passes arr[0 .. $] to a function, it will look as if the
original array was passed, although it wasn't. Seems like it's a lot
uglier than I'd thought...
arr[0..$] shouldn't be an lvalue (it is, but I think that's a bug).
Therefore, you shouldn't be able to pass it as a ref argument (is this
in
bugzilla?).
-Steve
I think it is a little bit more subtle than "not lvalue", because you
actually
have to be able to do something like:
arr[0..2] = arr[3..5];
Where a slice appears on the left hand side of the assignment. (also
a[]=b[]).
ref is not required in those cases, the array can be passed by value (and
by value, I mean the pointer and length, not the data).
You only need the array to be an lvalue when you want the passed-in
array's pointer and length to be modified.
-Steve
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 1:15 PM, Vladimir Panteleev wrote: On Mon, 16 May 2011 23:11:50 +0300, Andrei Alexandrescu wrote: I tested the function above. OP's intention was to create a version of removeAt which modifies the array in-place only when there are no other aliases towards the data. The initial approach was to attempt to check if the passed array is a slice of a larger array, but as it was discussed, it's flawed. +1
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 23:11:50 +0300, Andrei Alexandrescu wrote: I tested the function above. OP's intention was to create a version of removeAt which modifies the array in-place only when there are no other aliases towards the data. The initial approach was to attempt to check if the passed array is a slice of a larger array, but as it was discussed, it's flawed. -- Best regards, Vladimirmailto:vladi...@thecybershadow.net
Re: Arrays are sufficient for ArrayLists? Really??
Steven Schveighoffer wrote:
> On Mon, 16 May 2011 15:52:23 -0400, Mehrdad wrote:
>
>> On 5/16/2011 12:53 PM, Timon Gehr wrote:
In fact I even need to take that back. In order to work correctly, the
function would have to iterate downwards. It _is_ indeed buggy, and I
should stop emitting opinions when I'm short on time...
Andrei
>>>
>>> Whoops, you are right:
>>>
>>> void removeAt(T)(ref T[] arr, size_t index)
>>> {
>>> foreach (i, ref item; retro(arr[1 .. index+1]))
>>> item = arr[i - 1];
>>> arr = arr[1 .. $];
>>> }
>>>
>>> Timon
>>
>> Wouldn't that stomp on the super-slice of arr, though?
>>
>> As was pointed out on SO, the problem is actually always there: if
>> someone passes arr[0 .. $] to a function, it will look as if the
>> original array was passed, although it wasn't. Seems like it's a lot
>> uglier than I'd thought...
>
> arr[0..$] shouldn't be an lvalue (it is, but I think that's a bug).
> Therefore, you shouldn't be able to pass it as a ref argument (is this in
> bugzilla?).
>
> -Steve
I think it is a little bit more subtle than "not lvalue", because you actually
have to be able to do something like:
arr[0..2] = arr[3..5];
Where a slice appears on the left hand side of the assignment. (also a[]=b[]).
Timon
Re: Arrays are sufficient for ArrayLists? Really??
Andrei Alexandrescu wrote:
> A On 5/16/11 3:04 PM, Mehrdad wrote:
>> On 5/16/2011 12:59 PM, Andrei Alexandrescu wrote:
>>> On 5/16/11 2:56 PM, Timon Gehr wrote:
>>> I think it would take less time to actually paste the function in a file
>>> and try it. A cute possibility:
>>>
>>> void removeAt(T)(ref T[] arr, size_t index)
>>> {
>>> copy(retro(arr[0 .. index]), retro(arr[1 .. index + 1]));
>>> arr = arr[1 .. $];
>>> }
>>>
>>>
>>> Andrei
>
> Sorry guys, _none_ of these unfortunately works.
> See CyberShadow's reply on my question on SO:
> http://stackoverflow.com/q/6015008/541686
>
> I tested the function above.
>
> Andrei
Timon Gehr wrote:
> void removeAt(T)(ref T[] arr, size_t index){
> for(auto i = index; i; i--) arr[i] = arr[i - 1];
> arr = arr[1 .. $];
> }
This one works too.
Timon
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On Mon, 16 May 2011 16:06:34 -0400, KennyTM~ wrote:
The compiler blindly accepts an 'int' as a 'dchar' argument, not -1. For
instance,
void main(){
string ret;
ret ~= 0x10; // ok
ret ~= 0x11; // Error: cannot append type int to type string
int i = 0x11;
ret ~= i; // ok, but should fail at compile time
}
the issue is also that an 'int' shouldn't be implicitly convertible to a
'dchar'.
Yes, you are completely right, it seems the compiler isn't inconsistent
WRT string appending and calling a normal function, it fails/passes the
same way, depending on whether the argument passed is a variable or a
literal.
So it seems the correct solution is for it to do range propagation, and
refuse to compile implicit casts where it results in an invalid dchar
(which includes the case where the int could be anything).
Sorry for the confusion, I thought I tested one thing, and actually tested
another, yada yada...
-Steve
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/11 3:04 PM, Mehrdad wrote:
On 5/16/2011 12:59 PM, Andrei Alexandrescu wrote:
On 5/16/11 2:56 PM, Timon Gehr wrote:
I think it would take less time to actually paste the function in a file
and try it. A cute possibility:
void removeAt(T)(ref T[] arr, size_t index)
{
copy(retro(arr[0 .. index]), retro(arr[1 .. index + 1]));
arr = arr[1 .. $];
}
Andrei
Sorry guys, _none_ of these unfortunately works.
See CyberShadow's reply on my question on SO:
http://stackoverflow.com/q/6015008/541686
I tested the function above.
Andrei
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 15:52:23 -0400, Mehrdad wrote:
On 5/16/2011 12:53 PM, Timon Gehr wrote:
In fact I even need to take that back. In order to work correctly, the
function would have to iterate downwards. It _is_ indeed buggy, and I
should stop emitting opinions when I'm short on time...
Andrei
Whoops, you are right:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; retro(arr[1 .. index+1]))
item = arr[i - 1];
arr = arr[1 .. $];
}
Timon
Wouldn't that stomp on the super-slice of arr, though?
As was pointed out on SO, the problem is actually always there: if
someone passes arr[0 .. $] to a function, it will look as if the
original array was passed, although it wasn't. Seems like it's a lot
uglier than I'd thought...
arr[0..$] shouldn't be an lvalue (it is, but I think that's a bug).
Therefore, you shouldn't be able to pass it as a ref argument (is this in
bugzilla?).
-Steve
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On May 17, 11 03:58, Steven Schveighoffer wrote:
On Mon, 16 May 2011 15:47:55 -0400, KennyTM~ wrote:
On May 17, 11 02:25, Steven Schveighoffer wrote:
On Mon, 16 May 2011 13:51:55 -0400, Steven Schveighoffer
wrote:
Currently, this works:
void foo(dchar i)
{
}
void main(string[] args)
{
foo(args.length);
}
Damn, this originally started out as argc and argv, and I forgot how D
accepts arguments, so I switched it to this. Unsigned ints are
convertable to dchar, but signed ones are not (except for a couple
cases, which doesn't make sense).
For example, this fails:
dchar c = -1;
foo(-1);
This fails because the compiler can check in compile-time that
0x_ is > 0x10_
That is not what the error suggests:
Error: cannot implicitly convert expression (-1) of type int to dchar
Seems like it's the type that's failing. At the very least, the error
message should stress it's the value, not the type, that is the culprit.
If I change the literal to 0x10_, indeed it passes, and 0x11_
fails.
I think this is probably the right behavior, it can be useful to use
binary or hex numbers to construct characters.
But this passes:
int i = -1;
dchar c = i;
but this cannot. 'dchar' should be treated as lower-rank than
'int' and use value-range propagation on it.
I'm not sure what you mean, I think the dchar init statement should
fail. I think this is what you are saying, right?
Right.
Note in the bug report previously referenced, the compiler blindly
accepts -1 as a dchar argument.
-Steve
The compiler blindly accepts an 'int' as a 'dchar' argument, not -1. For
instance,
void main(){
string ret;
ret ~= 0x10; // ok
ret ~= 0x11; // Error: cannot append type int to type string
int i = 0x11;
ret ~= i; // ok, but should fail at compile time
}
the issue is also that an 'int' shouldn't be implicitly convertible to a
'dchar'.
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 12:59 PM, Andrei Alexandrescu wrote:
On 5/16/11 2:56 PM, Timon Gehr wrote:
I think it would take less time to actually paste the function in a file
and try it. A cute possibility:
void removeAt(T)(ref T[] arr, size_t index)
{
copy(retro(arr[0 .. index]), retro(arr[1 .. index + 1]));
arr = arr[1 .. $];
}
Andrei
Sorry guys, _none_ of these unfortunately works.
See CyberShadow's reply on my question on SO:
http://stackoverflow.com/q/6015008/541686
Re: Arrays are sufficient for ArrayLists? Really??
foreach is a very bad choice for solving this. I blindly took it over from the
original code. Need to get some sleep :).
This now definitely works, and is also the shortest:
void removeAt(T)(ref T[] arr, size_t index){
for(auto i = index; i; i--) arr[i] = arr[i - 1];
arr = arr[1 .. $];
}
Timon
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/11 2:56 PM, Timon Gehr wrote:
Timon Gehr wrote:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; retro(arr[1 .. index+1]))
item = arr[i - 1];
arr = arr[1 .. $];
}
Sorry, still wrong:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; retro(arr[1 .. index+1]))
item = arr[index - i - 1];
arr = arr[1 .. $];
}
I think it would take less time to actually paste the function in a file
and try it. A cute possibility:
void removeAt(T)(ref T[] arr, size_t index)
{
copy(retro(arr[0 .. index]), retro(arr[1 .. index + 1]));
arr = arr[1 .. $];
}
Andrei
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 12:53 PM, Andrei Alexandrescu wrote: On 5/16/11 2:44 PM, Mehrdad wrote: The function as implemented first assigns a[1]=a[0] and then a[2]=a[1], leading to [0, 0, 0, 3, 4] which is not what's needed. Oh yeah good point. Is there any way to find out if an array is a slice of another array? No. If there would, it would be a rather advanced function. Allow me to reiterate my suggestion: it's likely you have a simple problem with a simple solution based on slices, which are a simple abstraction. Therefore I suggest you rethink a bit of your design, and it's possible you won't need assumeSafeAppend et al. Andrei Yeah, seems like even if it was, it wouldn't solve the problem in all cases. I'll try going around the problem, thanks.
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On Mon, 16 May 2011 15:47:55 -0400, KennyTM~ wrote:
On May 17, 11 02:25, Steven Schveighoffer wrote:
On Mon, 16 May 2011 13:51:55 -0400, Steven Schveighoffer
wrote:
Currently, this works:
void foo(dchar i)
{
}
void main(string[] args)
{
foo(args.length);
}
Damn, this originally started out as argc and argv, and I forgot how D
accepts arguments, so I switched it to this. Unsigned ints are
convertable to dchar, but signed ones are not (except for a couple
cases, which doesn't make sense).
For example, this fails:
dchar c = -1;
foo(-1);
This fails because the compiler can check in compile-time that
0x_ is > 0x10_
That is not what the error suggests:
Error: cannot implicitly convert expression (-1) of type int to dchar
Seems like it's the type that's failing. At the very least, the error
message should stress it's the value, not the type, that is the culprit.
If I change the literal to 0x10_, indeed it passes, and 0x11_
fails.
I think this is probably the right behavior, it can be useful to use
binary or hex numbers to construct characters.
But this passes:
int i = -1;
dchar c = i;
but this cannot. 'dchar' should be treated as lower-rank than 'int'
and use value-range propagation on it.
I'm not sure what you mean, I think the dchar init statement should fail.
I think this is what you are saying, right?
Note in the bug report previously referenced, the compiler blindly accepts
-1 as a dchar argument.
-Steve
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On 2011-05-16 12:47, KennyTM~ wrote:
> On May 17, 11 02:25, Steven Schveighoffer wrote:
> > On Mon, 16 May 2011 13:51:55 -0400, Steven Schveighoffer
> >
> > wrote:
> >> Currently, this works:
> >>
> >> void foo(dchar i)
> >> {
> >> }
> >>
> >> void main(string[] args)
> >> {
> >> foo(args.length);
> >> }
> >
> > Damn, this originally started out as argc and argv, and I forgot how D
> > accepts arguments, so I switched it to this. Unsigned ints are
> > convertable to dchar, but signed ones are not (except for a couple
> > cases, which doesn't make sense).
> >
> > For example, this fails:
> >
> > dchar c = -1;
> > foo(-1);
>
> This fails because the compiler can check in compile-time that
> 0x_ is > 0x10_
>
> > But this passes:
> >
> > int i = -1;
> > dchar c = i;
>
> but this cannot. 'dchar' should be treated as lower-rank than 'int'
> and use value-range propagation on it.
I'd argue that assigning an int to a dchar should fail in all cases where
there is no cast and where the compiler cannot use value-range propogation to
verify that the value being assigned to the dchar is a valid value for a
dchar. So, in the above case, if the compiler fails to do value-range
propagation for some reason (which it shouldn't in a case this small), the
assignment should be illegal, because there is no cast. And if the compiler
manages to do value-range propagation (as it should), then it should still
fail, because -1 is not a value value for a dchar.
- Jonathan M Davis
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/11 2:44 PM, Mehrdad wrote: On 5/16/2011 12:41 PM, Andrei Alexandrescu wrote: I was mistaken and removed my post. The code ingeniously redefines the problem - instead of removing from the array by shifting its tail downwards, it shifts elements upwards from the head of the array. A nice hack, but I don't think it does a lot in helping your situation. In fact I even need to take that back. In order to work correctly, the function would have to iterate downwards. It _is_ indeed buggy, and I should stop emitting opinions when I'm short on time... Andrei I'm not sure what you mean by iterating downwards, but I just noticed another bug: That would _still_ stomp over a superarray if this array is a slice of it, right? Iterating downwards would mean: if you want to remove the third element from a = [0, 1, 2, 3, 4]; by using head shifting, you'd need to assign a[2]=a[1], a[1]=a[0] (backwards that is) to obtain [0, 0, 1, 3, 4]. Then you take a[1 .. $] which is the result. The function as implemented first assigns a[1]=a[0] and then a[2]=a[1], leading to [0, 0, 0, 3, 4] which is not what's needed. Is there any way to find out if an array is a slice of another array? No. If there would, it would be a rather advanced function. Allow me to reiterate my suggestion: it's likely you have a simple problem with a simple solution based on slices, which are a simple abstraction. Therefore I suggest you rethink a bit of your design, and it's possible you won't need assumeSafeAppend et al. Andrei
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 15:53:38 -0400, Timon Gehr wrote:
In fact I even need to take that back. In order to work correctly, the
function would have to iterate downwards. It _is_ indeed buggy, and I
should stop emitting opinions when I'm short on time...
Andrei
Whoops, you are right:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; retro(arr[1 .. index+1]))
item = arr[i - 1];
arr = arr[1 .. $];
}
Hehe, won't work. Ranges can only support a single foreach parameter.
I think you may have to resort to for loops here (/me purposely ignores
the hideous beast foreach_reverse). Be very careful with indexes on that
one.
-Steve
Re: Arrays are sufficient for ArrayLists? Really??
Timon Gehr wrote:
> void removeAt(T)(ref T[] arr, size_t index)
> {
>foreach (i, ref item; retro(arr[1 .. index+1]))
> item = arr[i - 1];
> arr = arr[1 .. $];
> }
Sorry, still wrong:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; retro(arr[1 .. index+1]))
item = arr[index - i - 1];
arr = arr[1 .. $];
}
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 12:53 PM, Timon Gehr wrote:
In fact I even need to take that back. In order to work correctly, the
function would have to iterate downwards. It _is_ indeed buggy, and I
should stop emitting opinions when I'm short on time...
Andrei
Whoops, you are right:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; retro(arr[1 .. index+1]))
item = arr[i - 1];
arr = arr[1 .. $];
}
Timon
Wouldn't that stomp on the super-slice of arr, though?
As was pointed out on SO, the problem is actually always there: if
someone passes arr[0 .. $] to a function, it will look as if the
original array was passed, although it wasn't. Seems like it's a lot
uglier than I'd thought...
Re: Arrays are sufficient for ArrayLists? Really??
> In fact I even need to take that back. In order to work correctly, the
> function would have to iterate downwards. It _is_ indeed buggy, and I
> should stop emitting opinions when I'm short on time...
>
> Andrei
Whoops, you are right:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; retro(arr[1 .. index+1]))
item = arr[i - 1];
arr = arr[1 .. $];
}
Timon
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On May 17, 11 02:25, Steven Schveighoffer wrote:
On Mon, 16 May 2011 13:51:55 -0400, Steven Schveighoffer
wrote:
Currently, this works:
void foo(dchar i)
{
}
void main(string[] args)
{
foo(args.length);
}
Damn, this originally started out as argc and argv, and I forgot how D
accepts arguments, so I switched it to this. Unsigned ints are
convertable to dchar, but signed ones are not (except for a couple
cases, which doesn't make sense).
For example, this fails:
dchar c = -1;
foo(-1);
This fails because the compiler can check in compile-time that
0x_ is > 0x10_
But this passes:
int i = -1;
dchar c = i;
but this cannot. 'dchar' should be treated as lower-rank than 'int'
and use value-range propagation on it.
So clearly there are some inconsistencies that need to be fixed, but the
situation is not as bad as I thought it was.
-Steve
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 12:41 PM, Andrei Alexandrescu wrote: I was mistaken and removed my post. The code ingeniously redefines the problem - instead of removing from the array by shifting its tail downwards, it shifts elements upwards from the head of the array. A nice hack, but I don't think it does a lot in helping your situation. In fact I even need to take that back. In order to work correctly, the function would have to iterate downwards. It _is_ indeed buggy, and I should stop emitting opinions when I'm short on time... Andrei I'm not sure what you mean by iterating downwards, but I just noticed another bug: That would _still_ stomp over a superarray if this array is a slice of it, right? Is there any way to find out if an array is a slice of another array?
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 12:38 PM, Andrei Alexandrescu wrote: Oh, I see. Wait, what bug are you referring to, though? I was mistaken and removed my post. The code ingeniously redefines the problem - instead of removing from the array by shifting its tail downwards, it shifts elements upwards from the head of the array. A nice hack, but I don't think it does a lot in helping your situation. There's a bit of history wrt ~=. Initially the behavior was to never reallocate if there was enough capacity. But that caused a stomping issue - people would create an array, pass a slice of it to a function, and then the function would clobber elements in the arrays outside the slice. This was a major breakage of modular behavior so we decided to change the behavior: as long as you keep references to the original array, appending to a slice of it will cause reallocation of the array. One thing you could do is to think of what you're trying to achieve on a larger scale. Assuming a slice to grow over the original array is arguably a bad behavior that I'm sure could be fixed by small changes in the design. Andrei It seems to be fine for what I need, although I'm going to put a comment in my code because I'm sure I'm going to run into a problem with it down the road... for now it seems safe though. :) Thanks!
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/11 2:38 PM, Andrei Alexandrescu wrote:
On 5/16/11 2:31 PM, Mehrdad wrote:
On 5/16/2011 12:25 PM, Andrei Alexandrescu wrote:
On 5/16/11 2:25 PM, Timon Gehr wrote:
Mehrdad wrote:
Timon:
What about:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[1 .. index+1])
item = arr[i - 1];
arr = arr[1 .. $]; //note how no valid data is beyond the end of the
array
}
Clever, but if you do this with a big enough number of items, you'll
exhaust all memory. Be careful. :P
Total memory usage is at most 2 times the maximum size the array
reaches during
its lifetime. Test it.
Testing would also reveal that this is a buggy replica of
std.algorithm.remove. At best we should suggest reusing library code
over attempting to reinvent it.
Thanks,
Andrei
Oh, I see.
Wait, what bug are you referring to, though?
I was mistaken and removed my post. The code ingeniously redefines the
problem - instead of removing from the array by shifting its tail
downwards, it shifts elements upwards from the head of the array. A nice
hack, but I don't think it does a lot in helping your situation.
In fact I even need to take that back. In order to work correctly, the
function would have to iterate downwards. It _is_ indeed buggy, and I
should stop emitting opinions when I'm short on time...
Andrei
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 15:23:31 -0400, Mehrdad wrote: Steven: You need to do arr.assumeSafeAppend(); Otherwise, the runtime is no aware of your invalidation of the last element. See my edit on SO: http://stackoverflow.com/q/6015008/541686 I'm not an SO user, so I'll post my answer here: assumeSafeAppend is named that way because you are making an assumption, possibly an unsafe one :) You are saying, "hey, runtime, assume that despite what you know about this array's data, it's safe to append in this case," and the runtime says "ok, boss, no problem." It cannot tell that you intended to use that data you have just invalidated. What you want to do is determine if the array is safe to append *before* you eliminate the elements at the end (i.e. the elements you are "throwing away" are at the end of valid data anyways). You can do this with the capacity property: auto oldcap = arr.capacity; // do your dirty work ... if(oldcap) // capacity != 0 means you could originally append to this array, or that it filled up the block. arr.assumeSafeAppend(); This should do the trick. -Steve
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/11 2:31 PM, Mehrdad wrote:
On 5/16/2011 12:25 PM, Andrei Alexandrescu wrote:
On 5/16/11 2:25 PM, Timon Gehr wrote:
Mehrdad wrote:
Timon:
What about:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[1 .. index+1])
item = arr[i - 1];
arr = arr[1 .. $]; //note how no valid data is beyond the end of the
array
}
Clever, but if you do this with a big enough number of items, you'll
exhaust all memory. Be careful. :P
Total memory usage is at most 2 times the maximum size the array
reaches during
its lifetime. Test it.
Testing would also reveal that this is a buggy replica of
std.algorithm.remove. At best we should suggest reusing library code
over attempting to reinvent it.
Thanks,
Andrei
Oh, I see.
Wait, what bug are you referring to, though?
I was mistaken and removed my post. The code ingeniously redefines the
problem - instead of removing from the array by shifting its tail
downwards, it shifts elements upwards from the head of the array. A nice
hack, but I don't think it does a lot in helping your situation.
There's a bit of history wrt ~=. Initially the behavior was to never
reallocate if there was enough capacity. But that caused a stomping
issue - people would create an array, pass a slice of it to a function,
and then the function would clobber elements in the arrays outside the
slice. This was a major breakage of modular behavior so we decided to
change the behavior: as long as you keep references to the original
array, appending to a slice of it will cause reallocation of the array.
One thing you could do is to think of what you're trying to achieve on a
larger scale. Assuming a slice to grow over the original array is
arguably a bad behavior that I'm sure could be fixed by small changes in
the design.
Andrei
Re: Arrays are sufficient for ArrayLists? Really??
On 2011-05-16 12:26, Mehrdad wrote:
> On 5/16/2011 12:25 PM, Timon Gehr wrote:
> > Mehrdad wrote:
> >> Timon:
> >>> What about:
> >> void removeAt(T)(ref T[] arr, size_t index)
> >> {
> >>
> >> foreach (i, ref item; arr[1 .. index+1])
> >>
> >> item = arr[i - 1];
> >>
> >> arr = arr[1 .. $]; //note how no valid data is beyond the end of
> >> the array
> >>
> >> }
> >>
> >> Clever, but if you do this with a big enough number of items, you'll
> >> exhaust all memory. Be careful. :P
> >
> > Total memory usage is at most 2 times the maximum size the array reaches
> > during its lifetime. Test it.
>
> Whoops, my bad, I was under the impression that the array could
> potentially grow forever if I added elements, which obviously isn't
> true... yeah, seems like this would work too, interesting! I'll
> definitely test it, thanks!
It'll grow, but if it doesn't have enough space to grow, it has to reallocate,
which means that any other slices/arrays which pointed to any portion of that
array will not point to the new array. And you obviously can't have an array
bigger than the amount of memory that you have, so you definitely can't make
it grow forever. But it would be incredibly rare to have an application which
needed an array larger than you could make.
- Jonathan M Davis
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 12:25 PM, Andrei Alexandrescu wrote:
On 5/16/11 2:25 PM, Timon Gehr wrote:
Mehrdad wrote:
Timon:
What about:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[1 .. index+1])
item = arr[i - 1];
arr = arr[1 .. $]; //note how no valid data is beyond the end of the
array
}
Clever, but if you do this with a big enough number of items, you'll
exhaust all memory. Be careful. :P
Total memory usage is at most 2 times the maximum size the array
reaches during
its lifetime. Test it.
Testing would also reveal that this is a buggy replica of
std.algorithm.remove. At best we should suggest reusing library code
over attempting to reinvent it.
Thanks,
Andrei
Oh, I see.
Wait, what bug are you referring to, though?
On 5/16/2011 12:25 PM, Steven Schveighoffer wrote:
>> Steven:
>>> For the OP, you may want to consider using ArrayList from
>> dcollections, which supports removing an individual or range of
>> elements, or using foreach to purge the list. Note that this class
>> does call assumeSafeAppend on its data storage since it can make
>> more assumptions.
>>
>> Ah, seems to be what I need.
>>
>> I guess I was also trying to argue this should really be part of the
>> language too, though (not just Phobos), given that concatenation and
>> even hashtables are also part of the language. It seems to be
>> missing badly. :\
>
> builtin arrays are full of features that are "good enough" for most
> usages. Removing elements the way that is least confusing is
> inefficient, and removing elements the most efficient way can be
> confusing or incorrect depending on the application. Rather than guess
> what the user expects, the runtime just leaves it up to the
> developer/standard lib.
>
> -Steve
Hm... is it really that atypical, though? We already have addition, is
removal really that unlikely of an operation?
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 15:16:16 -0400, %u wrote:
Timon:
What about:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[1 .. index+1])
item = arr[i - 1];
arr = arr[1 .. $]; //note how no valid data is beyond the end of
the array
}
Clever, but if you do this with a big enough number of items, you'll
exhaust all memory. Be careful. :P
It should be fine, as long as you are using appending to grow the array.
When an append requires a reallocation, it will only reserve enough space
for the valid elements. However, there are certain cases that will make
the array continue to grow in place by adding more pages. But this will
only happen if you get more than half a page of data, and pages are
available to concatenate.
It's actually a pretty clever way to avoid hitting the runtime when you
need to remove an element. As long as it doesn't invalidate other
references to the array elements, you should be good.
Steven:
For the OP, you may want to consider using ArrayList from
dcollections, which supports removing an individual or range of
elements, or using foreach to purge the list. Note that this class
does call assumeSafeAppend on its data storage since it can make
more assumptions.
Ah, seems to be what I need.
I guess I was also trying to argue this should really be part of the
language too, though (not just Phobos), given that concatenation and
even hashtables are also part of the language. It seems to be
missing badly. :\
builtin arrays are full of features that are "good enough" for most
usages. Removing elements the way that is least confusing is inefficient,
and removing elements the most efficient way can be confusing or incorrect
depending on the application. Rather than guess what the user expects,
the runtime just leaves it up to the developer/standard lib.
-Steve
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/2011 12:25 PM, Timon Gehr wrote:
Mehrdad wrote:
Timon:
What about:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[1 .. index+1])
item = arr[i - 1];
arr = arr[1 .. $]; //note how no valid data is beyond the end of the array
}
Clever, but if you do this with a big enough number of items, you'll
exhaust all memory. Be careful. :P
Total memory usage is at most 2 times the maximum size the array reaches during
its lifetime. Test it.
Whoops, my bad, I was under the impression that the array could
potentially grow forever if I added elements, which obviously isn't
true... yeah, seems like this would work too, interesting! I'll
definitely test it, thanks!
Re: reddit discussion about Go turns to D again
On 5/16/2011 11:51 AM, Russel Winder wrote: On Sun, 2011-05-15 at 13:41 -0700, Walter Bright wrote: I like to think of myself as an expert on this . . . you'll have to ask others if I actually am :-) I asked google, and they said 9,750 hits on Russel Winder Data Flow Concurrency !! Is that good or bad ;-) I'd say good, since Russel Winder Talentless Hack only turned up a handful of hits :-)
Re: Arrays are sufficient for ArrayLists? Really??
Mehrdad wrote:
> Timon:
> > What about:
> void removeAt(T)(ref T[] arr, size_t index)
> {
>foreach (i, ref item; arr[1 .. index+1])
> item = arr[i - 1];
> arr = arr[1 .. $]; //note how no valid data is beyond the end of the array
> }
>
> Clever, but if you do this with a big enough number of items, you'll
> exhaust all memory. Be careful. :P
Total memory usage is at most 2 times the maximum size the array reaches during
its lifetime. Test it.
Arguably, this is not a very good guarantee, because you might have multiple
arrays that oscillate from big to small with some offset.
But I was only trying to improve on runtime.
Btw: removing an element always takes linear time. relocating happens only
straight after removal and takes, well, linear time. In an asymptotic view, your
implementation performs optimally, while never using too much memory.
Timon
Re: Arrays are sufficient for ArrayLists? Really??
Steven: > You need to do arr.assumeSafeAppend(); Otherwise, the runtime is no aware of your invalidation of the last element. See my edit on SO: http://stackoverflow.com/q/6015008/541686
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 15:12:17 -0400, Timon Gehr wrote:
As an example, let's say I define:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[index .. $ - 1])
item = arr[i + 1];
arr = arr[0 .. $ - 1];
}
and then I have:
auto arr = [1, 2, 3];
arr.removeAt(0);
arr ~= 4;
The last statement ***WILL*** cause a reallocation, so that doesn't
help.
You need to do arr.assumeSafeAppend(); Otherwise, the runtime is not
aware of your invalidation of the last element.
What about:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[1 .. index+1])
item = arr[i - 1];
arr = arr[1 .. $]; //note how no valid data is beyond the end of the
array
}
I did not test it but in theory this should give you the amortized
constant
guarantee for ~=.
Yes, this should solve the problem, depending on the situation.
Essentially, as long as you are not keeping other references to the array
you expect to be updated properly (this solution "moves" the original
pointer to the right each time), it should work.
-Steve
Re: Arrays are sufficient for ArrayLists? Really??
Timon:
> What about:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[1 .. index+1])
item = arr[i - 1];
arr = arr[1 .. $]; //note how no valid data is beyond the end of
the array
}
Clever, but if you do this with a big enough number of items, you'll
exhaust all memory. Be careful. :P
Steven:
> For the OP, you may want to consider using ArrayList from
dcollections, which supports removing an individual or range of
elements, or using foreach to purge the list. Note that this class
does call assumeSafeAppend on its data storage since it can make
more assumptions.
Ah, seems to be what I need.
I guess I was also trying to argue this should really be part of the
language too, though (not just Phobos), given that concatenation and
even hashtables are also part of the language. It seems to be
missing badly. :\
Re: Arrays are sufficient for ArrayLists? Really??
> 1. Are you sure you need an array rather than a linked list?
>
> Yes. I'm only asking about a functionality already present in every other
language I know (C++'s vector.erase,
> C#'s List.RemoveAt, Java's removeAt) so it's not something out of the blue. ;)
>
> 2. The garbage collector guarantees amortized constant runtime for insertion
> at
the back of the array.
>
> The problem is that that is _only_ true if the array is NOT a slice of another
array!
>
> As an example, let's say I define:
>
> void removeAt(T)(ref T[] arr, size_t index)
> {
>foreach (i, ref item; arr[index .. $ - 1])
> item = arr[i + 1];
> arr = arr[0 .. $ - 1];
> }
>
> and then I have:
>
> auto arr = [1, 2, 3];
> arr.removeAt(0);
> arr ~= 4;
>
> The last statement ***WILL*** cause a reallocation, so that doesn't help.
What about:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[1 .. index+1])
item = arr[i - 1];
arr = arr[1 .. $]; //note how no valid data is beyond the end of the array
}
I did not test it but in theory this should give you the amortized constant
guarantee for ~=.
Timon
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 16 May 2011 14:53:21 -0400, Andrei Alexandrescu wrote: On 5/16/11 1:29 PM, Mehrdad wrote: As has been mentioned, std.algorithm.remove can be of help. You may want to look at three of its capabilities in particular: (a) remove multiple offsets in one pass, e.g. remove(a, 0, 4) removes the first and fifth element, (b) you can remove subranges, e.g. remove(a, tuple(1, 3)) removes the second through fourth element, and (c) if you don't care about the order in which elements are left after removal you may want to look into unstable remove, which does considerably less work. Thanks for the idea. This seems great, except for a couple of things: - I _do_ need the order to stay the same, so I can't just put in the last element. Then you'd need to use the default swapping strategy. - I only need to remove one element at a time. Then you may want to pass only one index. - I still don't understand how this helps. Either this modifies the array directly, in which case adding a new element to the array after a removal would still cause a reallocation every time, or this returns a filtered range, in which case it doesn't do what I need (since it doesn't modify the array directly)... am I missing something? After you remove some elements from an array by using std.algorithm.remove, the array capacity stays the same. Appending to it again will not reallocate the array if the capacity is sufficient. This is not true. In order to adjust the used array length, you must call assumeSafeAppend on the resulting array (which I would recommend you do after using remove in this case). Remove can't do it for you, because it is a generic algorithm which takes a range, it doesn't "know" that it's working with an array. Plus it doesn't know that it's working with a dynamic array. Calling assumeSafeAppend could be a completely useless call. For the OP, you may want to consider using ArrayList from dcollections, which supports removing an individual or range of elements, or using foreach to purge the list. Note that this class does call assumeSafeAppend on its data storage since it can make more assumptions. http://www.dsource.org/projects/dcollections -Steve
Re: Arrays are sufficient for ArrayLists? Really??
Andrei: > After you remove some elements from an array by using std.algorithm.remove, > the array capacity stays the same. Sean: > Removing and then appending one element to an array won't cause any > allocations to occur. How is that possible, though? (See the example in my response to Timon.)
Re: Arrays are sufficient for ArrayLists? Really??
1. Are you sure you need an array rather than a linked list?
Yes. I'm only asking about a functionality already present in every other
language I know (C++'s vector.erase,
C#'s List.RemoveAt, Java's removeAt) so it's not something out of the blue. ;)
2. The garbage collector guarantees amortized constant runtime for insertion at
the back of the array.
The problem is that that is _only_ true if the array is NOT a slice of another
array!
As an example, let's say I define:
void removeAt(T)(ref T[] arr, size_t index)
{
foreach (i, ref item; arr[index .. $ - 1])
item = arr[i + 1];
arr = arr[0 .. $ - 1];
}
and then I have:
auto arr = [1, 2, 3];
arr.removeAt(0);
arr ~= 4;
The last statement ***WILL*** cause a reallocation, so that doesn't help.
Re: Arrays are sufficient for ArrayLists? Really??
You could try checking capacity() and using reserve() in non time-critical code.
Re: Arrays are sufficient for ArrayLists? Really??
On 5/16/11 1:29 PM, Mehrdad wrote: As has been mentioned, std.algorithm.remove can be of help. You may want to look at three of its capabilities in particular: (a) remove multiple offsets in one pass, e.g. remove(a, 0, 4) removes the first and fifth element, (b) you can remove subranges, e.g. remove(a, tuple(1, 3)) removes the second through fourth element, and (c) if you don't care about the order in which elements are left after removal you may want to look into unstable remove, which does considerably less work. Thanks for the idea. This seems great, except for a couple of things: - I _do_ need the order to stay the same, so I can't just put in the last element. Then you'd need to use the default swapping strategy. - I only need to remove one element at a time. Then you may want to pass only one index. - I still don't understand how this helps. Either this modifies the array directly, in which case adding a new element to the array after a removal would still cause a reallocation every time, or this returns a filtered range, in which case it doesn't do what I need (since it doesn't modify the array directly)... am I missing something? After you remove some elements from an array by using std.algorithm.remove, the array capacity stays the same. Appending to it again will not reallocate the array if the capacity is sufficient. It is possible a reallocation does occur even if the capacity would be sufficient, but that's a rare phenomenon caused by implementation paraphernalia. The formal guarantee is that ~= takes amortized constant time. If you need an absolute guarantee that the array stays as initially allocated, you can't use ~=, but you can still use std.algorithm.remove. Andrei
Re: reddit discussion about Go turns to D again
On Sun, 2011-05-15 at 13:41 -0700, Walter Bright wrote: > On 5/15/2011 5:24 AM, Russel Winder wrote: > > On Sun, 2011-05-15 at 05:53 -0400, Gilbert Dawson wrote: > > [ . . . ] > >> You could shed some llight on this if you're an expert. I don't know > >> what's so different between actors and CSP. How does D support dataflow > >> concurrency? > > > > I like to think of myself as an expert on this . . . you'll have to ask > > others if I actually am :-) > > I asked google, and they said 9,750 hits on Russel Winder Data Flow > Concurrency !! Is that good or bad ;-) -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: Arrays are sufficient for ArrayLists? Really??
On Mon, 2011-05-16 at 18:45 +, Timon Gehr wrote: [ . . . ] > > What exactly do you need the data structure for? I have to admit, my reaction was, "and about b~ time someone asked that question". How on earth can one answer a question about data structures without knowing what the use case is? > Timon -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: reddit discussion about Go turns to D again
On 5/16/11 1:00 PM, Sean Kelly wrote: On May 16, 2011, at 10:32 AM, Jonathan M Davis wrote: bearophile wrote: Sean Kelly: std.paralellogrithm ;-) The module name I like more so far is the simple "parallel_algorithm". But I don't mind the "p" prefix for the parallel function names. Bye, bearophile It is also possible to have both "p" prefix and identical names. The latter would be mere aliases and would be activated by a version declaration or similar. I think both possibilities have their merits, so letting the user choose would be an option. parallel_algorithm is to the point, but not very concise. I do not think we can do very much better. That doesn't work. As soon as the non-p versions exist, you have name clashes. So, if the point of starting the function names with p is to avoid name clashes, then you've gained nothing. Besides, we generally try to avoid aliases like that in Phobos. Such a scheme would never be accepted. I don't foresee a simple solution to the 'p' vs. non-'p' problem. It's a complete mess. Indeed it is widely conjectured that p!=np... :o) Andrei
Re: Arrays are sufficient for ArrayLists? Really??
> Thanks for the idea. This seems great, except for a couple of things: > > - I _do_ need the order to stay the same, so I can't just put in the last > element. > > - I only need to remove one element at a time. > > - I still don't understand how this helps. Either this modifies the array directly, in which case > adding a new element to the array after a removal would still cause a reallocation every time, or > this returns a filtered range, in which case it doesn't do what I need (since > it doesn't modify the > array directly)... am I missing something? 1. Are you sure you need an array rather than a linked list? Removal and insertion in the middle of the array is very inefficient if you have many elements. (but you cannot index into a list efficiently, duh) 2. The garbage collector guarantees amortized constant runtime for insertion at the back of the array. What exactly do you need the data structure for? Timon
Re: Arrays are sufficient for ArrayLists? Really??
On May 16, 2011, at 11:29 AM, Mehrdad wrote: > >> As has been mentioned, std.algorithm.remove can be of help. You may want to >> look at three of its > capabilities in particular: (a) remove multiple offsets in one pass, e.g. > remove(a, 0, 4) removes > the first and fifth element, (b) you can remove subranges, e.g. remove(a, > tuple(1, 3)) removes the > second through fourth element, and (c) if you don't care about the order in > which elements are left > after removal you may want to look into unstable remove, which does > considerably less work. > > Thanks for the idea. This seems great, except for a couple of things: > > - I _do_ need the order to stay the same, so I can't just put in the last > element. Thus the stable remove, which doesn't affect element order. > - I still don't understand how this helps. Either this modifies the array > directly, in which case > adding a new element to the array after a removal would still cause a > reallocation every time, or > this returns a filtered range, in which case it doesn't do what I need (since > it doesn't modify the > array directly)... am I missing something? Removing and then appending one element to an array won't cause any allocations to occur. The GC lock may be acquired to determine whether the memory block backing the array has room for the new element, but that's it.
Re: should int/short/byte implicitly cast to dchar/wchar/char?
On Mon, 16 May 2011 13:51:55 -0400, Steven Schveighoffer
wrote:
Currently, this works:
void foo(dchar i)
{
}
void main(string[] args)
{
foo(args.length);
}
Damn, this originally started out as argc and argv, and I forgot how D
accepts arguments, so I switched it to this. Unsigned ints are
convertable to dchar, but signed ones are not (except for a couple cases,
which doesn't make sense).
For example, this fails:
dchar c = -1;
foo(-1);
But this passes:
int i = -1;
dchar c = i;
So clearly there are some inconsistencies that need to be fixed, but the
situation is not as bad as I thought it was.
-Steve
Re: Arrays are sufficient for ArrayLists? Really??
> They're called Arrays, not Lists (or ArrayLists), so way would you expect a > delete functions? I thought they're supposed to substitute for lists (otherwise we'd have an ArrayList type in Phobos). > If you want something like an ArrayList in D, have a look at > std.container.Array :-) That doesn't work well the the garbage collector. :\ > As has been mentioned, std.algorithm.remove can be of help. You may want to > look at three of its capabilities in particular: (a) remove multiple offsets in one pass, e.g. remove(a, 0, 4) removes the first and fifth element, (b) you can remove subranges, e.g. remove(a, tuple(1, 3)) removes the second through fourth element, and (c) if you don't care about the order in which elements are left after removal you may want to look into unstable remove, which does considerably less work. Thanks for the idea. This seems great, except for a couple of things: - I _do_ need the order to stay the same, so I can't just put in the last element. - I only need to remove one element at a time. - I still don't understand how this helps. Either this modifies the array directly, in which case adding a new element to the array after a removal would still cause a reallocation every time, or this returns a filtered range, in which case it doesn't do what I need (since it doesn't modify the array directly)... am I missing something?
Re: reddit discussion about Go turns to D again
*actually a compile-time argument would be a bad idea, it's much more useful to know the core count at runtime, doh.
Re: reddit discussion about Go turns to D again
Could templates in std.algorithm be expanded to have an optional
thread-count compile-time argument? Maybe they'd be used like so:
import std.stdio;
import std.array;
import std.range;
import std.functional;
enum Threads
{
x1 = 1,
x2 = 2,
x3 = 3,
x4 = 4, // etc..
}
void main()
{
int[] data = [1, 2, 3];
count!("a < 0", Threads.x4)(data);
}
size_t count(alias pred = "true", alias th = Threads.x1, Range)(Range
r) if (isInputRange!(Range))
{
static if (th == Threads.x1)
{
// call normal count
}
else
{
// call parallel count
}
return -1;
}
Re: reddit discussion about Go turns to D again
> On May 16, 2011, at 10:32 AM, Jonathan M Davis wrote: > >> bearophile wrote: > >>> Sean Kelly: > std.paralellogrithm ;-) > >>> > >>> The module name I like more so far is the simple "parallel_algorithm". > >>> But I don't mind the "p" prefix for the parallel function names. > >>> > >>> Bye, > >>> bearophile > >> > >> It is also possible to have both "p" prefix and identical names. The > >> latter would be mere aliases and would be activated by a version > >> declaration or similar. I think both possibilities have their merits, > >> so letting the user choose would be an option. > >> > >> parallel_algorithm is to the point, but not very concise. I do not think > >> we can do very much better. > > > > That doesn't work. As soon as the non-p versions exist, you have name > > clashes. So, if the point of starting the function names with p is to > > avoid name clashes, then you've gained nothing. Besides, we generally > > try to avoid aliases like that in Phobos. Such a scheme would never be > > accepted. > > I don't foresee a simple solution to the 'p' vs. non-'p' problem. It's a > complete mess. At least it's not p vs np. ;) - Jonathan M Davis
Re: Making D Lang More easy.
On Mon, 2011-05-16 at 13:32 -0400, bearophile wrote: > Russel Winder: > > > If you want to branch the branch using Bazaar then use the URL > > http://www.russel.org.uk/Bazaar/Pi_Quadrature. If you want to browse > > the code then there is an instance of Loggerhead running so point your > > browser at the URL http://www.russel.org.uk:8080/Pi_Quadrature > > There's even a Chapel version :-) I like several ideas of the Chapel language > design. > Where are the results of the benchmarks? Chapel is very good if you like PGAS (which I am not sure I do), it is also quite efficient in use of resources (kudos to Brad et al.). X10 is profligate with resources so is definitely not on my "like" list just now (apologies to Igor and the rest of the X10 team, but . . . ). Fortress is interesting as it goes for the "you write mathematics and we'll code generate" approach, which is very interesting -- assuming Oracle continue to fund the activity. There are no published results from my codes, it is all very much "do it yourself" as far as numbers are concerned. The codes are all about anecdotal evidence not about structured statistical results. Yes structured statistical results could be created, but is it worth it? -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: reddit discussion about Go turns to D again
On May 16, 2011, at 10:32 AM, Jonathan M Davis wrote: >> bearophile wrote: >>> Sean Kelly: std.paralellogrithm ;-) >>> >>> The module name I like more so far is the simple "parallel_algorithm". >>> But I don't mind the "p" prefix for the parallel function names. >>> >>> Bye, >>> bearophile >> >> It is also possible to have both "p" prefix and identical names. The latter >> would be mere aliases and would be activated by a version declaration or >> similar. I think both possibilities have their merits, so letting the user >> choose would be an option. >> >> parallel_algorithm is to the point, but not very concise. I do not think we >> can do very much better. > > That doesn't work. As soon as the non-p versions exist, you have name > clashes. > So, if the point of starting the function names with p is to avoid name > clashes, then you've gained nothing. Besides, we generally try to avoid > aliases like that in Phobos. Such a scheme would never be accepted. I don't foresee a simple solution to the 'p' vs. non-'p' problem. It's a complete mess.
should int/short/byte implicitly cast to dchar/wchar/char?
Currently, this works:
void foo(dchar i)
{
}
void main(string[] args)
{
foo(args.length);
}
Does this make any sense? When is it useful to be able to call a
dchar-accepting function with a random integer?
I would say dchar should be settable from an (unsigned) integer literal
(for those cases where you want to use a specific bit pattern), but from
any integer or unsigned integer?
Same goes for short to wchar and byte to char, and the unsigned
equivalents.
What do you think?
There is a bug report related to this:
http://d.puremagic.com/issues/show_bug.cgi?id=5995
-Steve
Re: reddit discussion about Go turns to D again
Jonathan M Davis wrote: > > bearophile wrote: > > > Sean Kelly: > > > > std.paralellogrithm ;-) > > > > > > The module name I like more so far is the simple "parallel_algorithm". > > > But I don't mind the "p" prefix for the parallel function names. > > > > > > Bye, > > > bearophile > > > > It is also possible to have both "p" prefix and identical names. The latter > > would be mere aliases and would be activated by a version declaration or > > similar. I think both possibilities have their merits, so letting the user > > choose would be an option. > > > > parallel_algorithm is to the point, but not very concise. I do not think we > > can do very much better. > > That doesn't work. As soon as the non-p versions exist, you have name clashes. > So, if the point of starting the function names with p is to avoid name > clashes, then you've gained nothing. Besides, we generally try to avoid > aliases like that in Phobos. Such a scheme would never be accepted. > > - Jonathan M Davis I think the point of having the same names is that you can parallelize code easily that was sequential before (or even for mere cosmetic reasons, by substituting the import, so no name clashes) But if there is only one option, then the "p" prefix is as far as I can see to be preferred, users can always do the aliases manually. Timon
Re: reddit discussion about Go turns to D again
> bearophile wrote: > > Sean Kelly: > > > std.paralellogrithm ;-) > > > > The module name I like more so far is the simple "parallel_algorithm". > > But I don't mind the "p" prefix for the parallel function names. > > > > Bye, > > bearophile > > It is also possible to have both "p" prefix and identical names. The latter > would be mere aliases and would be activated by a version declaration or > similar. I think both possibilities have their merits, so letting the user > choose would be an option. > > parallel_algorithm is to the point, but not very concise. I do not think we > can do very much better. That doesn't work. As soon as the non-p versions exist, you have name clashes. So, if the point of starting the function names with p is to avoid name clashes, then you've gained nothing. Besides, we generally try to avoid aliases like that in Phobos. Such a scheme would never be accepted. - Jonathan M Davis
Re: Making D Lang More easy.
Russel Winder: > If you want to branch the branch using Bazaar then use the URL > http://www.russel.org.uk/Bazaar/Pi_Quadrature. If you want to browse > the code then there is an instance of Loggerhead running so point your > browser at the URL http://www.russel.org.uk:8080/Pi_Quadrature There's even a Chapel version :-) I like several ideas of the Chapel language design. Where are the results of the benchmarks? Bye, bearophile
Re: reddit discussion about Go turns to D again
bearophile wrote: > > Sean Kelly: > > > std.paralellogrithm ;-) > > The module name I like more so far is the simple "parallel_algorithm". But > I don't mind the "p" prefix for the parallel function names. > > Bye, > bearophile It is also possible to have both "p" prefix and identical names. The latter would be mere aliases and would be activated by a version declaration or similar. I think both possibilities have their merits, so letting the user choose would be an option. parallel_algorithm is to the point, but not very concise. I do not think we can do very much better. Timon
Re: reddit discussion about Go turns to D again
Sean Kelly: > std.paralellogrithm ;-) The module name I like more so far is the simple "parallel_algorithm". But I don't mind the "p" prefix for the parallel function names. Bye, bearophile
Re: reddit discussion about Go turns to D again
Matthew Ong wrote: > Hopefully, does D currently support such ability? [go interface] You can do it with templates. Most of Phobos' range functions are written in a similar style.
Re: Making D Lang More easy.
On Mon, 2011-05-16 at 22:54 +0800, Matthew Ong wrote: > On 5/12/2011 7:19 PM, Russel Winder wrote: > Hi Russel, > I looked into this URL, > http://www.russel.org.uk/Bazaar/ > > > I put my little experimental codes in microbenchmarking in Bazaar > > branches accessible on my web site. Note this is microbenchmarking with > > all the hassles that go with it. Especially when using JITs. > > Not quite sure which one you are refering to. Apologies I underspecified. The one you are probably looking for is Pi_Quadrature. This is a simple embarrasingly parallel calculation (of the value of Pi using a straightforward quadrature. This allows for simple comparison of data parallelism and the various message passing and explicit thread usage systems. It is not really intended for serious benchmarking, it is more a "toy" problem for showing all the issues -- include JVM JIT startup issues. If you want to branch the branch using Bazaar then use the URL http://www.russel.org.uk/Bazaar/Pi_Quadrature. If you want to browse the code then there is an instance of Loggerhead running so point your browser at the URL http://www.russel.org.uk:8080/Pi_Quadrature Hopefully that is a better explanation :-) -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@russel.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: reddit discussion about Go turns to D again
std.paralellogrithm ;-) Sent from my iPhone On May 15, 2011, at 8:01 AM, bearophile wrote: > Andrei: > >> (That's also why we should think >> of a shorter name instead of parallel_algorithm... > > The idea of adding parallel algorithms to Phobos is good, people may use them > more than std.algorithm. > Regarding the module name, std.palgorithm? :-) > > Bye, > bearophile
Re: Making D Lang More easy.
On 5/12/2011 7:19 PM, Russel Winder wrote: Hi Russel, I looked into this URL, http://www.russel.org.uk/Bazaar/ I put my little experimental codes in microbenchmarking in Bazaar branches accessible on my web site. Note this is microbenchmarking with all the hassles that go with it. Especially when using JITs. Not quite sure which one you are refering to. -- Matthew Ong email: on...@yahoo.com
Some features should NOT include from GO/Java.
Hi All,
The reason I am starting this thread is to gather some valid/experience that
people do not like about using Go or even Java. Naturally D-programming might
not wants to repeat the same error.
Java:
1) Swing API --- The inheritance tree is too deep. New OO encourages
flatten object (1-2 level inheritance) for easy memory management from the
JVM/GC.
2) final --- does not protect the content of the LinkedList/Hashmap.
I think immutable keywords managed that? Informed me if I am wrong.
3) Do not have keywords to allow conditional compilation like version in D.
4) Threads are rather heavy and synchronization, that is why people introduced
kilim. Something like the fiber-thread concept.
5) No marco-preprocessor like in C #define, when the limitations of the
syntax is met. Sometime, that allows
6) No void* to allow passing of function pointer into any method to be
called. That can be tricked:
by using
interface F1{
R func1(P p1);
}
7) No closure...Or lamda J.
8) No clear IPC like COM+, DCOM. Need to do networks sockets with full stack
overheads.
9) GUI inter-component messaging is complex and can only do share memory.
I hope that is defined with D. Yes, there MVC, but how to inter-comm
without exposing too much info, there should be a way to do in-memory JMS
model.
GO:
1) No while/do-while/foreach... ONLY for loop.
2) Only Public/private scope... That claim is semi-true.
It allowed to have method introduction to a library defined type. And
allows user of that type to use the new method without knowing.
3) there is no auto promotion of data type even when it is safe.
int does not go into long automatically. byte to int...etc (Aiks!!!)
Both,
There is no way to have a centralized Exception handling routine that
allow routing of error message & filtering for both language like in the JSP.
JSP.
Over all, those are modeling issues that I do see.
Matthew Ong
on...@yahoo.com
Re: Flagshit concurrency
On 15/05/11 19.49, Robert Clipsham wrote: On 15/05/2011 10:08, Gilbert Dawson wrote: Hello I'm new to D. Welcome :) I've been studying new languages for new concurrency ideas. I'd like to implement a small and lightweight Lighttpd replacement and replace the scripting with MiniD. One has heard in many places that D supports a new idea of "flagship message passing". You've hit your first roadblock here - MiniD is written in D1/Tango, the concurrency model you speak of is D2 only (not that it couldn't be implemented in D1). Your options are then - port MiniD to D2 - that will take a while, it's heavily dependent on Tango, which is D1 only (currently, D2 support is underway it seems); Use another scripting language that has D bindings; Use D - dmd is fast enough that you could in fact use D as a scripting language. So I'd like to use this with async i/o and one threaded server process. Is it suitable for this? I believe someone is working on async i/o as a GSoC project. I don't believe support for it exists in Phobos currently - I could be wrong. Again, you could use bindings to C for this. Noone is working on the async I/O project for GSoC unfortunately since we did not get enough GSoC slots assigned. Message passing is mostly used for multithreaded use. Since your app is single threaded I do not believe message passing is the way to go. A pure async reactor based design would probably give the best performance. /Jonas
Re: reddit discussion about Go turns to D again
On 5/16/2011 12:06 PM, Andrei Alexandrescu wrote: On 05/15/2011 10:04 PM, Daniel Gibson wrote: So you have to write std.parallel_algorithm.map() instead of map() all the time? alias std.parallel_algorithm p; Andrei Or this, which I prefer to alias: import p = std.parallel_algorithm;
Re: Builtin regex (Was: How to complex switch?)
Hi, Seems like some people also had the same idea about D supporting more complex switch syntax and given more interesting reasons. Perhaps can be consider for D 3.0 or ... I am new here, please understand, but wish to see D take off because I do see some nice syntax in D.
Re: reddit discussion about Go turns to D again
Hi Walter, alias is indeed a very very useful feature in D. It make string easy and well define. Just need some Auto Documentation search-able tool for new developer to help them find. As for now, I am using grepwin to help figure things out. Matthew Ong
Re: Arrays are sufficient for ArrayLists? Really??
On 05/16/2011 04:54 AM, Mehrdad wrote: Hi! I posted this question on StackOverflow about D: http://stackoverflow.com/questions/6015008/how-to-delete-an-element- from-an-array-in-d and the answers are quite surprising to me. Are arrays really supposed to be substitutes for array lists if we can't remove anything from them? It seems to me like that's a really basic feature we're missing here... it's really annoying whenever I find out that each of my arrays needs its own "counter" variable, even though it already has a length and a capacity. Doesn't that mean we still need an ArrayList(T) type, as powerful as arrays are supposed to be in D? As has been mentioned, std.algorithm.remove can be of help. You may want to look at three of its capabilities in particular: (a) remove multiple offsets in one pass, e.g. remove(a, 0, 4) removes the first and fifth element, (b) you can remove subranges, e.g. remove(a, tuple(1, 3)) removes the second through fourth element, and (c) if you don't care about the order in which elements are left after removal you may want to look into unstable remove, which does considerably less work. Andrei
Re: reddit discussion about Go turns to D again
Hi,
Oh a few more thing that got my interest is how the Go model their data. Not
entirely like how other conventional OO does it. They do NOT
have Object inheritance. But uses interface to some how 'bypass' that.
http://golang.org/doc/effective_go.html
Conversions
...
// this function is now available for the object type Sequence.
func (s Sequence) MyFunction() string {
// My Function has access to all the public/internal data of the
// Sequence here.
}
...
s Sequence;
s.MyFunction(); // now that can be used.
That is some how like JRuby ability to 'add' method into the java final String
class without really touching that API.
Generality
It also >avoids< the need to repeat the documentation on every instance of a
common method.
Interfaces and methods
Since almost anything can have methods attached, almost anything can satisfy an
interface. One illustrative example is in the http package, which defines the
Handler interface. Any object that implements Handler can serve HTTP requests.
It is not just entirely like java interfaces, but java has to somehow, code the
api in a very strange pattern to support this.
Hopefully, does D currently support such ability?
Matthew Ong
on...@yahoo.com
Re: AA Troubles
On Tue, 10 May 2011 21:54:30 -0400, Jonathan Crapuchettes wrote: Hey all, I have been working with a lot of associative arrays of late and been running into some problems with the built-in implementation. It appears that very heavy use in an application can cause the garbage collector to have issues. Most of the time I have found ways around the problems, but the one that I ran into today has made me wonder if there is a less error prone/slower implementation of AAs out there for D that doesn't pound the GC as hard. I have looked at the aa project at dsource and noticed some discussion on the forums from a little while ago, but I wasn't sure how up-to-date everything was. I realize this is very late response, but you can try an alternative map type in dcollections. There are both HashMap and TreeMap. Please use the latest d2 branch, as it has recently been updated to work with 64-bit linux (need to do a beta release). http://www.dsource.org/projects/dcollections In most cases, the HashMap actually is faster than the builtin AA. -Steve
Re: struct destructors
On Mon, 09 May 2011 17:04:49 -0400, Sean Kelly wrote: On May 9, 2011, at 12:51 PM, Alexander wrote: On 09.05.2011 19:38, Sean Kelly wrote: Not currently. I thought I wrote some explanation of why... OK, thanks - I've read the ticket. Though, the problem can be solved relatively simple, IMHO - adding one more pointer to BlkInfo, where address of the finalizer is stored, which will be used only for cases when collected object is not a class. Well... BlkInfo is generated from a selection of internal sources. You're right that the GC would need to store a finalizer reference for structs, but that's just the simple case. It would also need to finalize structs in dynamic arrays and AA nodes as well (though it's possible the latter already works, since the AA is now template code). And of course the finalizer reference would have to be passed to GC.malloc() or whatever so that portion of the API would change as well. I originally held off on fixing this because its requirements seemed to fit nicely with the requirements for precise scanning (ie. passing the GC some TypeInfo so it can be smarter about how it deals with allocated memory blocks in general), and I wanted to do both at once. But I have a feeling that I came up with some other problem regarding struct finalization, so I want to see if I can find that old email or whatever to be sure. A few things here: 1. the reason why a struct is not finalized when allocated on the heap is because a struct does not have a vtable pointer (which is how the GC is able to find the dtor for a class). 2. A struct allocated on the heap is actually transformed by the compiler into an allocation of an array of one struct instance. So essentially there is one problem to solve -- how to finalize an array of structs. 3. AA's are templated, but the templated functions simply forward to the runtime functions. The template allows us to add extra functionality to an AA without modifying the compiler, but everything is very much done by druntime. However, I think we would have the ability to add this without modifying the compiler. -Steve
Re: Implementing std.log
On 14/05/11 19.04, Andrei Alexandrescu wrote: On 5/9/11 1:52 AM, Andrei Alexandrescu wrote: [snip] I updated my std.log draft. Added a lot of features including formatted writing, delayed logging, and a variety of configuration options. Replaced the redundant log.xyz with logXyz. The implementation is getting close to reviewable form. Documentation: http://d-programming-language.org/phobos-prerelease/std_log.html Source: https://github.com/andralex/phobos Feedback welcome. I like it! Some minor suggestions: The command line flags seem inconsistent in their naming e.g --minloglevel vs. --log_dir (the latter using underscores). Also single letter flags are usually written with a single '-' in front instead of '--' at least on posix platforms. /Jonas
Re: reddit discussion about Go turns to D again
On 5/15/2011 5:56 PM, Timon Gehr wrote: You are to a certain extent right, but Go is appealing in a few ways. Many Go users are coming from C or scripting languages, so Go is an evolution for them, even if the language is a downgrade from major programming language features. Then many of the developers that are impressed by Go's multicore features, are not aware of the nice libraries available for C++, JVM or .Net. That is because of the goroutine and channel syntax. I can emulate some of the channel syntax using my own wrapper class for from NIO piped. But the goroutine part is more like java kilim(but without) the nasty bytecode postprocessor (a "weaver"), http://www.malhar.net/sriram/kilim/ Perhaps D can approach this person to make things as interesting but keep the dmd process simple like javac. There is the possibility that Go will make it into Android. The web site is always up to date with the latest language specification and they have weekly and stable releases. There not much to say about Go, other than the language looks like a new version of Alef from Plan9 with a bit of Oberon. But Google's backing, plus the way they deal with the community is increasing its use. I wish D would evolve the same way. -- Paulo I think D has difficulties getting new users, although it is superior to any programming language I know in almost every way. Yes. D has far more syntax and well above Java & I think even C#. Probably the main 'show brakes' for D are: 1. Lack of documentation. The documentation we have on digitalmars.com/d/2.0 is sufficient for me, but it is not up-to-date and it is too complicated for a newcomer to get started with. I think many will be turned off by the fact that there is no tutorial for newcomers on the main site, but you can get all details about some old version of the D grammar. It also stops D from becoming a teaching language at institutions. Apart from that, the website does not look half as professional as D is well designed. It is not structured at all. If you want to learn about what D is about, you have to read the whole website. Also, the documentation comments in some Phobos modules should improve, regardless of their formatting. Yes. This part I agrees. As I am a new comer. It seems to me that I need to go all over the places within the wiki to figure things out. Go build a tool to do that automatically. http://golang.org/cmd/godoc/ With the -http flag, it runs as a web server and presents the documentation as a web page. godoc -http=:6060 From the browser, you can view the entire build in API. D can also do that without a build in server, but the navigation is not as organized as javadoc api format. 2. Someone who is curious about D will google 'd', which takes them straight to http://www.digitalmars.com/d/. They will then press the back button on their browser, because it does not look appealing. Then they will find the link to Wikipedia. If they are really curious, they will read the whole thing, to learn that they really can get the official compiler from digitalmars.com they will also see very little of D but read everything about "Problems and Controversies". The Wikipedia article, in my eyes, fails to give sufficient information about what D is about. It only lists features and gives code samples. They will then go back to http://www.digitalmars.com/d/ , where the most important link is not only the most important, but well, the smallest in size as well: 2.0 After clicking it, you they have to scroll down to find a link to the download site, where they need to read the whole table, because there are no OS-symbols leading them to the one-click-install. Many will just download the first thing and end up with the source and some binaries. (I have seen it happen multiple times!) A better process would be: google 'd', get to a totally beautiful website, have some display of D philosophy, a big section DOWNLOAD DMD D COMPILER that cannot be missed, beneath it there are symbols representing different OS's that can be clicked to get the appropriate installer and done. The next thing on the site should be a big link D TUTORIAL, linking to a very well written tutorial. This needs fixing, badly. But it is much work... 3. The reference compiler is somewhat buggy. But after seeing the changelog for 2.053 I am optimistic this will change very soon. Timon -- Matthew Ong email: on...@yahoo.com
