Hacker News Poll
There is a poll about programming languages on Hacker News. Of course, it is totally bogus, but it might be a nice opportunity to get people talking about D. ;) https://news.ycombinator.com/item?id=6527104
Re: Arch Linux D news digest
Good Job! What's about gtkd, qtd? I'd love to see them in the official repositories :-) On Wednesday, 2 October 2013 at 15:14:08 UTC, Dicebot wrote: Small Archy update: 1) dub has been just adopted into [community] 2) all three compiler phobos versions now provide 'd-runtime` and `d-stdlib` meta-dependencies
Re: Arch Linux D news digest
On Thursday, 10 October 2013 at 18:43:32 UTC, hsul wrote: Good Job! What's about gtkd, qtd? I'd love to see them in the official repositories :-) Arch Linux policies prohibit pure source packages (there are always exception but it is not the case). I tend to agree, such stuff makes much more sense in dub registry.
Facebook is using D in production starting today
Today I committed the first 5112 lines of D code to Facebook's repository. The project is in heavy daily use at Facebook. Compared to the original version (written in C++) we've measured massive wins in all of source code size, build speed, and running speed. In all likelihood we'll follow up with a blog post describing the process. Andrei
Re: Facebook is using D in production starting today
On 10/10/2013 08:36 PM, Andrei Alexandrescu wrote: Today I committed the first 5112 lines of D code to Facebook's repository. The project is in heavy daily use at Facebook. Compared to the original version (written in C++) we've measured massive wins in all of source code size, build speed, and running speed. In all likelihood we'll follow up with a blog post describing the process. Andrei Congratulations! This is fantastic, I was wondering when there would be some official announcement, after your AMA. I look forward to reading the blog post. -- Matt Soucy http://msoucy.me/ signature.asc Description: OpenPGP digital signature
Re: Facebook is using D in production starting today
On Friday, 11 October 2013 at 00:36:12 UTC, Andrei Alexandrescu wrote: Today I committed the first 5112 lines of D code to Facebook's repository. The project is in heavy daily use at Facebook. Compared to the original version (written in C++) we've measured massive wins in all of source code size, build speed, and running speed. In all likelihood we'll follow up with a blog post describing the process. Andrei Pretty cool. Look forward to the blog post. It looks like the call to arms you made at DConf this year to focus on quality and professionalism are paying off. In particular, I've noticed the speed with which bugs are being handled has improved recently. Lots of good stuff. Congratulations on your work. Joseph
Re: Facebook is using D in production starting today
On Friday, 11 October 2013 at 00:36:12 UTC, Andrei Alexandrescu wrote: Today I committed the first 5112 lines of D code to Facebook's repository. The project is in heavy daily use at Facebook. Compared to the original version (written in C++) we've measured massive wins in all of source code size, build speed, and running speed. In all likelihood we'll follow up with a blog post describing the process. Congratulations! This is wonderful news.
Re: Facebook is using D in production starting today
On Thu, 10 Oct 2013 17:36:17 -0700 Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Today I committed the first 5112 lines of D code to Facebook's repository. The project is in heavy daily use at Facebook. Compared to the original version (written in C++) we've measured massive wins in all of source code size, build speed, and running speed. In all likelihood we'll follow up with a blog post describing the process. Andrei Awesome! Great bragging rights for D :)
Re: Facebook is using D in production starting today
On 10/10/2013 10:05 PM, Nick Sabalausky wrote: Awesome! Great bragging rights for D :) It's the first battle signaling the end of Middle Earth, and the rise of the Age of D. The old guard will be sailing to the Grey Havens soon.
Re: Facebook is using D in production starting today
Congratulations!!
Re: Facebook is using D in production starting today
On Friday, 11 October 2013 at 00:36:12 UTC, Andrei Alexandrescu wrote: Today I committed the first 5112 lines of D code to Facebook's repository. The project is in heavy daily use at Facebook. Compared to the original version (written in C++) we've measured massive wins in all of source code size, build speed, and running speed. In all likelihood we'll follow up with a blog post describing the process. Andrei This is fantastic news! And also the time when having an upvote button would be awesome. They can be placed here though: http://www.reddit.com/r/d_language/comments/1o6p55/first_d_language_commit_at_facebook/
Re: Facebook is using D in production starting today
On Thu, 10 Oct 2013 17:36:17 -0700, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: Today I committed the first 5112 lines of D code to Facebook's repository. The project is in heavy daily use at Facebook. Compared to the original version (written in C++) we've measured massive wins in all of source code size, build speed, and running speed. In all likelihood we'll follow up with a blog post describing the process. Andrei It's been a long road getting here. But congratulations to the D community and Facebook. Looking forward to the blog post. :-) -- Adam Wilson IRC: LightBender Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: Facebook is using D in production starting today
On Friday, 11 October 2013 at 05:11:49 UTC, Walter Bright wrote: On 10/10/2013 10:05 PM, Nick Sabalausky wrote: Awesome! Great bragging rights for D :) It's the first battle signaling the end of Middle Earth, and the rise of the Age of D. The old guard will be sailing to the Grey Havens soon. ;-P -- Paolo
Re: The no gc crowd
On 2013-10-10 02:22, Sean Kelly wrote: Only that this would have to be communicated to the user, since moving data later is problematic. Today, I think it's common to construct an object as unshared and then cast it. What is the reason to not create it as shared in the first place? -- /Jacob Carlborg
Re: The no gc crowd
On 2013-10-10 05:55, Jonathan M Davis wrote: That depends. It works with objects I think (for both shared and immutable), but you definitely have to cast to immutable if you want an immutable array or AA. Also, casting _away_ shared is going to be a very common operation due to how shared works. In order to use shared, you basically have to protect the variable with a mutex or synchronized block, cast away shared, and then use it as thread-local for whatever you're doing until you release the lock (in which case, you have to be sure that there are no more thread-local references to the shared object). As such, while it might be possible to construct stuff directly as shared, it's going to have to be cast to thread-local just to use it in any meaningful way. So, at this point, I don't think that it even vaguely flies to try and make it so that casting away shared is something that isn't typically done. It's going to be done about as often as shared is used for anything other than very basic stuff. What's the reason for casting away shared, is it to pass it to a function that doesn't accept shared? The it should be safe if you synchronize around the call? But that function could put away, the now , unshared data, which is actually shared and cause problem? -- /Jacob Carlborg
Re: The no gc crowd
On 2013-10-10 06:24, Jonathan M Davis wrote: And given that std.concurrency requires casting to and from shared or immutable in order to pass objects across threads, it seems ilke most of D's concurrency model requires casting to and/or from shared or immutable. The major exception is structs or classes which are shared or synchronized rather than a normal object which is used as shared, and I suspect that that's done fairly rarely at this point. In fact, it seems like the most common solution is to ignore shared altogether and use __gshared, which is far worse than casting to and from shared IMHO. Isn't the whole point of std.concurrency that is should only accept shared for reference types? If you want to use std.concurrency create a shared object in the first place? So, it's my impression that being able to consider casting to or from shared as abnormal in code which uses shared is a bit of a pipe dream at this point. The current language design pretty much requires casting when doing much of anything with concurrency. There must be a better way to solve this. -- /Jacob Carlborg
Re: draft proposal for ref counting in D
On 2013-10-10 04:35, Walter Bright wrote: Steven Schveighoffer wrote: On Jul 1, 2013, at 12:17 PM, Walter Bright wrote: I really urge you to make this a separate project. It's not trivial. Logically, it's sound, but the implementation will be very difficult. I also think Sean (and probably others) should be involved for that discussion. Make what a separate project? The destruction of objects by the GC in local threads? It already is not part of the ref counting proposal. As far as I can tell, the ref counting proposal is not viable without it, as long as you insist on non-atomic RC increments and decrements. How can it possibly not be a prerequisite to this, and therefore part of the proposal? Unless you are saying now that atomic ref counting is OK? I'm going by your previous statement: I very much want to avoid requiring atomic counts - it's a major performance penalty. -Steve Is this the last email in the conversation? In that case I think you clearly mark that with a post. -- /Jacob Carlborg
Re: The no gc crowd
On 10/9/2013 11:34 PM, Jacob Carlborg wrote: On 2013-10-10 02:22, Sean Kelly wrote: Only that this would have to be communicated to the user, since moving data later is problematic. Today, I think it's common to construct an object as unshared and then cast it. What is the reason to not create it as shared in the first place? 1. Shared data cannot be passed to regular functions. 2. Functions that create data structures would have to know in advance that they'll be creating a shared object. I'm not so sure this would not be an invasive change. 3. Immutable data is implicitly shared. But it is not created immutable - it is created as mutable data, then set to some state, then cast to immutable.
Re: The no gc crowd
On Thursday, October 10, 2013 08:38:31 Jacob Carlborg wrote:
On 2013-10-10 05:55, Jonathan M Davis wrote:
That depends. It works with objects I think (for both shared and
immutable), but you definitely have to cast to immutable if you want an
immutable array or AA.
Also, casting _away_ shared is going to be a very common operation due to
how shared works. In order to use shared, you basically have to protect
the variable with a mutex or synchronized block, cast away shared, and
then use it as thread-local for whatever you're doing until you release
the lock (in which case, you have to be sure that there are no more
thread-local references to the shared object). As such, while it might be
possible to construct stuff directly as shared, it's going to have to be
cast to thread-local just to use it in any meaningful way. So, at this
point, I don't think that it even vaguely flies to try and make it so
that casting away shared is something that isn't typically done. It's
going to be done about as often as shared is used for anything other than
very basic stuff.
What's the reason for casting away shared, is it to pass it to a
function that doesn't accept shared? The it should be safe if you
synchronize around the call? But that function could put away, the now ,
unshared data, which is actually shared and cause problem?
Pretty much nothing accepts shared. At best, templated functions accept
shared. Certainly, shared doesn't work at all with classes and structs unless
the type is specifically intended to be used as shared, because you have to
mark all of its member functions shared to be able to call them. And if you
want to use that class or struct as both shared and unshared, you have to
duplicate all of its member functions.
That being the case, the only way in general to use a shared object is to
protect it with a lock, cast it to thread-local (so that it can actually use
its member functions or be passed to other functions to be used), and then use
it. e.g.
synchronized
{
auto tl = cast(T)mySharedT;
auto result = tl.foo();
auto result2 = bar(tl);
}
Obviously, you then have to make sure that there are no thread-local
references to the shared object when the lock is released, but without casting
away shared like that, you can't do much of anything with it. So, similar to
when you cast away const, it's up to you to guarantee that the code doesn't
violate the type system's guarantees - i.e. that a thread-local variable is
not accessed by multiple threads. So, you use a lock of some kind to protect
the shared variable while it's treated as a thread-local variable in order to
ensure that that guarantee holds. Like with casting away const or with
@trusted, there's obviously risk in doing this, but there's really no other
way to use shared at this point - certainly not without it being incredibly
invasive to your code and forcing code duplication.
- Jonathan M Davis
Re: The no gc crowd
On Thursday, October 10, 2013 08:41:19 Jacob Carlborg wrote: On 2013-10-10 06:24, Jonathan M Davis wrote: And given that std.concurrency requires casting to and from shared or immutable in order to pass objects across threads, it seems ilke most of D's concurrency model requires casting to and/or from shared or immutable. The major exception is structs or classes which are shared or synchronized rather than a normal object which is used as shared, and I suspect that that's done fairly rarely at this point. In fact, it seems like the most common solution is to ignore shared altogether and use __gshared, which is far worse than casting to and from shared IMHO. Isn't the whole point of std.concurrency that is should only accept shared for reference types? If you want to use std.concurrency create a shared object in the first place? You might do that if you're creating the object simply to send it across, but it's frequently the case that the object was created well before it was sent across, and it frequently had to have operations done it other than simply creating it (which wouldn't work if it were shared). So, it often wouldn't make sense for the object being passed to be shared except when being passed. And once it's been passed, it's rarely the case that you want it to be shared. You're usually passing ownership. You're essentially taking a thread-local variable from one thread and making it a thread-local variable on another thread. Unfortunately, the type system does not support the concept of thread ownership (beyond thread-local vs shared), so it's up to the programmer to make sure that no references to the object are kept on the original thread, but there's really no way around that unless you're always creating a new object when you pass it across, which would result in which would usually be a unnecessary copy. So, it becomes like @trusted in that sense. So, it's my impression that being able to consider casting to or from shared as abnormal in code which uses shared is a bit of a pipe dream at this point. The current language design pretty much requires casting when doing much of anything with concurrency. There must be a better way to solve this. I honestly don't think we can solve it a different way without completely redesigning shared. shared is specifically designed such that you have to either cast it way to do anything with it or write all of your code to explicitly work with shared, which is not something that generally makes sense to do unless you're creating a type whose only value is in being shared across threads. Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting. - Jonathan M Davis
Re: The no gc crowd
On 10/9/2013 9:45 PM, Manu wrote: The are a few problems with mangling the type; I don't understand that. It breaks when you need to interact with libraries. That's true if the library persists copies of the data. But I think it's doable if the library API is stateless, i.e. 'pure'. It's incompatible with struct alignment, and changes the struct size. These are very carefully managed properties of structures. Nobody says there can be only one variant of RefCounted. It obscures/complicates generic code. It seems to not be a problem in C++ with shared_ptrT. It doesn't deal with circular references, which people keep bringing up as a very important problem. ARC doesn't deal with it automatically, either, it requires the user to insert weak pointers at the right places. But, if the RefCounted data is actually allocated on the GC heap, an eventual GC sweep will delete them. What happens when a library receives a T* arg? Micro managing the ref-count at library boundaries sounds like a lot more trouble than manual memory management. Aside from purity mentioned above, another way to deal with that is to encapsulate uses of a RefCounted data structure so that raw pointers into it are unnecessary.
Re: draft proposal for ref counting in D
On 10/10/2013 12:31 AM, Walter Bright wrote: And that's the last email in the original thread! Durn, no it isn't.
Re: draft proposal for ref counting in D
And that's the last email in the original thread!
Re: draft proposal for ref counting in D
On 10/9/2013 11:52 PM, Jacob Carlborg wrote: Is this the last email in the conversation? Yes, I posted them in chronological order.
Re: The no gc crowd
On Thursday, October 10, 2013 00:30:55 Walter Bright wrote: It doesn't deal with circular references, which people keep bringing up as a very important problem. ARC doesn't deal with it automatically, either, it requires the user to insert weak pointers at the right places. But, if the RefCounted data is actually allocated on the GC heap, an eventual GC sweep will delete them. That may be true, but if you're using RefCounted because you can't afford the GC, then using the GC heap with them is not an option, because that could trigger a sweep, which is precisely what you're trying to avoid. More normal code may be fine with it but not the folks who can't afford the interruption of stop the world or any of the other costs that come with the GC. So, if RefCounted (or a similar type) is going to be used without the GC, it's going to need some type of weak-ref, even if it's just a normal pointer - though as you've pointed out, that pretty much throws @safety out the window as no GC is involved. But since you've arguably already done that by using malloc instead of the GC anyway, I think that it's debatable how much that matters. However, the GC would allow for more normal code to not worry about circular references with RefCounted. - Jonathan M Davis
Re: Range interface for std.serialization
On 2013-08-27 22:12, Dmitry Olshansky wrote: Feel free to nag me on the NG and personally for any deficiency you come across on the way there ;) I'm bumping this again with a new question. I'm thinking about how to output the data to output range. If the output range is of type ubyte[] how should I output serialized data looking like this: object runtimeType=main.Foo type=main.Foo key=0 id=0 int key=a id=13/int /object Should I output this in one chunk or in parts like this: object runtimeType=main.Foo type=main.Foo key=0 id=0 Then int key=a id=13/int Then /object If the first case is chosen I guess this data: object runtimeType=main.Foo type=main.Foo key=0 id=0 int key=a id=13/int /object object runtimeType=main.Foo type=main.Foo key=1 id=2 int key=a id=33/int /object Would be outputted in two chunks. -- /Jacob Carlborg
Re: The no gc crowd
On 2013-10-10 09:18, Walter Bright wrote: 1. Shared data cannot be passed to regular functions. That I understand. 2. Functions that create data structures would have to know in advance that they'll be creating a shared object. I'm not so sure this would not be an invasive change. If the function doesn't know it creates shared data it will assume it's not and it won't use any synchronization. Then suddenly someone casts it to shared and you're in trouble. 3. Immutable data is implicitly shared. But it is not created immutable - it is created as mutable data, then set to some state, then cast to immutable. It should be possible to create immutable data in the first place. No cast should be required. -- /Jacob Carlborg
Re: The no gc crowd
On 10/10/2013 09:33 AM, Jonathan M Davis wrote: I honestly don't think we can solve it a different way without completely redesigning shared. shared is specifically designed such that you have to either cast it way to do anything with it or write all of your code to explicitly work with shared, which is not something that generally makes sense to do unless you're creating a type whose only value is in being shared across threads. Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting. - Jonathan M Davis +1
Re: draft proposal for ref counting in D
On 10/10/2013 03:45 AM, Walter Bright wrote: Rainer Schuetze wrote: You have to put the lock around the pair of AddRef and Release, but if the compiler already splits this into two function calls, this cannot be done in the implementation. I would imagine the counter to be manipulated with atomic_add_and_fetch operations, so no locks are required.
Re: The no gc crowd
On 2013-10-10 09:33, Jonathan M Davis wrote: You might do that if you're creating the object simply to send it across, but it's frequently the case that the object was created well before it was sent across, and it frequently had to have operations done it other than simply creating it (which wouldn't work if it were shared). So, it often wouldn't make sense for the object being passed to be shared except when being passed. I guess if you're not creating it as shared to being with there's not way to tell that the given object now is shared an no thread local references are allowed. And once it's been passed, it's rarely the case that you want it to be shared. You're usually passing ownership. You're essentially taking a thread-local variable from one thread and making it a thread-local variable on another thread. Unfortunately, the type system does not support the concept of thread ownership (beyond thread-local vs shared), so it's up to the programmer to make sure that no references to the object are kept on the original thread, but there's really no way around that unless you're always creating a new object when you pass it across, which would result in which would usually be a unnecessary copy. So, it becomes like @trusted in that sense. It sounds like we need a way to transfer ownership of an object to a different thread. I honestly don't think we can solve it a different way without completely redesigning shared. shared is specifically designed such that you have to either cast it way to do anything with it or write all of your code to explicitly work with shared, which is not something that generally makes sense to do unless you're creating a type whose only value is in being shared across threads. Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting. I guess it wouldn't be possible to solve it without changing the type system. -- /Jacob Carlborg
Re: [OT] Liability of Moderator
On Wednesday, 9 October 2013 at 18:15:37 UTC, Walter Bright wrote: On 10/9/2013 2:01 AM, Chris wrote: The question always remains what is libelous or abusive? It's usually pretty obvious if you apply common sense. I also suggest simply googling for the what libel and slander actually are (many people have false assumptions about that). Yes and no. I know people who have taken or threatened to take legal action against others because they said something like I don't think this is the best way to say or do it. I propose ... Of course they will never succeed, but the hassle of having to deal with letters from solicitors is just annoying. I've seen the most ridiculous cases, things you couldn't make up. The problem is that unlike in the U.S. in the Republic of Ireland and GB the libel law is ridiculously overprotective. I do wonder how journalists cope with it. Mind you, that you apply common sense does not mean that others do so too. Lawyers vary enormously in the quality of their advice. They're a lot like programmers in that regard :-) In fairness, it's not only the lawyers, it's also the legislation that is often highly ambiguous. So lawyer A tells you something, and lawyer B will tell you something completely different. That's why it's best to check out court rulings and the different interpretations judges give the law in question. Else you pay $500 and all you know is it might but it might not be the case. Anyway, it's got nothing to do with D.
Re: The no gc crowd
On 2013-10-10 09:24, Jonathan M Davis wrote:
Pretty much nothing accepts shared. At best, templated functions accept
shared. Certainly, shared doesn't work at all with classes and structs unless
the type is specifically intended to be used as shared, because you have to
mark all of its member functions shared to be able to call them. And if you
want to use that class or struct as both shared and unshared, you have to
duplicate all of its member functions.
That being the case, the only way in general to use a shared object is to
protect it with a lock, cast it to thread-local (so that it can actually use
its member functions or be passed to other functions to be used), and then use
it. e.g.
synchronized
{
auto tl = cast(T)mySharedT;
auto result = tl.foo();
auto result2 = bar(tl);
}
Obviously, you then have to make sure that there are no thread-local
references to the shared object when the lock is released, but without casting
away shared like that, you can't do much of anything with it. So, similar to
when you cast away const, it's up to you to guarantee that the code doesn't
violate the type system's guarantees - i.e. that a thread-local variable is
not accessed by multiple threads. So, you use a lock of some kind to protect
the shared variable while it's treated as a thread-local variable in order to
ensure that that guarantee holds. Like with casting away const or with
@trusted, there's obviously risk in doing this, but there's really no other
way to use shared at this point - certainly not without it being incredibly
invasive to your code and forcing code duplication.
Sounds like we need a way to tell that a parameter is thread local but
not allowed to escape a reference to it.
Object foo;
void bar (shared_tls Object o)
{
foo = o; // Compile error, cannot escape a shared thread local
}
void main ()
{
auto o = new shared(Object);
synchronized { bar(o); }
}
Both shared can thread local be passed to shared_tls. If shared is
passed it assumes to be synchronized during the call to bar.
This will still have the problem of annotating all code with this
attribute. Or this needs to be default, which would cause a lot of code
breakage.
--
/Jacob Carlborg
Re: std.d.lexer : voting thread
Am 10.10.2013 03:25, schrieb Martin Nowak: On 10/06/2013 10:18 AM, Sönke Ludwig wrote: I also see no fundamental reason why the API forbids extension for shared sting tables or table-less lexing. The current API requires to copy slices of the const(ubyte)[] input to string values in every token. This can't be done efficiently without a string table. But a string table is unnecessary for many use-cases, so the API has a built-in performance/memory issue. But it could be extended later to accept immutable input as a special case, thus removing that requirement, if I'm not overlooking something. In that case it still is a pure implementation detail.
Re: Voting/Scoring and final decision discussion
On Thursday, 10 October 2013 at 03:14:30 UTC, Jesse Phillips wrote: There is little documentation on how to handle the situation the Review Manager is currently facing. I would like to open this discussion to point out why, and to poll for if we should have any. The reason is that our process comes from the the Boost review, and there is no such definition[1]. The way we have it makes it appear that the module is accepted as a majority vote. However that isn't the intention of the Boost process. The Review Manager wields a lot of power during the vote tally. The key line in our documentation: Tallies votes and decides if there is consensus to accept the library and under what conditions. The Review Manager is trying to judge based on the input how well this library fits with the goals of Phobos and the D community. That doesn't mean a landslide victory is needed. So with that I, and probably Dicebot, would like to hear feedback. Dicebot, consider what information may help make your decision. Would yes votes including positive feedback help (it is easier to side with those providing an argument)? 1. http://www.boost.org/community/reviews.html#Introduction The final accept or reject decision is made by the Review Manager, based on the review comments received from boost mailing list members. Boost doesn't do a review then vote separation. I think that we should use consensus model, not a majority vote. It's useful for small groups. For example, Wikipedia use it: This page in a nutshell: Consensus is Wikipedia's fundamental model for editorial decision-making. http://en.wikipedia.org/wiki/Wikipedia:Consensus
Re: Improvements to std.typecons.Nullable
On 10/9/13 10:07 AM, monarch_dodra wrote: On Wednesday, 9 October 2013 at 16:34:52 UTC, BLM768 wrote: On Wednesday, 9 October 2013 at 06:48:31 UTC, monarch_dodra wrote: OK, so that's two functions already. What about opCmp? What about toHash? Since ordered comparisons make no sense with null values, opCmp would need to throw an exception when working with null values anyway. That's exactly what it does right now. I've considered the other operators, and the same logic seems to apply to them. As for toHash, that needs to be overloaded whenever opEquals is. That's true for all types. What if T is a range? Then Nullable!T.empty should return true if the Nullable is empty. IF we don't, we'll get a crash in foreach. It does. Null is not an empty range; it's the _absence_ of a range. They are not the same concept, and a null range _cannot_ be empty because the range does not exist. Therefore, it's a logic error and should throw an exception. That was my point. Writting Nullable!T == T is the exact same thing: Comparison of a value with the absence of a value. It's neither equal nor different, it's an error. I'm confused. I thought Nullable!T == T is well defined to mean true if a value is present and equal to the right-hand side, or false otherwise (the absence of a value is a singularity unequal with all objects). What's harmful about that? Andrei
Re: The no gc crowd
On 2013-10-10 01:21:25 +, deadalnix deadal...@gmail.com said: On Wednesday, 9 October 2013 at 23:37:53 UTC, Michel Fortin wrote: In an ideal world, we'd be able to choose between using a GC or using ARC when building our program. A compiler flag could do the trick. But that becomes messy when libraries (static and dynamic) get involved as they all have to agree on the same codegen to work together. Adding something to mangling that would cause link errors in case of mismatch might be good enough to prevent accidents though. ObjC guys used to think that. It turns out it is a really bad idea. Things were much worse with Objective-C because at the time there was no ARC, reference-counting was manual and supporting both required a lot of manual work. Supporting the GC wasn't always a easy either, as the GC only tracked pointers inside of Objective-C objects and on the stack, not in structs on the heap. The GC had an implementation problem for pointers inside static segments, and keeping code working on both a GC and reference-counted had many perils. I think it can be done better in D. We'd basically just be changing the GC algorithm so it uses reference counting. The differences are: 1. unpredictable lifetimes - predictable lifetime 2. no bother about cyclic references - need to break them with weak The later is probably the most problematic, but if someone has leaks because he uses a library missing weak annotations he can still run the GC to collect them while most memory is reclaimed through ARC, or he can fix the problematic library by adding weak at the right places. -- Michel Fortin michel.for...@michelf.ca http://michelf.ca
Re: The no gc crowd
On 2013-10-10 06:41:19 +, Jacob Carlborg d...@me.com said: On 2013-10-10 06:24, Jonathan M Davis wrote: So, it's my impression that being able to consider casting to or from shared as abnormal in code which uses shared is a bit of a pipe dream at this point. The current language design pretty much requires casting when doing much of anything with concurrency. There must be a better way to solve this. http://michelf.ca/blog/2012/mutex-synchonization-in-d/ -- Michel Fortin michel.for...@michelf.ca http://michelf.ca
Re: Improvements to std.typecons.Nullable
On Thursday, 10 October 2013 at 10:09:23 UTC, Andrei Alexandrescu
wrote:
I'm confused. I thought Nullable!T == T is well defined to mean
true if a value is present and equal to the right-hand side,
or false otherwise (the absence of a value is a singularity
unequal with all objects). What's harmful about that?
Andrei
That in itself, I think is actually OK. It would be OK, because I
think we can agree that a no-value is different from any value.
In this case, we are making a call to Nullable's opEqual. I'm
actually fine with this, because it is a call to Nullable's
member function.
The point though is that this crashed at runtime, and nobody
until now noticed it, because of the alias this. Ditto for
toString. Ditto for to hash.
My argument is against the alias this itself. It is making a
cast when we don't actually expect it. Basically, any time you do
a call on said nullable, you have to really think about what you
are doing, because your nullable *will* be referenced on the
first chance it gets.
Basically, passing a Nullable!T to any function that expects a T
is a silent runtime danger, which we really shouldn't have to
accept.
Ultimately, it seems to boil down to a personal preference:
should Nullable!T emulate the behavior of D's existing nullable
types, or should it use a more explicit syntax? I personally
lean toward consistency, in part because doing otherwise would
be a breaking change that doesn't really seem justified unless
we can solve the issue with _all_ nullable references.
Well, not quite, AFAIK, the only two Nullable types that exist
in D are pointers, and class references.
a pointer will *never* implicitly degrade to its pointed type,
unless you actually dereference it, or call a function on its
member.
Watch:
struct S{void doit();}
void foo(S);
S* p;
Nullable!S n;
p.do_it(); //passes: implicitly dereferences
//thanks to an *explicit* call to do it.
foo(p); //Nope
p.foo(); //Nope
n.doit(); //Fine;
foo(n); //Fine...
n.foo(); //Fine too...
In those last to calls, and unexpected dereference happens:
You thought you were passing n to foo()? You were wrong.
As for class references, they behave pretty much the same.
//
I think opDispatch would have done a *much* better job at
emulating a nullable type. I threw this together:
//
struct Nullable(T)
{
private T _value;
template opDispatch(string s)
{
enum ss = format(q{
static if (is(typeof({enum tmp = T.%1$s;})))
enum opDispatch = T.%1$s;
else
{
auto ref opDispatch(Args...)(auto ref Args args)
{
return _value.%1$s(args);
}
}
}, s);
pragma(msg, ss);
mixin(ss);
}
@property ref inout(T) get()() inout
{
//@@@6169@@@: We avoid any call that might evaluate
nullValue's %s,
//Because it might messup get's purity and safety
inference.
enum message = Called `get' on null Nullable!( ~
T.stringof ~ ,nullValue).;
assert(!isNull, message);
return _value;
}
//rest of the struct
}
//
And now I get this:
//
struct S
{
void doit(){}
enum a = 1;
}
void foo(S){};
void main()
{
Nullable!S p;
int i = Nullable!S.a; //OK
int j = p.a; //NO problem either;
p.doit(); //Fine
//foo(p); //Error: function main.foo (S _param_0) is not
callable using argument types (Nullable!(S))
//p.foo(); //Error: no property 'foo' for type 'S'
foo(p.get); //OK! You want p's *get*. Now I get it.
p.get.foo(); //OK! You want p's *get*. Now I get it.
}
//
Disclaimer: My opDispatch code is not perfect. In particular,
p.foo() may compile depending on what is imported in the module
that defines the Nullable.
Re: Voting/Scoring and final decision discussion
On Thursday, 10 October 2013 at 03:14:30 UTC, Jesse Phillips wrote: Dicebot, consider what information may help make your decision. Would yes votes including positive feedback help (it is easier to side with those providing an argument)? Yes, that will help. If anyone who has voted Yes will comment in this topic with more detailed explanation, it will be taken into consideration. Right now I am more leaning towards declining the module adoption, mostly because No vote amount is rather high among Phobos developers, which is a really bad sign. It is pretty clear though that this voting has shown certain weak spot in out adoption process for more controversial proposals. Once person (review manager) simply should not have that much decision power in such situation.
Re: The no gc crowd
On Thursday, 10 October 2013 at 04:24:31 UTC, Jonathan M Davis wrote: Also, casting _away_ shared is going to be a very common operation due to how shared works. It is yet another use case for `scope` storage class. Locking `shared` variable via mutex should return same variable but casted to non-shared `scope` (somewhere inside the locking library function). Then it is safe to pass it to functions accepting scope parameters as reference won't possibly escape.
Re: Range interface for std.serialization
On Thursday, 10 October 2013 at 07:45:51 UTC, Jacob Carlborg wrote: ... I thought the very point of output ranges is that you can output in any chunks that seem most convenient / efficient to you.
Re: std.d.lexer : voting thread
On Thursday, 10 October 2013 at 04:33:15 UTC, Jonathan M Davis wrote: On Wednesday, October 02, 2013 16:41:54 Dicebot wrote: After brief discussion with Brian and gathering data from the review thread, I have decided to start voting for `std.d.lexer` inclusion into Phobos. I'm going to have to vote no. While Brian has done some great work, I think that it's clear from the discussion that there are still some potential issues (e.g. requiring a string table) that need further discussion and possibly API changes. Also, while I question that a generated lexer can beat a hand-written one, I think that we really should look at what Andrei's proposing and look at adjusting whan Brian has done accordingly - or at least do enough so that we can benchmark the two approaches. As such, accepting the lexer right now doesn't really make sense. However, we may want to make it so that the lexer is in some place of prominence (outside of Phobos - probably on dub but mentioned somewhere on dlang.org) as an _experimental_ module which is clearly marked as not finalized but which is ready for people to use and bang on. That way, we may be able to get some better feedback generated from more real world use. - Jonathan M Davis Vote: No. Same reason as Jonathan above.
Re: dub: should we make it the de jure package manager for D?
On 08/10/2013 13:38, Jacob Carlborg wrote: On 2013-10-08 12:51, Bruno Medeiros wrote: From what I understand, for dependency graph locking to work at all, then each package (as stored in the central package repository) would have to specify its full dependency graph in the package specification. So the foo package would have to specify not only the bar dependency, but also xyz=0.0.1 as a dependency. Isn't that how it would work? No, now necessarily. Using Bundler it works like this: * You have your Gemfile, corresponds to package.json in dub * You run bundle install * When you do that it will create a new file, Gemfile.lock. This file contains the complete dependency graph of what it just installed. * Running bundle install when Gemfile.lock exist, it will read Gemfile.lock instead of Gemfile Rigth, but then you would need to share the .lock file to the other machine you want to install/compile foo in. The repository alone would not be enough for that. Rather, I think dub should adopt Semantic Versioning as part of its recommended practices for package versioning: http://semver.org/spec/v2.0.0.html In this practice, stuff like xyz: =0.0.1 is not recommended, an upper bound on the version is required, to allow breaking changes in xyz. Semantic versioning helps, but it won't solve the problem. I wouldn't trust that a library actually follows the semantic versioning scheme. It's quite easy to accidentally add new API without incrementing the middle number. Or break the API. Hum that is true. Especially without a tool that can verify API compatibility. -- Bruno Medeiros - Software Engineer
Target hook - compiler specific pragmas.
As part of the front-end harmonising work I'm doing, will be adding in a Target hook to handle compiler-specific pragmas. This would abstract out code like this in LDC. https://github.com/ldc-developers/ldc/blob/master/dmd2/attrib.c#L1113 https://github.com/ldc-developers/ldc/blob/master/dmd2/attrib.c#L1182 Which implements these pragmas. http://wiki.dlang.org/LDC-specific_language_changes#Pragmas I have plans for adding in pragmas into GDC at a later date too as I plan to make the magic modules of GDC re-locatable (core.stdc.stdarg, gcc.builtins, etc...) David, you OK with this? :-) Regards Iain.
Re: The no gc crowd
On 2013-10-10 13:17, Michel Fortin wrote: http://michelf.ca/blog/2012/mutex-synchonization-in-d/ This looks similar to what I described here: http://forum.dlang.org/thread/bsqqfmhgzntryyaqr...@forum.dlang.org?page=19#post-l35rql:24og2:241:40digitalmars.com -- /Jacob Carlborg
Re: The no gc crowd
On 2013-10-10 13:17, Michel Fortin wrote: http://michelf.ca/blog/2012/mutex-synchonization-in-d/ I think I like the idea, but won't you have the same problem as Jonathan described? You can't pass these variables to another function that doesn't expect it to be passed a synchronized variable. You can pass it to pure functions which mean you can probably pass it to a couple of more functions compared to using shared. -- /Jacob Carlborg
Re: The no gc crowd
On 2013-10-10 13:02:14 +, Jacob Carlborg d...@me.com said: On 2013-10-10 13:17, Michel Fortin wrote: http://michelf.ca/blog/2012/mutex-synchonization-in-d/ I think I like the idea, but won't you have the same problem as Jonathan described? You can't pass these variables to another function that doesn't expect it to be passed a synchronized variable. You can pass it to pure functions which mean you can probably pass it to a couple of more functions compared to using shared. Well, it's one piece of a puzzle. In itself it already is better than having to cast every time. Combined with a way to pass those variables to other functions safely, it should solve practically all the remaining problems that currently require a cast. But I don't have nice a solution for the later problem (short of adding more attributes). -- Michel Fortin michel.for...@michelf.ca http://michelf.ca
Re: Range interface for std.serialization
On 2013-10-10 14:13, Dicebot wrote: I thought the very point of output ranges is that you can output in any chunks that seem most convenient / efficient to you. Outputting it like the first suggestion will be a lot more convenient, especially how std.xml currently works. Although it will probably not be as efficient. This basically means a complete object graph will be outputted in one chunk. That is, unless the top element is a range. -- /Jacob Carlborg
Re: The no gc crowd
On 2013-10-10 13:03:37 +, Jacob Carlborg d...@me.com said: On 2013-10-10 13:17, Michel Fortin wrote: http://michelf.ca/blog/2012/mutex-synchonization-in-d/ This looks similar to what I described here: http://forum.dlang.org/thread/bsqqfmhgzntryyaqr...@forum.dlang.org?page=19#post-l35rql:24og2:241:40digitalmars.com Somewhat similar. I don't think it's a good practice to make a mutex part of the public interface of an object (or any public interface for that matter), which is why they're kept private inside the class in my examples. Public mutexes can be locked from anywhere in your program, they lack encapsulation and this makes them prone to deadlocks. -- Michel Fortin michel.for...@michelf.ca http://michelf.ca
Re: dub: should we make it the de jure package manager for D?
On 2013-10-10 14:25, Bruno Medeiros wrote: Rigth, but then you would need to share the .lock file to the other machine you want to install/compile foo in. The repository alone would not be enough for that. Yes, absolutely. In the Rails world it's recommended that you add both Gemfile and Gemfile.lock to the SCM for you're Rails applications. For Rails plugins you should only add Gemfile to SCM. You usually don't have your Rails application uploaded to RubyGems. -- /Jacob Carlborg
Re: Range interface for std.serialization
10-Oct-2013 11:45, Jacob Carlborg пишет: On 2013-08-27 22:12, Dmitry Olshansky wrote: Feel free to nag me on the NG and personally for any deficiency you come across on the way there ;) I'm bumping this again with a new question. I'm thinking about how to output the data to output range. If the output range is of type ubyte[] how should I output serialized data looking like this: object runtimeType=main.Foo type=main.Foo key=0 id=0 int key=a id=13/int /object Should I output this in one chunk or in parts like this: object runtimeType=main.Foo type=main.Foo key=0 id=0 Then int key=a id=13/int Then /object [snip] I do believe it's a very minor detail of a specific implementation of XML archiver. Speaking of Archivers in general the main point is to try to avoid accumulating lots of data in memory if possible and put things as they go. This however doesn't preclude the 2nd goal - outputting as much as possible in one go (and not 2 chunks) is preferable (1 call vs 2 calls to put of the output range etc) if doesn't harm memory usage (O(1) is OK, anything else not) and doesn't complicate archiver. -- Dmitry Olshansky
Re: The no gc crowd
On 09/10/13 06:25, Andrei Alexandrescu wrote: The way I see it we must devise a robust solution to that, NOT consider the state of the art immutable (heh, a pun). Must say I have had a miserable experience with immutability and any kind of complex data structure, particularly when concurrency is involved. As a practical fact I've often found it necessary to convert to immutable (not always via a cast or std.conv.to; sometimes via assumeUnique) to pass a complex data structure to a thread, but then to convert _away_ from immutable inside the thread in order for that data (which is never actually mutated) to be practically usable. I'm sure there are things that I could do better, but I did not find a superior solution that was also performance-friendly.
Re: The no gc crowd
On 10/10/2013 12:51 AM, Jacob Carlborg wrote: On 2013-10-10 09:18, Walter Bright wrote: 1. Shared data cannot be passed to regular functions. That I understand. 2. Functions that create data structures would have to know in advance that they'll be creating a shared object. I'm not so sure this would not be an invasive change. If the function doesn't know it creates shared data it will assume it's not and it won't use any synchronization. Then suddenly someone casts it to shared and you're in trouble. Same comment as for immutable data - create the data structure as thread local, because of (1), and then cast to shared and hand it to another thread. 3. Immutable data is implicitly shared. But it is not created immutable - it is created as mutable data, then set to some state, then cast to immutable. It should be possible to create immutable data in the first place. No cast should be required.
Re: The no gc crowd
On Oct 9, 2013, at 9:24 PM, Jonathan M Davis jmdavisp...@gmx.com wrote: And given that std.concurrency requires casting to and from shared or immutable in order to pass objects across threads, it seems ilke most of D's concurrency model requires casting to and/or from shared or immutable. std.concurrency won't be this way forever though. We could fake move semantics with something like assumeUnique!T, so send could be modified to accept a non-shared class that's marked as Unique. The other option would be deep copying or serialization.
Re: The no gc crowd
On Oct 9, 2013, at 11:34 PM, Jacob Carlborg d...@me.com wrote: On 2013-10-10 02:22, Sean Kelly wrote: Only that this would have to be communicated to the user, since moving data later is problematic. Today, I think it's common to construct an object as unshared and then cast it. What is the reason to not create it as shared in the first place? The same as immutable--you may not have all the shared functions available to establish the desired state. But I'll grant that this is obviously way more common with immutable than shared.
D minions - time to vote!
https://news.ycombinator.com/item?id=6527104
Re: The no gc crowd
On Thursday, October 10, 2013 19:23:14 Joseph Rushton Wakeling wrote: On 09/10/13 06:25, Andrei Alexandrescu wrote: The way I see it we must devise a robust solution to that, NOT consider the state of the art immutable (heh, a pun). Must say I have had a miserable experience with immutability and any kind of complex data structure, particularly when concurrency is involved. As a practical fact I've often found it necessary to convert to immutable (not always via a cast or std.conv.to; sometimes via assumeUnique) to pass a complex data structure to a thread, but then to convert _away_ from immutable inside the thread in order for that data (which is never actually mutated) to be practically usable. I'm sure there are things that I could do better, but I did not find a superior solution that was also performance-friendly. std.concurrency's design basically requires that you cast objects to shared or immutable in order to pass them across threads (and using assumeUnique is still doing the cast, just internally). And then you have to cast them back to thread-local mutable on the other side to complete the pass and make the object useable again. There's really no way around that at this point, not without completely redesigning shared. Arguably, it's better to use shared when doing that rather than immutable, but at least in the past, shared hasn't worked right with std.concurrency even though it's supposed to (though that's an implementation issue rather than a design one, and it might be fixed by now - I haven't tried recently). And whether you're using shared or immutable, you're still having to cast. I'm honestly surprised that Andrei is rejecting the idea of casting to/from shared or immutable being normal given how it's required by our current concurrency model. And changing that would be a _big_ change. - Jonathan M Davis
Re: std.d.lexer : voting thread
On 10/9/13 6:10 PM, Martin Nowak wrote:
On 10/08/2013 05:05 PM, Andrei Alexandrescu wrote:
But no matter. My most significant bit is, we need a trie lexer
generator ONLY from the token strings, no TK_XXX user-provided symbols
necessary. If all we need is one language (D) this is a non-issue
because the library writer provides the token definitions. If we need to
support user-provided languages, having the library manage the string -
small integer mapping becomes essential.
It's good to get rid of the symbol names.
You should try to map the strings onto an enum so that final switch works.
final switch (t.type_)
{
case t!: break;
// ...
}
Excellent point! In fact one would need to use t!.id instead of t!.
I'll work on that next.
Andrei
Re: The no gc crowd
On Oct 10, 2013, at 4:17 AM, Michel Fortin michel.for...@michelf.ca wrote: On 2013-10-10 06:41:19 +, Jacob Carlborg d...@me.com said: On 2013-10-10 06:24, Jonathan M Davis wrote: So, it's my impression that being able to consider casting to or from shared as abnormal in code which uses shared is a bit of a pipe dream at this point. The current language design pretty much requires casting when doing much of anything with concurrency. There must be a better way to solve this. http://michelf.ca/blog/2012/mutex-synchonization-in-d/ Good article. But why didn't you mention core.sync? It has both a Mutex and a ReadWriteMutex (ie. shared_mutex).
Re: The no gc crowd
On 10/10/13 19:31, Jonathan M Davis wrote: I'm honestly surprised that Andrei is rejecting the idea of casting to/from shared or immutable being normal given how it's required by our current concurrency model. And changing that would be a _big_ change. I'm starting to incline towards the view that type qualifications of _any_ kind become problematic once you start working with any types other than built-in, and not just in the context of concurrency. See e.g.: http://d.puremagic.com/issues/show_bug.cgi?id=11148 http://d.puremagic.com/issues/show_bug.cgi?id=11188 I'd really appreciate advice on how to handle issues like these, because it's becoming a serious obstacle to my work on std.rational.
Re: The no gc crowd
On Oct 10, 2013, at 10:23 AM, Joseph Rushton Wakeling joseph.wakel...@webdrake.net wrote: On 09/10/13 06:25, Andrei Alexandrescu wrote: The way I see it we must devise a robust solution to that, NOT consider the state of the art immutable (heh, a pun). Must say I have had a miserable experience with immutability and any kind of complex data structure, particularly when concurrency is involved. As long as the reference itself can be reassigned (tail-immutable, I suppose) I think immutable is occasionally quite useful for complex data structures. It basically formalizes the RCU (read-copy-update) approach to wait-free concurrency. I'd tend to use this most often for global data structures built up on app start, and updated rarely to never as the program runs.
Re: The no gc crowd
On 10/10/13 19:39, Sean Kelly wrote: As long as the reference itself can be reassigned (tail-immutable, I suppose) I think immutable is occasionally quite useful for complex data structures. It basically formalizes the RCU (read-copy-update) approach to wait-free concurrency. I'd tend to use this most often for global data structures built up on app start, and updated rarely to never as the program runs. This kind of stuff is outside my experience, so if you'd like to offer a more detailed explanation/example, I'd be very grateful :-)
Re: The no gc crowd
On Oct 10, 2013, at 10:36 AM, Joseph Rushton Wakeling joseph.wakel...@webdrake.net wrote: On 10/10/13 19:31, Jonathan M Davis wrote: I'm honestly surprised that Andrei is rejecting the idea of casting to/from shared or immutable being normal given how it's required by our current concurrency model. And changing that would be a _big_ change. I'm starting to incline towards the view that type qualifications of _any_ kind become problematic once you start working with any types other than built-in, and not just in the context of concurrency. See e.g.: http://d.puremagic.com/issues/show_bug.cgi?id=11148 http://d.puremagic.com/issues/show_bug.cgi?id=11188 I'm inclined to agree about shared. But I see this largely as more encouragement to keep data thread-local in D. If we can clean up move semantics via std.concurrency, I would be reasonably happy with data sharing in D. As for const / immutable, I guess I don't see this as such an issue because I've been dealing with it in C++ for so long. You either have to commit 100% to using const attributes or not use them at all. Anything in between is fraught with problems.
Re: The no gc crowd
On Thursday, October 10, 2013 10:27:24 Sean Kelly wrote: On Oct 9, 2013, at 9:24 PM, Jonathan M Davis jmdavisp...@gmx.com wrote: And given that std.concurrency requires casting to and from shared or immutable in order to pass objects across threads, it seems ilke most of D's concurrency model requires casting to and/or from shared or immutable. std.concurrency won't be this way forever though. We could fake move semantics with something like assumeUnique!T, so send could be modified to accept a non-shared class that's marked as Unique. I take it that you mean something other than std.exception.assumeUnique which simply casts to immutable? All that std.exception.assumeUnique does for you over casting is document why the cast is happening. If you're talking about creating a wrapper type which indicates that the object is unique, I'd still expect that the casting would have to be happening underneath the hood when the object was passed (though then for better or worse, it would be encapsulated). And unless the objecte were always in that Unique wrapper, the programmer would still have to be promising that the object was actually unique and not being shared across threads rather than the type system doing it, in which case, I don't see much gain over simply casting. And if it's always in the wrapper, then you're in a similar boat to shared or immutable in that it's not the correct type. I expect that there are nuances in what you're suggesting that I don't grasp at the moment, but as far as I can tell, the type system fundamentally requires a cast when passing objects across threads. It's just a question of whether that cast is hidden or not, and depending on how you hide it, I think that there's a real risk of the situation being worse than if you require explicit casting, because then what you're doing and what you have to be careful about are less obvious, since what's going on is hidden. The other option would be deep copying or serialization. That would be far too costly IMHO. In the vast majority of cases (in my experience at least and from what I've seen others do), what you really want to do is pass ownership of the object from one thread to the other, and while deep copying would allow you to avoid type system issues, it's completely unnecessary otherwise. So, we'd be introducing overhead just to satisfy our very restrictive type system. The only way that I can think of to fix that would be for objects to all have a concept of what thread owns them (so that the type system would be able to understand the concept of an object's ownership being passed from one thread to another), but that would be a _big_ change and likely way too complicated in general. - Jonathan M Davis
Re: The no gc crowd
On Oct 10, 2013, at 10:43 AM, Joseph Rushton Wakeling joseph.wakel...@webdrake.net wrote: On 10/10/13 19:39, Sean Kelly wrote: As long as the reference itself can be reassigned (tail-immutable, I suppose) I think immutable is occasionally quite useful for complex data structures. It basically formalizes the RCU (read-copy-update) approach to wait-free concurrency. I'd tend to use this most often for global data structures built up on app start, and updated rarely to never as the program runs. This kind of stuff is outside my experience, so if you'd like to offer a more detailed explanation/example, I'd be very grateful :-) Configuration data, for example. On app start you might load a config file, generate information about the user, and so on, before real processing begins. This data needs to be visible everywhere and it rarely if ever changes as the program runs, so you fill the data structures and then make them immutable. Assuming, of course, that the data structures have immutable versions of all the necessary functions (which is unfortunately a pretty big assumption).
Re: The no gc crowd
On 10/10/13 12:18 AM, Walter Bright wrote: On 10/9/2013 11:34 PM, Jacob Carlborg wrote: On 2013-10-10 02:22, Sean Kelly wrote: Only that this would have to be communicated to the user, since moving data later is problematic. Today, I think it's common to construct an object as unshared and then cast it. What is the reason to not create it as shared in the first place? 1. Shared data cannot be passed to regular functions. I don't understand this. If a function/method accepts shared, then it can be passed shared data. 2. Functions that create data structures would have to know in advance that they'll be creating a shared object. I'm not so sure this would not be an invasive change. There is no other way around it. And this is not a change - it's fixing something. 3. Immutable data is implicitly shared. But it is not created immutable - it is created as mutable data, then set to some state, then cast to immutable. That all must happen in the runtime, NOT in user code. Andrei
Re: The no gc crowd
On Oct 10, 2013, at 10:50 AM, Jonathan M Davis jmdavisp...@gmx.com wrote: On Thursday, October 10, 2013 10:27:24 Sean Kelly wrote: On Oct 9, 2013, at 9:24 PM, Jonathan M Davis jmdavisp...@gmx.com wrote: And given that std.concurrency requires casting to and from shared or immutable in order to pass objects across threads, it seems ilke most of D's concurrency model requires casting to and/or from shared or immutable. std.concurrency won't be this way forever though. We could fake move semantics with something like assumeUnique!T, so send could be modified to accept a non-shared class that's marked as Unique. I take it that you mean something other than std.exception.assumeUnique which simply casts to immutable? All that std.exception.assumeUnique does for you over casting is document why the cast is happening. If you're talking about creating a wrapper type which indicates that the object is unique, I'd still expect that the casting would have to be happening underneath the hood when the object was passed (though then for better or worse, it would be encapsulated). And unless the objecte were always in that Unique wrapper, the programmer would still have to be promising that the object was actually unique and not being shared across threads rather than the type system doing it, in which case, I don't see much gain over simply casting. And if it's always in the wrapper, then you're in a similar boat to shared or immutable in that it's not the correct type. I expect that there are nuances in what you're suggesting that I don't grasp at the moment, but as far as I can tell, the type system fundamentally requires a cast when passing objects across threads. It's just a question of whether that cast is hidden or not, and depending on how you hide it, I think that there's a real risk of the situation being worse than if you require explicit casting, because then what you're doing and what you have to be careful about are less obvious, since what's going on is hidden. Yes, we couldn't use assumeUnique as-is because then the object would land on the other side as immutable. It would have to wrap the object to tell send() that the object, while not shared or immutable, is safe to put in a message. Then send() would discard the wrapper while constructing the message.
Re: The no gc crowd
On 10/10/13 12:33 AM, Jonathan M Davis wrote: I honestly don't think we can solve it a different way without completely redesigning shared. shared is specifically designed such that you have to either cast it way to do anything with it no or write all of your code to explicitly work with shared, which is not something that generally makes sense to do unless you're creating a type whose only value is in being shared across threads. yes Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting. no Andrei
Re: D minions - time to vote!
On 10/10/2013 12:28 PM, Walter Bright wrote: https://news.ycombinator.com/item?id=6527104 Minions away!
Re: The no gc crowd
On Oct 10, 2013, at 10:55 AM, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 10/10/13 12:33 AM, Jonathan M Davis wrote: Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting. no Yeah, I'd want to see this claim backed up by some examples. The only data I share globally in my own apps is the occasional container. Configuration data, a user database, whatever. I'll also frequently move data between threads while dispatching tasks, but otherwise everything is thread-local. I imagine there are other reasonable methods for using shared data, but I don't know what they are.
Re: The no gc crowd
On Wed, 09 Oct 2013 22:30:05 +0200, Adam D. Ruppe wrote:
On Wednesday, 9 October 2013 at 20:10:40 UTC, Justin Whear wrote:
Related to the latter, it would be really nice to be able to prove that
a section of code makes no heap allocations/GC collections.
As a quick temporary thing, how about gc_throw_on_next(); ?
That'd just set a thread local flag that gc_malloc checks and if it is
set, immediately resets it and throws an AllocAssertError. My thought is
this could be quickly and easily implemented pending a better solution
and in the mean time can be used in unit tests to help check this stuff.
So user-code would look like this?
// Set up code, GC is fine here
...
// Entering critical loop (which may run for months at a time)
debug GC.throw_on_next(true);
while (true)
{
...
}
// Tear-down code, GC is fine here
// (though unnecessary as the process is about to exit)
debug GC.throw_on_next(false);
...
Something like this would make testing simpler and is probably much more
feasible than deep static analysis.
Re: The no gc crowd
On 10/10/13 19:50, Sean Kelly wrote: Configuration data, for example. On app start you might load a config file, generate information about the user, and so on, before real processing begins. This data needs to be visible everywhere and it rarely if ever changes as the program runs, so you fill the data structures and then make them immutable. Assuming, of course, that the data structures have immutable versions of all the necessary functions (which is unfortunately a pretty big assumption). Yup, you're right, it's a big assumption. In my case I was interested in loading a graph (network) and running many simulations on it in parallel. The graph itself was static, so could readily be made immutable. However, I found that it was difficult to write code that would accept both immutable and mutable graphs as input, without impacting performance. So, I opted for threads to receive an immutable graph and cast it to mutable, even though it was never actually altered. My experience was no doubt partially due to issues with the overall design I chose, and maybe I could have found a way around it, but it just seemed easier to use this flawed approach than to re-work everything.
Re: The no gc crowd
On 10/10/13 19:46, Sean Kelly wrote:
As for const / immutable, I guess I don't see this as such an issue because
I've been dealing with it in C++ for so long. You either have to commit 100%
to using const attributes or not use them at all. Anything in between is
fraught with problems.
Well, the problem is essentially that you can have a function like:
void foo(int i) { ... }
... and if you pass it an immutable or const int, this is not a problem, because
you're passing by value.
But now try
void foo(BigInt i) { ... }
... and it won't work when passed a const/immutable variable, even though again
you're passing by value. That's not nice, not intuitive, and generally speaking
makes working with complex data types annoying.
It's why, for example, std.math.abs currently works with BigInt but not with
const or immutable BigInt -- which is very irritating indeed.
Re: The no gc crowd
On Oct 10, 2013, at 11:17 AM, Joseph Rushton Wakeling joseph.wakel...@webdrake.net wrote: On 10/10/13 19:50, Sean Kelly wrote: Configuration data, for example. On app start you might load a config file, generate information about the user, and so on, before real processing begins. This data needs to be visible everywhere and it rarely if ever changes as the program runs, so you fill the data structures and then make them immutable. Assuming, of course, that the data structures have immutable versions of all the necessary functions (which is unfortunately a pretty big assumption). Yup, you're right, it's a big assumption. In my case I was interested in loading a graph (network) and running many simulations on it in parallel. The graph itself was static, so could readily be made immutable. However, I found that it was difficult to write code that would accept both immutable and mutable graphs as input, without impacting performance. So, I opted for threads to receive an immutable graph and cast it to mutable, even though it was never actually altered. My experience was no doubt partially due to issues with the overall design I chose, and maybe I could have found a way around it, but it just seemed easier to use this flawed approach than to re-work everything. That's kind of the issue I ran into with shared in Druntime. It seemed like what I had to do was have a shared method that internally cast this to unshared and then called the real function, which I knew was safe but the type system hated. But this seemed like a horrible approach and so I didn't ever qualify anything as shared.
Re: The no gc crowd
On Oct 10, 2013, at 11:21 AM, Joseph Rushton Wakeling
joseph.wakel...@webdrake.net wrote:
On 10/10/13 19:46, Sean Kelly wrote:
As for const / immutable, I guess I don't see this as such an issue because
I've been dealing with it in C++ for so long. You either have to commit
100% to using const attributes or not use them at all. Anything in between
is fraught with problems.
Well, the problem is essentially that you can have a function like:
void foo(int i) { ... }
... and if you pass it an immutable or const int, this is not a problem,
because you're passing by value.
But now try
void foo(BigInt i) { ... }
... and it won't work when passed a const/immutable variable, even though
again you're passing by value. That's not nice, not intuitive, and generally
speaking makes working with complex data types annoying.
It's why, for example, std.math.abs currently works with BigInt but not with
const or immutable BigInt -- which is very irritating indeed.
Isn't BigInt a struct? I'd expect it to work via copying just like concrete
types.
Re: The no gc crowd
On Thursday, 10 October 2013 at 17:23:20 UTC, Joseph Rushton Wakeling wrote: On 09/10/13 06:25, Andrei Alexandrescu wrote: The way I see it we must devise a robust solution to that, NOT consider the state of the art immutable (heh, a pun). Must say I have had a miserable experience with immutability and any kind of complex data structure, particularly when concurrency is involved. I feel your pain. See also this thread in D.learn: http://forum.dlang.org/post/sdefkajobwcfikkel...@forum.dlang.org
Re: The no gc crowd
On 2013-10-10 17:34:47 +, Sean Kelly s...@invisibleduck.org said: On Oct 10, 2013, at 4:17 AM, Michel Fortin michel.for...@michelf.ca wrote: http://michelf.ca/blog/2012/mutex-synchonization-in-d/ Good article. But why didn't you mention core.sync? It has both a Mutex and a ReadWriteMutex (ie. shared_mutex). Because that would have required a ton of explanations about why you need casts everywhere to remove shared, and I don't even know where to begin to explain shared semantics. Shared just doesn't make sense to me the way it works right now. The examples in C++ are much clearer than anything I could have done in D2. I don't want to have to explain why I have to bypass the type system every time I need to access a variable. I'll add that I'm coding in C++ right now so it's much easier to come up with C++ examples. That said, it might be a good idea to add a note at the end about core.sync in case someone wants to try that technique in D. -- Michel Fortin michel.for...@michelf.ca http://michelf.ca
Re: The no gc crowd
On Thu, Oct 10, 2013 at 07:36:06PM +0200, Joseph Rushton Wakeling wrote:
On 10/10/13 19:31, Jonathan M Davis wrote:
I'm honestly surprised that Andrei is rejecting the idea of casting
to/from shared or immutable being normal given how it's required by
our current concurrency model. And changing that would be a _big_
change.
I'm starting to incline towards the view that type qualifications of
_any_ kind become problematic once you start working with any types
other than built-in, and not just in the context of concurrency.
See e.g.:
http://d.puremagic.com/issues/show_bug.cgi?id=11148
http://d.puremagic.com/issues/show_bug.cgi?id=11188
I'd really appreciate advice on how to handle issues like these,
because it's becoming a serious obstacle to my work on std.rational.
I left some comments on these bugs. Basically, BigInt should not be
implicitly castable from const/immutable to unqual, because unlike the
built-in types, it's *not* a value type:
BigInt x = 123;
BigInt y = x; // creates an alias to x's data.
Allowing implicit conversion to unqual would break immutability:
immutable(BigInt) x = 123;
const(BigInt) sneaky = x; // sneaky aliases x
BigInt y = sneaky; // now y aliases sneaky, which aliases x (oops)
Of course, the way BigInt is implemented, any operation on it causes new
data to be created (essentially it behaves like a copy-on-write type),
so it's not as though you can directly modify immutable this way, but
it breaks the type system and opens up possible loopholes.
What you need to do is to use inout for functions that need to handle
both built-in ints and BigInts, e.g.:
inout(Num) abs(Num)(inout(Num) x) {
return (x = 0) ? x : -x;
}
This *should* work (I think -- I didn't check :-P).
Arguably, a *lot* of generic code involving numerical operations is
broken, because they assume built-in types' behaviour of being
implicitly convertible to/from immutable (due to being value types).
I don't know about shared, though. Last I heard, shared was one big mess
so I'm not even going to touch it.
T
--
If the comments and the code disagree, it's likely that *both* are wrong. --
Christopher
Re: The no gc crowd
On 10/10/2013 10:54 AM, Andrei Alexandrescu wrote: On 10/10/13 12:18 AM, Walter Bright wrote: On 10/9/2013 11:34 PM, Jacob Carlborg wrote: On 2013-10-10 02:22, Sean Kelly wrote: Only that this would have to be communicated to the user, since moving data later is problematic. Today, I think it's common to construct an object as unshared and then cast it. What is the reason to not create it as shared in the first place? 1. Shared data cannot be passed to regular functions. I don't understand this. If a function/method accepts shared, then it can be passed shared data. I meant regular functions as in they are not typed as taking shared arguments. Shared cannot be implicitly cast to unshared. I say regular because very, very few functions are typed as accepting shared arguments. 2. Functions that create data structures would have to know in advance that they'll be creating a shared object. I'm not so sure this would not be an invasive change. There is no other way around it. And this is not a change - it's fixing something. I'm not convinced of that at all. 3. Immutable data is implicitly shared. But it is not created immutable - it is created as mutable data, then set to some state, then cast to immutable. That all must happen in the runtime, NOT in user code. Andrei
Re: The no gc crowd
On 10/10/13 20:28, Sean Kelly wrote: Isn't BigInt a struct? I'd expect it to work via copying just like concrete types. Yes, it's a struct, but somewhere inside its internals I think it contains arrays. I'm not sure how that affects copying etc., but suffice to say that if you try the following: BigInt a = 2; BigInt b = a; b = 3; assert(a != b); assert(a !is b); ... then it passes. So it behaves at least in this extent like a value type. But suffice to say that it was an unpleasant surprise that I couldn't just take it and pass to a function accepting an unqualified BigInt argument.
Re: The no gc crowd
On 10/10/13 20:28, H. S. Teoh wrote: I don't know about shared, though. Last I heard, shared was one big mess so I'm not even going to touch it. Yes, that seems to be the consensus.
Re: The no gc crowd
On 2013-10-10 16:14, Michel Fortin wrote: similar. I don't think it's a good practice to make a mutex part of the public interface of an object (or any public interface for that matter), which is why they're kept private inside the class in my examples. Public mutexes can be locked from anywhere in your program, they lack encapsulation and this makes them prone to deadlocks. Right, it's better to keep them private. -- /Jacob Carlborg
Re: The no gc crowd
On 10/10/13 20:28, H. S. Teoh wrote:
I left some comments on these bugs. Basically, BigInt should not be
implicitly castable from const/immutable to unqual, because unlike the
built-in types, it's *not* a value type:
BigInt x = 123;
BigInt y = x; // creates an alias to x's data.
BigInt a = 2;
BigInt b = a;
b = 3;
writeln(a);
writeln(b);
... gives you:
2
3
So, even though there's an array hidden away inside std.BigInt, it still seems
to copy via value.
Of course, the way BigInt is implemented, any operation on it causes new
data to be created (essentially it behaves like a copy-on-write type),
so it's not as though you can directly modify immutable this way, but
it breaks the type system and opens up possible loopholes.
I guess that explains my result above ?
What you need to do is to use inout for functions that need to handle
both built-in ints and BigInts, e.g.:
inout(Num) abs(Num)(inout(Num) x) {
return (x = 0) ? x : -x;
}
This *should* work (I think -- I didn't check :-P).
I did, and it results in issues with BigInt's opCmp. But that may say more
about BigInt's opCmp than about your solution.
Arguably, a *lot* of generic code involving numerical operations is
broken, because they assume built-in types' behaviour of being
implicitly convertible to/from immutable (due to being value types).
How would you suggest correcting that?
Re: The no gc crowd
On Thu, Oct 10, 2013 at 08:39:52PM +0200, Joseph Rushton Wakeling wrote: On 10/10/13 20:28, Sean Kelly wrote: Isn't BigInt a struct? I'd expect it to work via copying just like concrete types. Yes, it's a struct, but somewhere inside its internals I think it contains arrays. I'm not sure how that affects copying etc., but suffice to say that if you try the following: BigInt a = 2; BigInt b = a; b = 3; assert(a != b); assert(a !is b); ... then it passes. So it behaves at least in this extent like a value type. I took a glance over the BigInt code, and it appears to have some kind of copy-on-write semantics. For example, in your code above, when you wrote b=a, b actually *aliases* a, but when you assign 3 to b, a new data array is created and b is updated to point to the new data instead. So it's not really a true value type, but more like a COW reference type. But suffice to say that it was an unpleasant surprise that I couldn't just take it and pass to a function accepting an unqualified BigInt argument. That only works with true value types, but BigInt isn't really one of them. :) T -- Everybody talks about it, but nobody does anything about it! -- Mark Twain
Re: The no gc crowd
On 10/10/2013 10:27 AM, Sean Kelly wrote: [...] Sean - whatever means you're using to reply breaks the thread.
Re: The no gc crowd
On Oct 10, 2013, at 12:11 PM, Walter Bright newshou...@digitalmars.com wrote: On 10/10/2013 10:27 AM, Sean Kelly wrote: [...] Sean - whatever means you're using to reply breaks the thread. The mailing list Brad set up--I can't do NNTP from most locations. I guess I'll use the website.
Re: The no gc crowd
On 10/10/2013 12:38 PM, Sean Kelly wrote: On Oct 10, 2013, at 12:11 PM, Walter Bright newshou...@digitalmars.com wrote: On 10/10/2013 10:27 AM, Sean Kelly wrote: [...] Sean - whatever means you're using to reply breaks the thread. The mailing list Brad set up--I can't do NNTP from most locations. I guess I'll use the website. I'm curious why NNTP would be blocked. I've been able to access it from any wifi hotspots I've tried it from.
Re: The no gc crowd
On Thursday, 10 October 2013 at 20:50:10 UTC, Walter Bright wrote: I'm curious why NNTP would be blocked. I've been able to access it from any wifi hotspots I've tried it from. My only guess is that usenet may be perceived as a illegal file sharing resource. But it's been a while since I've tried, so I'll give it another shot.
Re: The no gc crowd
On Thursday, 10 October 2013 at 17:36:11 UTC, Joseph Rushton
Wakeling wrote:
On 10/10/13 19:31, Jonathan M Davis wrote:
I'm honestly surprised that Andrei is rejecting the idea of
casting to/from
shared or immutable being normal given how it's required by
our current
concurrency model. And changing that would be a _big_ change.
I'm starting to incline towards the view that type
qualifications of _any_ kind become problematic once you start
working with any types other than built-in, and not just in the
context of concurrency. See e.g.:
http://d.puremagic.com/issues/show_bug.cgi?id=11148
http://d.puremagic.com/issues/show_bug.cgi?id=11188
I'd really appreciate advice on how to handle issues like
these, because it's becoming a serious obstacle to my work on
std.rational.
As qnzc pointed out - check out this thread:
http://forum.dlang.org/post/sdefkajobwcfikkel...@forum.dlang.org
Your problems with BigInt occur because the language has a
special optimization for assignment of structs with no mutable
aliasing. Fundamental math types have no aliasing so that
assignment from any to all is fine and efficient via a data copy.
For types like BigInt with mutable aliasing crossing from mutable
to immutable and back is a no-go because of the reasons pointed
out in the response to bug 11148. You can not and should not be
able to do what you are asking - pass mutable with aliasing into
immutable because then immutable would not be guaranteed.
Two options are copy BigInt beforehand if you want to keep by
value semantics on your function signatures or maybe pass by
reference (they are big after all so why copy)?
Passing by ref won't really be the full solution to your problem
on comment 6 of bug 11148. What you really want to do is take a
const(BigInt) or ref to it and make a mutable copy. So do that!
But wait, how can you do that? You need to have a dup type
function. Ideally there would be a generic way to do this. I have
one that works for most cases and including yours:
https://github.com/patefacio/d-help/blob/master/d-help/opmix/dup.d
This support could easily be put in the standard.
import std.bigint, std.stdio;
import opmix.mix;
void foo(const(BigInt) n) {
// make mutable copy
auto bi = n.gdup;
bi *= 2;
writeln(bi);
}
void main() {
const cbi = BigInt(1234567890987654321);
foo(cbi);
writeln(cbi);
}
--
2469135781975308642
1234567890987654321
Thanks,
Dan
Re: The no gc crowd
On Thursday, 10 October 2013 at 17:39:55 UTC, Sean Kelly wrote: On Oct 10, 2013, at 10:23 AM, Joseph Rushton Wakeling joseph.wakel...@webdrake.net wrote: On 09/10/13 06:25, Andrei Alexandrescu wrote: The way I see it we must devise a robust solution to that, NOT consider the state of the art immutable (heh, a pun). Must say I have had a miserable experience with immutability and any kind of complex data structure, particularly when concurrency is involved. As long as the reference itself can be reassigned (tail-immutable, I suppose) I think immutable is occasionally quite useful for complex data structures. It basically formalizes the RCU (read-copy-update) approach to wait-free concurrency. I'd tend to use this most often for global data structures built up on app start, and updated rarely to never as the program runs. Nice. Please show an example that includes complex data with associative arrays. Thanks Dan
Re: The no gc crowd
On Thursday, 10 October 2013 at 21:15:39 UTC, Sean Kelly wrote: On Thursday, 10 October 2013 at 20:50:10 UTC, Walter Bright wrote: I'm curious why NNTP would be blocked. I've been able to access it from any wifi hotspots I've tried it from. My only guess is that usenet may be perceived as a illegal file sharing resource. But it's been a while since I've tried, so I'll give it another shot. No luck. I can get to the SSL port (563) on usenet hosts that offer it, but not port 119 from anywhere I care to check news from.
Re: std.d.lexer : voting thread
On Thursday, 10 October 2013 at 17:34:01 UTC, Andrei Alexandrescu wrote: Excellent point! In fact one would need to use t!.id instead of t!. I'll work on that next. Andrei I don't suppose this new lexer is on Github or something. I'd like to help get this new implementation up and running.
Re: Improvements to std.typecons.Nullable
On 2013-10-10, 13:28, monarch_dodra wrote: On Thursday, 10 October 2013 at 10:09:23 UTC, Andrei Alexandrescu wrote: I'm confused. I thought Nullable!T == T is well defined to mean true if a value is present and equal to the right-hand side, or false otherwise (the absence of a value is a singularity unequal with all objects). What's harmful about that? Andrei That in itself, I think is actually OK. It would be OK, because I think we can agree that a no-value is different from any value. In this case, we are making a call to Nullable's opEqual. I'm actually fine with this, because it is a call to Nullable's member function. The point though is that this crashed at runtime, and nobody until now noticed it, because of the alias this. Ditto for toString. Ditto for to hash. My argument is against the alias this itself. It is making a cast when we don't actually expect it. Basically, any time you do a call on said nullable, you have to really think about what you are doing, because your nullable *will* be referenced on the first chance it gets. Basically, passing a Nullable!T to any function that expects a T is a silent runtime danger, which we really shouldn't have to accept. Hear hear! I've just played around with implementing a tagged union myself, and opted for explicit everywhere[0], with this being the preferred method for accessing the stored value: TaggedUnion!(string, float, int, Tuple!(long, long)) a; a.match!( (string s) = writeln(It's a string!), (float f) = writeln(It's a float!), (Else) = writeln(It's something else!), ); This way, I'm always forced to handle the other cases. This work also gave me a free Nullable: alias NullableT(T) = TaggedUnion!(T, typeof(null)); I admit I have not tested the latter, so it might in fact not work very well. :p Also, opEquals proved troublesome to implement, as typeof(null) is not comparable to typeof(null). Oh, and for such a generic type, should TaggedUnion!(int, string) be comparable to TaggedUnion(int, float)? [0]: If I want to play unsafe, I can also access the stored value like so: float f = a.as!float; -- Simen
Re: The no gc crowd
On Thursday, October 10, 2013 11:28:02 Walter Bright wrote: On 10/10/2013 10:54 AM, Andrei Alexandrescu wrote: On 10/10/13 12:18 AM, Walter Bright wrote: 1. Shared data cannot be passed to regular functions. I don't understand this. If a function/method accepts shared, then it can be passed shared data. I meant regular functions as in they are not typed as taking shared arguments. Shared cannot be implicitly cast to unshared. I say regular because very, very few functions are typed as accepting shared arguments. Yeah. The only times that something is going to accept shared is when it was specifically designed to work as shared (which most code isn't), or if it's templated and the template happens to work with shared. Regular functions just aren't going to work with shared without casting away shared, because that would usually mean either templating everything or duplicating functions all over the place. - Jonathan M Davis
Re: The no gc crowd
On Thursday, October 10, 2013 11:11:12 Sean Kelly wrote: On Oct 10, 2013, at 10:55 AM, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 10/10/13 12:33 AM, Jonathan M Davis wrote: Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting. no Yeah, I'd want to see this claim backed up by some examples. The only data I share globally in my own apps is the occasional container. Configuration data, a user database, whatever. I'll also frequently move data between threads while dispatching tasks, but otherwise everything is thread-local. I imagine there are other reasonable methods for using shared data, but I don't know what they are. Yeah, but it's that moving data between threads while dispatching tasks which requires casting. Pretty much anything which isn't a value type has to be cast to either immutable or shared in order to pass it across threads, and then it needs to then be cast back to thread-local mutable on the other side for to be useable. Unless you're only passing really basic stuff like int, or the types that your passing are only used for being passed across threads (and thus are designed to work as shared), you end up having to cast. The fact that you can only pass shared or immutable objects across combined with the fact that shared objects are generally unusable makes it so that you're at minimum going to have to either cast the object once it gets to the other thread, even if it was constructed as shared. And since shared is so useless means that if you need to do anything more than simply construct the object before passing it across, you're going have to have it as thread-local in the originating thread as well. I just don't see how you could avoid casting when passing ownership of an object from one thread to another without having a way to pass an object across threads without having to make it shared or immutable to pass it. - Jonathan M Davis
Re: The no gc crowd
On Thursday, 10 October 2013 at 23:33:27 UTC, Jonathan M Davis wrote: Yeah. The only times that something is going to accept shared is when it was specifically designed to work as shared (which most code isn't), or if it's templated and the template happens to work with shared. Regular functions just aren't going to work with shared without casting away shared, because that would usually mean either templating everything or duplicating functions all over the place. I think that's pretty reasonable though. Shared data needs to be treated differently, explicitly, or things go downhill fast.
Re: The no gc crowd
On Thursday, 10 October 2013 at 23:33:17 UTC, Jonathan M Davis wrote: I just don't see how you could avoid casting when passing ownership of an object from one thread to another without having a way to pass an object across threads without having to make it shared or immutable to pass it. Well, the restriction to only pass immutable and shared data is simply enforced statically by the API. So if there were an assumeUnique analog, the check could be modified to accept that as well, and then the class would arrive as unshared. This could be accomplished pretty easily. It would be yet another step towards not having thread-local pools though. I was initially pretty conservative in what was an acceptable type to send, because it's always easier to loosen restrictions than tighten them.
Re: The no gc crowd
On 2013-10-10, 20:28, H. S. Teoh wrote:
On Thu, Oct 10, 2013 at 07:36:06PM +0200, Joseph Rushton Wakeling wrote:
On 10/10/13 19:31, Jonathan M Davis wrote:
I'm honestly surprised that Andrei is rejecting the idea of casting
to/from shared or immutable being normal given how it's required by
our current concurrency model. And changing that would be a _big_
change.
I'm starting to incline towards the view that type qualifications of
_any_ kind become problematic once you start working with any types
other than built-in, and not just in the context of concurrency.
See e.g.:
http://d.puremagic.com/issues/show_bug.cgi?id=11148
http://d.puremagic.com/issues/show_bug.cgi?id=11188
I'd really appreciate advice on how to handle issues like these,
because it's becoming a serious obstacle to my work on std.rational.
I left some comments on these bugs. Basically, BigInt should not be
implicitly castable from const/immutable to unqual, because unlike the
built-in types, it's *not* a value type:
[snip]
What you need to do is to use inout for functions that need to handle
both built-in ints and BigInts, e.g.:
[snip]
Here's a COW reference type that I can easily pass to a function
requiring a mutable version of the type:
struct S {
immutable(int)[] arr;
}
And usage:
void foo(S s) {}
void main() {
const S s;
foo(s);
}
This compiles and works beautifully. Of course, no actual COW is
happening here, but COW is what the type system says has to happen.
Another example COW type:
string;
Now, my point here is that BigInt could easily use an immutable
buffer internally, as long as it's purely COW. It could, and it should.
If it did, we would not be having this discussion, as bugs #11148 and
#11188 would not exist. Inventing rules like 'you should use inout'
does not help - it's obscuring the problem.
TLDR: Do not use inout(T). Fix BigInt.
--
Simen
Re: The no gc crowd
On Thursday, October 10, 2013 10:55:49 Andrei Alexandrescu wrote: On 10/10/13 12:33 AM, Jonathan M Davis wrote: I honestly don't think we can solve it a different way without completely redesigning shared. shared is specifically designed such that you have to either cast it way to do anything with it no or write all of your code to explicitly work with shared, which is not something that generally makes sense to do unless you're creating a type whose only value is in being shared across threads. yes Really? Do you honestly expect the average use of shared to involve creating structs or classes which are designed specifically to be used as shared? That definitely has its use, but I would expect it to be far more common that someone would want to share the exact same types that they're using in their thread-local code. In fact, if anything, the normal responses to discussions on shared go in the complete opposite direction of creating classes which are designed to work as shared. It seems like the normal thing to do is simply avoid shared altogether and use __gshared so that you don't have to deal with any of the problems that shared causes. Granted, I obviously haven't seen everyone's code, but I don't believe that I have ever seen anyone create a type designed to be used as shared, and that's certainly not what people discuss doing when shared comes up. TDPL discusses that - and again, I do think that that has its place - but I've never seen it done, and I've never run into any place in my own code where I would have even considered it. Usually, you want to share an object of the same type that you're using in your thread-local code. And even if a struct or class is set up so that its member functions work great as shared, very little code seems to be written with shared in mind (since thread-local is the default), so the only functions which will work with it are its member functions, functions written specifically to work with that type, and templated functions that happen to work with shared. As such, I fully expect casting away shared to be a very common idiom. Without that, the number of things you can do with a shared object is very limited. Far more frequently, you want to share a type which you would also use normally as a thread-local variable, and that means casting. no What else do you expect to be doing with std.concurrency? That's what it's _for_. Unless all of the stuff that you're passing across threads are value types or are designed to work as immutable or shared (which most types aren't), the objects which get passed across need to be cast to thread-local mutable on the target thread in order to be used there, and if you have to do much of anything with the object other than constructing it before passing it across, then you're going to have to have it as thread-local on the originating thread as well, because most functions are going to be unusable with shared. - Jonathan M Davis
Re: The no gc crowd
On 10/10/13 10:36 AM, Joseph Rushton Wakeling wrote: On 10/10/13 19:31, Jonathan M Davis wrote: I'm honestly surprised that Andrei is rejecting the idea of casting to/from shared or immutable being normal given how it's required by our current concurrency model. And changing that would be a _big_ change. I'm starting to incline towards the view that type qualifications of _any_ kind become problematic once you start working with any types other than built-in, and not just in the context of concurrency. See e.g.: http://d.puremagic.com/issues/show_bug.cgi?id=11148 http://d.puremagic.com/issues/show_bug.cgi?id=11188 I'd really appreciate advice on how to handle issues like these, because it's becoming a serious obstacle to my work on std.rational. I'll look into this soon. Andrei
