Re: Wanted: 128 bit integers
"Manu" wrote in message news:mailman.657.1331715447.4860.digitalmar...@puremagic.com... > > Why aren't they implemented in a library for the time being at least, so > code can compile and work? > The obvious answer is because nobody's done it yet. I have a branch that partially implements ucent in the dmd backend (somewhere in my fork on github), but it will likely be at least a month before I have time to work on it again. It looks like it will be pretty easy on 64 bit.
Re: Wanted: 128 bit integers
"Jonathan M Davis" wrote in message news:mailman.685.1331774722.4860.digitalmar...@puremagic.com... > On Thursday, March 15, 2012 11:55:46 Daniel Murphy wrote: >> "Jonathan M Davis" wrote in message >> news:mailman.666.1331749219.4860.digitalmar...@puremagic.com... >> >> > long long is 128 bits on 64-bit Linux. >> >> Are you sure about this? I think we already had this discussion... > > If it isn't, then whatever that discussion was obviously didn't stick in > my > head. > > - Jonathan M Davis long double (aka real, a much better name) is 128 bits (including padding) on some platforms/compilers, I'm pretty sure long long is 64 bits in every x86-64 memory model. Some compilers do however support 128 bit integer types on some platforms.
Idea for D Conference talk
Just this but for D: http://www.youtube.com/watch?v=XGF3Qu4dUqk Hastings just steps through the python interpreter attached to gdb (not live) and explains the structure of CPython as he goes. It's extremely informative for would-be CPython hackers. If anyone does it, be sure to record it and post it online :). Regards, Brad Anderson.
Re: Standalone AA implementation ready for review (Was: Re: Replacing
On Wed, Mar 14, 2012 at 08:47:23PM -0400, bearophile wrote: > H. S. Teoh: > > > - I haven't stress-tested AA performance yet. (Any volunteers? ;-)) > > I have just seen that the performance of the Sokoban solver is > unchanged. I've just run the Sokoban solver in a loop that runs the solve 20 times in a row. I consistently get around 7 seconds in total for both the current aaA.d AA implementation and my new implementation. So far it they seem to perform about the same. > > In spite of its flaws, this implementation already addresses quite a > > few AA-related issues in the bugtracker, > > How efficiently is this AA working with keys like: > AssociativeArray!(immutable char[60], bool) aa; > Built-in AAs become snail-slow with such keys. [...] I'll try something like this next. Thanks for the feedback! T -- PNP = Plug 'N' Pray
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On Wed, Mar 14, 2012 at 11:58:01PM -0500, Andrei Alexandrescu wrote: > On 3/14/12 6:16 PM, H. S. Teoh wrote: > >- Declaring an AA with non-const array keys will cause reams and reams > > of compile errors. I'm not *too* worried about this at the moment > > since it doesn't make sense to have non-const AA keys anyway. I'm > > also seriously considering forcing *all* AA keys to be immutable, > > in which case this will become a non-issue. > > I think the built-in associative array must allow non-constant keys. > As long as there's no memory safety issue, the AA should work with > types that the user doesn't change for comparison purposes but can > otherwise modify. > > A practical matter is that if we introduce this restriction we'll > break a ton of code. [...] Understood. But if the user changes the keys then the AA will malfunction. (This is no worse than the current behaviour, I suppose.) So now I've to track down why non-const keys trigger a ton of errors... :-/ T -- "No, John. I want formats that are actually useful, rather than over-featured megaliths that address all questions by piling on ridiculous internal links in forms which are hideously over-complex." -- Simon St. Laurent on xml-dev
Re: Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On 3/14/12 6:16 PM, H. S. Teoh wrote: - Declaring an AA with non-const array keys will cause reams and reams of compile errors. I'm not *too* worried about this at the moment since it doesn't make sense to have non-const AA keys anyway. I'm also seriously considering forcing *all* AA keys to be immutable, in which case this will become a non-issue. I think the built-in associative array must allow non-constant keys. As long as there's no memory safety issue, the AA should work with types that the user doesn't change for comparison purposes but can otherwise modify. A practical matter is that if we introduce this restriction we'll break a ton of code. Andrei
Re: Standalone AA implementation ready for review (Was: Re: Replacing
On Wed, Mar 14, 2012 at 08:47:23PM -0400, bearophile wrote: > H. S. Teoh: > > > - I haven't stress-tested AA performance yet. (Any volunteers? ;-)) > > I have just seen that the performance of the Sokoban solver is > unchanged. > > > > In spite of its flaws, this implementation already addresses quite a > > few AA-related issues in the bugtracker, > > How efficiently is this AA working with keys like: > AssociativeArray!(immutable char[60], bool) aa; > Built-in AAs become snail-slow with such keys. [...] Well, I didn't invent any clever new algorithms or anything like that, I more-or-less followed the same implementation as aaA.d. So I don't think there will be any great performance gains. I'm more concerned about drastic performances *losses* caused by code that I didn't realize would cause a problem, perhaps an overlooked inefficiency that crept in when I was translating the AA code into template-based code. T -- Being able to learn is a great learning; being able to unlearn is a greater learning.
Re: Standalone AA implementation ready for review
James Miller: Any tips on where to start? You have a lot of search space to cover, so before doing a systematic search that requires a lot of time I suggest a quick random sampling of the search space. If there are significantly wide performance inefficiencies you will hit them in less than one hour of tests. Such approach is sometimes missed even by professional engineers :-) I have already given a link to a game solver that uses associative arrays a lot. Then, this is my currently "best short solution" of one of the Shootout benchmarks that stresses only hashing. The input data comes from the Fasta benchmark (just use the Python or C++ Fasta code if you want the test data): http://codepad.org/ZSlrcuad I have few other benchmarks. Bye, bearophile
Re: Wanted: 128 bit integers
On Thursday, March 15, 2012 11:55:46 Daniel Murphy wrote: > "Jonathan M Davis" wrote in message > news:mailman.666.1331749219.4860.digitalmar...@puremagic.com... > > > long long is 128 bits on 64-bit Linux. > > Are you sure about this? I think we already had this discussion... If it isn't, then whatever that discussion was obviously didn't stick in my head. - Jonathan M Davis
Re: Wanted: 128 bit integers
On 15 March 2012 13:55, Daniel Murphy wrote: > "Jonathan M Davis" wrote in message > news:mailman.666.1331749219.4860.digitalmar...@puremagic.com... >> >> long long is 128 bits on 64-bit Linux. >> > > Are you sure about this? I think we already had this discussion... > > Same discussion, different topic. I think the one before was sparked by size_t issues. -- James Miller
Re: Wanted: 128 bit integers
"Jonathan M Davis" wrote in message news:mailman.666.1331749219.4860.digitalmar...@puremagic.com... > > long long is 128 bits on 64-bit Linux. > Are you sure about this? I think we already had this discussion...
Re: Standalone AA implementation ready for review (Was: Re: Replacing
On 15 March 2012 13:47, bearophile wrote: > H. S. Teoh: > >> - I haven't stress-tested AA performance yet. (Any volunteers? ;-)) > > I have just seen that the performance of the Sokoban solver is unchanged. > > >> In spite of its flaws, this implementation already addresses quite a few >> AA-related issues in the bugtracker, > > How efficiently is this AA working with keys like: > AssociativeArray!(immutable char[60], bool) aa; > Built-in AAs become snail-slow with such keys. > > Bye, > bearophile Hmm, I might code up a stress-test suite for this if I get time/bored. Any tips on where to start? Obviously you want things like lots of entries, but is it important to test big Keys, particular types of keys, that sort of thing. I'll also probably do separate speed and memory benchmarks. -- James Miller
Re: Replacing AA's in druntime
On Wednesday, 14 March 2012 at 23:51:30 UTC, H. S. Teoh wrote:
On Thu, Mar 15, 2012 at 12:20:43AM +0100, Jakob Bornecrantz
wrote:
[...]
struct AAver1(K, V)
{
K[] tbl; V[] tlb2; uint size;
}
struct AAver2(K, V)
{
K[] tbl; V[] tbl2; V[] optimizationTbl;
}
Would break if a AAver1 table was ever passed to code that
was compiled against a AAver2 table. In sort you could never
add anything to the AA struct. Without going in roundabout
ways of making sure you never access outside of any struct
version ever out there.
Or for that matter change how the internal tables are
populated by add and remove.
[...]
How is this different from any other templates in
druntime/phobos?
Its not, I at least like to see it being possible to keep a
stable ABI in druntime, and as such making it possible to
share inbuilt language features like AA across libraries,
phobos is a lost cause.
And FYI, the current AA implementation *already* suffers
from this problem, because the Range interface already
assumes a specific implementation behind the opaque pImpl
pointer (see object_.d -- it *duplicates* the struct
definitions from aaA.d and casts the void* into pointers to
those structs). If aaA.d were to change its implementation
today, the Range stuff in struct AssociativeArray would break
horribly.
The motivation behind my rewriting AA's is to fix this
schizophrenic mess. The internal aaA.d structs should *not*
be duplicated in object_.d, but currently they are. So there
are two options, either (1) we move everything back into aaA.d
(and introduce a whole bunch more void* pImpl and C-linkage
functions for the structs that Range support requires), or
(2) we move everything out of aaA.d.
Yes that is bad, I of course would like to see (2). I don't
feel that just because I wasn't around to hit people over
the head when this was introduced as reason for losing the
ability of D libraries, or making it even harder for people
to do them.
Personally, I feel the second option is better. If we want
to improve or add to AA's API, we can just change it in
object_.d. Key and value types are directly accessible, so
we never have to play around with typeinfos and pointer
arithmetic to do simple stuff.
If we go with the first option, every little change will
require changes to aaA.d, and trying to add new functionality
will constantly introduce new C-linkage functions in aaA.d,
new void* pImpl's in object_.d, with the associated hacks
using typeinfos (because Key/Value types are essentially
opaque to aaA.d, so you have to rely on typeinfos and pointer
arithmetic instead of letting the compiler figure it out for
you). This is very hard to maintain, and much more bug-prone.
I'm assuming you mean "changes to aaA.d and _object.d".
I'm glad you listed cons of doing it as a opaque pointer,
I don't agree completely that it will be that much more
difficult to maintain or be that much more buggy.
Yes the current one of doing it in DMD, _object.d and in
druntime is bad and can be improved. Did you see my response
to Don? What do you think of that?
Yes there will be a cost to doing it the second way, but
we as a language provider are taking that cost so that
our users don't have to take it via being locked to the
exact same druntime/compiler version between libraries.
Yes doing a stable ABI isn't easy and free, so I think we
need to weigh the pros and cons of supporting language
level primitives easily (for users) across library
boundaries. I think we should just htfu[1] and provide
this feature.
I just wanted to make sure people understand what they are
loosing and what this means. And that they should't have
any illusions of things just magically working. I don't see
me changing my view on thism I want this feature, if I don't
get sure I'll be sad and D will be less attractive to me but
it isn't the end of the world. I want to thank you again for
taking the time to explain everything to me and taking the
time to respond.
What I really want to get to is: Do we really want to do this?
Cheers, Jakob.
[1] http://www.youtube.com/watch?v=unkIVvjZc9Y
Re: Standalone AA implementation ready for review (Was: Re: Replacing
H. S. Teoh: > - I haven't stress-tested AA performance yet. (Any volunteers? ;-)) I have just seen that the performance of the Sokoban solver is unchanged. > In spite of its flaws, this implementation already addresses quite a few > AA-related issues in the bugtracker, How efficiently is this AA working with keys like: AssociativeArray!(immutable char[60], bool) aa; Built-in AAs become snail-slow with such keys. Bye, bearophile
Re: Negative integer modulo/division
http://www.digitalmars.com/d/archives/digitalmars/D/Could_we_have_mod_in_std.math_152977.html
Re: Standalone AA implementation ready for review (Was: Re: Replacing
H. S. Teoh: > - As mentioned above, wstring or dstring keys (in fact any array keys > with literals that doesn't default to the key type) need an explicit > suffix when indexing with literals. I haven't figured out how to make > this work yet. At the very least, wstring and dstring needs to > support suffixless key literals; it would be nice if a general > solution could be found for all array-typed keys. This might be a problem worth fixing generally, if possible, so it's solved for other library-defined collections too. > I'm also > seriously considering forcing *all* AA keys to be immutable, in which > case this will become a non-issue. I am for not mutable keys. > - Some code (most notably that table of primes that I hoisted from > aaA.d) may need to be refactored to prevent excessive code bloat when > the AA template is instantiated. Ah, another usage for my "static static" (or for the @templated()). I'll count how many times they come out useful :-) > It would also be > nice if there was a uniform syntax for getting the hash of a type > without going through typeid (may be more efficient?). I agree! > - I haven't stress-tested AA performance yet. (Any volunteers? ;-)) I have several benchmarks, but you can start quickly adapting this one, comparing the performance of the built-in AAs with your ones: http://rosettacode.org/wiki/Sokoban#D Bye, bearophile
Re: Replacing AA's in druntime
On Thu, Mar 15, 2012 at 12:20:43AM +0100, Jakob Bornecrantz wrote:
[...]
> struct AAver1(K, V)
> {
>K[] tbl; V[] tlb2; uint size;
> }
>
> struct AAver2(K, V)
> {
>K[] tbl; V[] tbl2; V[] optimizationTbl;
> }
>
> Would break if a AAver1 table was ever passed to code that
> was compiled against a AAver2 table. In sort you could never
> add anything to the AA struct. Without going in roundabout
> ways of making sure you never access outside of any struct
> version ever out there.
>
> Or for that matter change how the internal tables are
> populated by add and remove.
[...]
How is this different from any other templates in druntime/phobos?
And FYI, the current AA implementation *already* suffers from this
problem, because the Range interface already assumes a specific
implementation behind the opaque pImpl pointer (see object_.d -- it
*duplicates* the struct definitions from aaA.d and casts the void* into
pointers to those structs). If aaA.d were to change its implementation
today, the Range stuff in struct AssociativeArray would break horribly.
The motivation behind my rewriting AA's is to fix this schizophrenic
mess. The internal aaA.d structs should *not* be duplicated in
object_.d, but currently they are. So there are two options, either (1)
we move everything back into aaA.d (and introduce a whole bunch more
void* pImpl and C-linkage functions for the structs that Range support
requires), or (2) we move everything out of aaA.d.
Personally, I feel the second option is better. If we want to improve or
add to AA's API, we can just change it in object_.d. Key and value types
are directly accessible, so we never have to play around with typeinfos
and pointer arithmetic to do simple stuff.
If we go with the first option, every little change will require changes
to aaA.d, and trying to add new functionality will constantly introduce
new C-linkage functions in aaA.d, new void* pImpl's in object_.d, with
the associated hacks using typeinfos (because Key/Value types are
essentially opaque to aaA.d, so you have to rely on typeinfos and
pointer arithmetic instead of letting the compiler figure it out for
you). This is very hard to maintain, and much more bug-prone.
T
--
"I suspect the best way to deal with procrastination is to put off the
procrastination itself until later. I've been meaning to try this, but haven't
gotten around to it yet. " -- swr
Re: Replacing AA's in druntime
On Wednesday, 14 March 2012 at 14:02:30 UTC, Steven Schveighoffer
wrote:
On Tue, 13 Mar 2012 22:39:25 -0400, Jakob Bornecrantz
wrote:
On Wednesday, 14 March 2012 at 00:52:32 UTC, H. S. Teoh wrote:
Hi all,
My AA implementation is slowly inching closer to being ready
to replace aaA.d. So far I've been writing the implementation
outside of object_.d for ease of testing & development; now
I'm
ready to start moving stuff into object_.d to start working on
integration with druntime.
Hi,
If I'm understanding this correctly you are moving the entire
implementation of the AA into object.d and as such letting
programs be purview to its inner working? In sort meaning you
are making the entire AA implementation D ABI locked.
This will make it impossible to either change the AA
implementation in any ABI breaking fashion or make it
impossible
to pass AA's between libraries compiled against different
versions of druntime.
Is this what we really want?
This is unavoidable, whether it's a template or not. What
changes do you envision would be transparent using an opaque
pImpl model (as was done in previous versions of phobos), but
would break using templates?
struct AAver1(K, V)
{
K[] tbl; V[] tlb2; uint size;
}
struct AAver2(K, V)
{
K[] tbl; V[] tbl2; V[] optimizationTbl;
}
Would break if a AAver1 table was ever passed to code that
was compiled against a AAver2 table. In sort you could never
add anything to the AA struct. Without going in roundabout
ways of making sure you never access outside of any struct
version ever out there.
Or for that matter change how the internal tables are
populated by add and remove.
Cheers, Jakob.
Standalone AA implementation ready for review (Was: Re: Replacing AA's in druntime)
On Tue, Mar 13, 2012 at 09:30:45PM -0500, Andrei Alexandrescu wrote: > On 3/13/12 7:54 PM, H. S. Teoh wrote: > >Hi all, > > > >My AA implementation is slowly inching closer to being ready to > >replace aaA.d. > > Great! This will need compiler restructuring, and in fact offers the > perfect opportunity for it. I suggest you to post your implementation > here for review first, and assume only the minimal lowerings from the > compiler. Alright, I've finished the basic functionality of my AA implementation. I still haven't solved that problem with using suffixless string literals to index X[dstring], so you'll have to write aa["abc"d] instead of just aa["abc"]. But I thought I should post my code now so that people can take a look at it: https://github.com/quickfur/New-AA-implementation/blob/master/newAA.d Currently, this code is still standalone, not integrated with druntime yet. Since that will require compiler changes and is potentially a very big change, I've decided to polish up the standalone version as much as possible before attempting druntime integration. There are still some outstanding issues: - As mentioned above, wstring or dstring keys (in fact any array keys with literals that doesn't default to the key type) need an explicit suffix when indexing with literals. I haven't figured out how to make this work yet. At the very least, wstring and dstring needs to support suffixless key literals; it would be nice if a general solution could be found for all array-typed keys. - Declaring an AA with non-const array keys will cause reams and reams of compile errors. I'm not *too* worried about this at the moment since it doesn't make sense to have non-const AA keys anyway. I'm also seriously considering forcing *all* AA keys to be immutable, in which case this will become a non-issue. - Some code (most notably that table of primes that I hoisted from aaA.d) may need to be refactored to prevent excessive code bloat when the AA template is instantiated. - Array literals don't work yet (requires compiler support). - X[Y] syntax doesn't work yet (requires druntime integration & compiler support); you have to explicitly declare AssociativeArray!(Y,X). - The use of typeid.getHash. This is probably OK, except that it forces a lot of AA methods not to be pure nothrow @safe. It would also be nice if there was a uniform syntax for getting the hash of a type without going through typeid (may be more efficient?). - I haven't stress-tested AA performance yet. (Any volunteers? ;-)) Some improvement in this area will probably be needed, though we may not need to address that until this is ready for druntime integration. - (There's also some temporary development-only code, syntactic sugar aliases, that shouldn't end up in object_.d -- those will be removed when I'm ready for druntime integration. There's also some code that needs a bit of cleanup.) In spite of its flaws, this implementation already addresses quite a few AA-related issues in the bugtracker, as verified by a bunch of unittests (search for "Issue" in the code). Comments? Flames? ;-) Pull requests?? :-D T -- It is the quality rather than the quantity that matters. -- Lucius Annaeus Seneca
Re: Tuple unpacking syntax [Was: Re: Multiple return values...]
On Wednesday, 14 March 2012 at 13:17:47 UTC, Robert Jacques wrote: But there's a reason we use /// instead of ⫻; we shouldn't require custom keyboard mappings in order to program efficiently in D. Aren't we supposed to be moving towards more natural interfaces in computing? I'm sure that once the touch interface is perfected for text input it would be trivial to have a "coding keyboard" or a "D keyboard". The keyboard mappings would simply come bundled with the compiler. The current text based programming is quite limiting considering that we actually deal with a tree of tokens. IDEs already manipulate code at the AST level in order to enable refactoring. The next logical step would be to eliminate the text form all together and store code at the AST level, thus avoiding lexing/parsing overhead and the limits of text based representation. e.g. subtextual.org has cool ideas how to design such a future non textual language. For example, representing Boolean logic in a table instead of arbitrary nested if statements.
Re: Replacing AA's in druntime
On Wednesday, 14 March 2012 at 13:55:23 UTC, Don Clugston wrote:
On 14/03/12 03:39, Jakob Bornecrantz wrote:
On Wednesday, 14 March 2012 at 00:52:32 UTC, H. S. Teoh wrote:
Hi all,
My AA implementation is slowly inching closer to being ready
to
replace aaA.d. So far I've been writing the implementation
outside of object_.d for ease of testing & development; now
I'm
ready to start moving stuff into object_.d to start working on
integration with druntime.
Hi,
If I'm understanding this correctly you are moving the entire
implementation of the AA into object.d and as such letting
programs be purview to its inner working? In sort meaning you
are making the entire AA implementation D ABI locked.
This will make it impossible to either change the AA
implementation in any ABI breaking fashion or make it
impossible
to pass AA's between libraries compiled against different
versions of druntime.
Much less so than the existing AA implementation.
In what way moving the entire implementation into templates
that gets compiled into the object code of all the uses, got
to be worse (for ABI) then having the implementation separate?
I think you are not seeing the implementation off calling
into the opaque implementation as two separate things. Yes the
current implementation off calling into is bad and should be
replaced.
Ponder this for example:
struct AA(K, V)
{
void *ptr;
void init() { ptr = rt_aaInit(typeinfo!K, typeinfo!V; }
static if (isData!K)
void add(K k, V ref v) { rt_aaAddData(ptr, cast(ulong)k, v);
}
else static if (isPointer!K)
void add(K k, V ref v) { rt_aaAddPointerWithHash(ptr, k,
toHash!K(k), v); }
else
void add(K ref k, V ref v) { rt_aaAdd(ptr, k, v); }
}
Adding to this implementation wouldn't require changing DMD
at all only _object.d and the aa implementation in druntime.
The use of the AA struct isn't required by the ABI, only
that functions gets called for doing things to a AA.
Cheers, Jakob.
Re: Replacing AA's in druntime
On Wednesday, 14 March 2012 at 09:07:40 UTC, Dmitry Olshansky wrote: On 14.03.2012 6:39, Jakob Bornecrantz wrote: On Wednesday, 14 March 2012 at 00:52:32 UTC, H. S. Teoh wrote: Hi all, My AA implementation is slowly inching closer to being ready to replace aaA.d. So far I've been writing the implementation outside of object_.d for ease of testing & development; now I'm ready to start moving stuff into object_.d to start working on integration with druntime. Hi, If I'm understanding this correctly you are moving the entire implementation of the AA into object.d and as such letting programs be purview to its inner working? In sort meaning you are making the entire AA implementation D ABI locked. This will make it impossible to either change the AA implementation in any ABI breaking fashion or make it impossible to pass AA's between libraries compiled against different versions of druntime. I will just point out that the major point of ABI is to make sure different D compiler produce compatible object code. Thus it makes AA implementation locked already anyway, right? Not true, as Steven said a opaque pImpl implementation would work, most modern C library design follow this principle with only using opaque except for pointers. This is because the ABI will then only require you call a certain set of functions (that can be extended to) not what that pointers points to, the details can be completely hidden. Cheers, Jakob.
Re: Multiple return values...
On Wed, 14 Mar 2012 22:52:26 +0100, Andrei Alexandrescu
wrote:
On 3/14/12 3:00 PM, Simen Kjærås wrote:
template to(T...) {
alias T to;
}
auto from(T...)(T t) {
struct Result { T t; alias t this; }
return Result( t );
}
void main( ) {
int a = 3;
int b = 4;
to!(a, b) = from(b, a);
assert( a == 4 );
assert( b == 3 );
}
I got reborn inside a little when seeing this code. Congratulations!
Andrei
Thank you, and you're welcome.
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 14:42:54 -0700, Alvaro wrote: El 14/03/2012 22:13, Adam Wilson escribió: > On Wed, 14 Mar 2012 13:45:13 -0700, Alvaro > wrote: > > The problem is that in DI generation, at least as near as I can tell, D > has now idea how big a function is during DI generation. In my > experience it was keeping all functions not just small ones. And frankly > DI generation is targeted at library builders who are well aware of the > inlining trade-offs. And then comes the question of how do you define > "small functions" to an adequately technical level.> OK, I rechecked. DMD -H is at least omitting function bodies containing foreach() loops (and maybe other cases). Maybe that led me to assume things. Hmmm, I hadn't seen it do this. But the whole point of DI files is to create an interface to compiled libraries, not be the absolute fastest it can be. It is generally accepted that you are going to take a minuscule (and on modern machines it is truly minuscule) hit every time you cross a library boundary, unless you import the whole source of said library, which is functionally what DI generation does now. AFAIK some C++ compilers use heuristics (don't really know what that is exactly) to decide what "small functions" deserved automatic inlining. Those heuristics are in the semantic analysis step and I really don't think building the DI file against the constantly changing AST during semantic analysis is a wise idea. DI's get built before that, and as such do no have access to those heuristics. If the compiler is doing something it's based on its own special, and limited analysis prior to the semantic analysis (I think I know which function is doing this, but it doesn't work very well at all). Also, trying to build DI's during semantic analysis would most likely drastically slow down the compiler as the DI's would need to be constantly updated to make the AST changes. Basically, it's an incredible amount of work for a minuscule speed improvement in the final step. There are areas of D that are MUCH more ripe for speed improvements than inlining in DI files ... *cough*GC*cough* And well, C++ library builders are aware of that and usually keep the small functions they want inlined in the header file (e.g. in the class declaration). I can see a lot of cases where D libraries would benefit from allowing some of their functions to be inlined. Think of @properties, they're often pretty short. Well, that's where the do-it-yourself part into play. This is an automated system for generating headers based on a naive AST. It will not be perfect. But C++ doesn't even have that and they have to hand maintain everything, so pretty much anything we do is an improvement. :-) @properties are a valid concern, and also one that can be addressed in the current method of DI generation as they are a valid token that can be checked for and excluded. If the community wishes for @properties to retain their implementations that would an easy thing to do. But be warned, you wont be able to call private functions as those are dropped COMPLETELY from the DI file. Private data will of course be accessible. Other than this detail, the improvement in DI generation is very welcome. -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: Multiple return values...
On Thu, 15 Mar 2012 08:52:26 +1100, Andrei Alexandrescu
wrote:
On 3/14/12 3:00 PM, Simen Kjærås wrote:
template to(T...) {
alias T to;
}
auto from(T...)(T t) {
struct Result { T t; alias t this; }
return Result( t );
}
void main( ) {
int a = 3;
int b = 4;
to!(a, b) = from(b, a);
assert( a == 4 );
assert( b == 3 );
}
I got reborn inside a little when seeing this code. Congratulations!
And I died a little ... I have no idea what this code is doing. What is
the generated code produced by this and why?
I'd like to break the nexus between science and magic here.
--
Derek
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wed, Mar 14, 2012 at 05:35:04PM -0400, Steven Schveighoffer wrote: [...] > I get that. What I was saying is, I thought even the signal handler > uses the stack (thereby it would abort if invalid). And even if it > doesn't, simply accessing the stack by loading it into a register > should be sufficient to "test" and see if the stack is valid to use > (i.e. cause another SEGV inside the signal handler forcing an abort > so we don't have an infinite loop). That's a good idea. So the signal handler reads the top of the stack into EAX (since we're already overwriting EAX anyway), and if the stack is invalid, that will segfault and abort the program. If that doesn't abort, then assume the stack is valid and proceed with the hack to divert the return address to the throwing handler. However, this still assumes that ESP is either valid or null. If the segfault was caused by, say, an exploit attempt, then ESP may be non-null but not pointing to a valid stack either. It's conceivable that someone might try to exploit a D program by crafting a bad stack and pointing ESP at it, then triggering a segfault intentionally. (The bad stack could, for example, contain strange stack frames that causes the stack unwinder to do something unintended, like execute arbitrary code.) I don't know how to solve this, though. T -- There is no gravity. The earth sucks.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On 14/03/12 21:59, Sean Kelly wrote:
On Mar 14, 2012, at 1:54 PM, FeepingCreature wrote:
I think that case is sufficiently rare that it'd have to count somewhere between "act of
god" and "outright developer malice". The assumption that the stack frame is valid
is, I'd say, safe to make in the vast majority of cases. You pretty much have to actively try to
break it, for no clearly discernible reason.
The prevalence of buffer overflow attacks might suggest otherwise.
void foo()
{
bar();
}
void bar()
{
int y;
int *p = &y;
p[1] = 0;
}
The assignment to p[1]=0 clobbers the location where EBP was pushed.
Then:
mov ESP, EBP; // ESP is OK
pop EBP;// EBP is now 0
ret;
now return to foo, where we get:
call bar;
-> mov ESP, EBP; // ESP is now 0
pop EBP;// segfault
ret
Unfortunately it's not difficult to corrupt ESP.
Re: Multiple return values...
On 3/14/12 3:00 PM, Simen Kjærås wrote:
template to(T...) {
alias T to;
}
auto from(T...)(T t) {
struct Result { T t; alias t this; }
return Result( t );
}
void main( ) {
int a = 3;
int b = 4;
to!(a, b) = from(b, a);
assert( a == 4 );
assert( b == 3 );
}
I got reborn inside a little when seeing this code. Congratulations!
Andrei
Re: DI Generation Needs your Help!
El 14/03/2012 22:13, Adam Wilson escribió: > On Wed, 14 Mar 2012 13:45:13 -0700, Alvaro > wrote: > > The problem is that in DI generation, at least as near as I can tell, D > has now idea how big a function is during DI generation. In my > experience it was keeping all functions not just small ones. And frankly > DI generation is targeted at library builders who are well aware of the > inlining trade-offs. And then comes the question of how do you define > "small functions" to an adequately technical level.> OK, I rechecked. DMD -H is at least omitting function bodies containing foreach() loops (and maybe other cases). Maybe that led me to assume things. AFAIK some C++ compilers use heuristics (don't really know what that is exactly) to decide what "small functions" deserved automatic inlining. And well, C++ library builders are aware of that and usually keep the small functions they want inlined in the header file (e.g. in the class declaration). I can see a lot of cases where D libraries would benefit from allowing some of their functions to be inlined. Think of @properties, they're often pretty short. Other than this detail, the improvement in DI generation is very welcome.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wed, 14 Mar 2012 17:25:28 -0400, deadalnix wrote: Le 14/03/2012 21:53, Steven Schveighoffer a écrit : On Wed, 14 Mar 2012 16:45:49 -0400, Don Clugston wrote: On 14/03/12 21:31, Steven Schveighoffer wrote: On Wed, 14 Mar 2012 16:08:29 -0400, Don Clugston wrote: Now, your user space handler will cause another segfault when it does the mov [ESP], 0. I think that gives you an infinite loop. SEGFAULT inside a SEGV signal handler aborts the program (no way to turn this off IIRC). -Steve But you're not inside the signal handler when it happens. You returned. Then how does the signal handler do anything? I mean, doesn't it need a stack? Or does it just affect register variables? Most signal handlers are normal functions, and isn't there some usage of the stack to save registers? It seems there should be a way to turn off the signal handler during the time when you are suspicous of the stack being the culprit, then re-engage the signal handler before throwing the error. -Steve The address of the instruction being executed is hijacked, so, instead of resuming normal operation after the signal handler exit, it get into the throwing handler. This is a very nice trick ! I get that. What I was saying is, I thought even the signal handler uses the stack (thereby it would abort if invalid). And even if it doesn't, simply accessing the stack by loading it into a register should be sufficient to "test" and see if the stack is valid to use (i.e. cause another SEGV inside the signal handler forcing an abort so we don't have an infinite loop). I honestly don't know enough to really be discussing, but it seems like a really neat idea, and I grasp how it works. I just don't know all the particulars of signal calling conventions. -Steve
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 14:17:28 -0700, Adam D. Ruppe wrote: Adam Wilson: I think I said this before, but for what it's worth, I think your new approach is spot-on correct. Thank you. More importantly though, Walter and Andrei also agree, so in the end, the inlinaholics can just deal. :-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 14:23:19 -0700, Andrej Mitrovic wrote: On 3/14/12, Adam Wilson wrote: I would fix the bug myself, but I am getting married in a month and am rather short on time. That's ok I'm in no hurry just curious. Congrats on getting married! :) Thank you! Hehe, well, I have big plans for D, but I need DI files to work better to execute them, so I am a little bit of a hurry, but real life wins for now. ;-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 21:59, Sean Kelly a écrit : On Mar 14, 2012, at 1:54 PM, FeepingCreature wrote: I think that case is sufficiently rare that it'd have to count somewhere between "act of god" and "outright developer malice". The assumption that the stack frame is valid is, I'd say, safe to make in the vast majority of cases. You pretty much have to actively try to break it, for no clearly discernible reason. The prevalence of buffer overflow attacks might suggest otherwise. And as a stack overflow is likely to create a SEGFAULT too, we are doomed !
Re: DI Generation Needs your Help!
On 3/14/12, Adam Wilson wrote: > I would fix the bug myself, but I am getting married in a month and am > rather short on time. That's ok I'm in no hurry just curious. Congrats on getting married! :)
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 21:53, Steven Schveighoffer a écrit : On Wed, 14 Mar 2012 16:45:49 -0400, Don Clugston wrote: On 14/03/12 21:31, Steven Schveighoffer wrote: On Wed, 14 Mar 2012 16:08:29 -0400, Don Clugston wrote: Now, your user space handler will cause another segfault when it does the mov [ESP], 0. I think that gives you an infinite loop. SEGFAULT inside a SEGV signal handler aborts the program (no way to turn this off IIRC). -Steve But you're not inside the signal handler when it happens. You returned. Then how does the signal handler do anything? I mean, doesn't it need a stack? Or does it just affect register variables? Most signal handlers are normal functions, and isn't there some usage of the stack to save registers? It seems there should be a way to turn off the signal handler during the time when you are suspicous of the stack being the culprit, then re-engage the signal handler before throwing the error. -Steve The address of the instruction being executed is hijacked, so, instead of resuming normal operation after the signal handler exit, it get into the throwing handler. This is a very nice trick !
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 21:28, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 20:20:05 UTC, deadalnix wrote: The topic is *Turning a SIGSEGV into a regular function call under Linux, allowing throw*, not only Exception. I don't understand what is the problem here ? Can't we talk about how we could keep trash register clean in case we don't throw - this doesn't make much sense if we throw anyway - ? What your are mentioning here is already done. Nothing to discuss about that. This is why I try to jump into the next topic : how can we do more than just throwing. OK. But (to me, at least) you sounded like you were criticizing the implementation for solving that specific task, so it would help if you were clearer of your intentions. For example, losing the contents EAX is relativery harmless, but the contents of EBP, EGS etc. can be very important. I'm not criticizing at all ! I think this is awesome ! I'm just trying to discuss way we can get to the next step. Loosing EAX is harmless in the throwing case, but it is a problem for other tasks. I didn't mentioned this into the topic, but I'm very enthusiastic about that !
Re: DI Generation Needs your Help!
Adam Wilson: I think I said this before, but for what it's worth, I think your new approach is spot-on correct.
Re: Tuple unpacking syntax [Was: Re: Multiple return values...]
On 14 March 2012 15:17, Robert Jacques wrote: > But there's a reason we use /// instead of ⫻; we shouldn't require custom > keyboard mappings in order to program efficiently in D. > Hold that thought, I think you're missing a major franchising opportunity right there... D branded 'pro-codah' keyboards! Nice! :P
Re: Multiple return values...
On 14 March 2012 22:10, Ary Manzana wrote:
> On 3/14/12 5:00 PM, Simen Kjærås wrote:
>
>> On Wed, 14 Mar 2012 20:02:50 +0100, Ary Manzana
>> wrote:
>>
>> Here's what you can do in Ruby:
>>>
>>> a = 1
>>> b = 2
>>>
>>> # Swap the contents
>>> a, b = b, a
>>>
>>> Can you do something like that with templates in D, with a nice syntax?
>>>
>>
>> template to(T...) {
>> alias T to;
>> }
>>
>> auto from(T...)(T t) {
>> struct Result { T t; alias t this; }
>> return Result( t );
>> }
>>
>> void main( ) {
>> int a = 3;
>> int b = 4;
>>
>> to!(a, b) = from(b, a);
>>
>> assert( a == 4 );
>> assert( b == 3 );
>> }
>>
>
> Awesome! :-)
Mmmm, I still kinda like the ruby way.
I agree, the coma operator is a serious liability in D. Multi assignments
without any other rubbish around it are useful in a whole bunch of of
contexts.
How much would really break if coma was deprecated? Is it used any more
than C++ coma? (Most C++ programs wouldn't even know if the coma operator
were removed)
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 13:45:13 -0700, Alvaro wrote: El 19/12/2011 9:11, Adam Wilson escribió: 1. Function Implementations are removed What about function inlining? would it work from a different module? I think the implementation of small functions should be kept so that client code can inline them. The current DMD does this apparently, it keeps small functions in .di files. The problem is that in DI generation, at least as near as I can tell, D has now idea how big a function is during DI generation. In my experience it was keeping all functions not just small ones. And frankly DI generation is targeted at library builders who are well aware of the inlining trade-offs. And then comes the question of how do you define "small functions" to an adequately technical level. Because theoretically you can inline ANYTHING. Yes, there are rules the prevent inlining, but you'll also note that the state of these rules is not guaranteed to be known at generation time. DI generation currently works as such. After the code has been parsed into an AST, the compiler runs a special set of virtual functions called toCBuffer (and it's variants) that is used to print out the AST in source form again. NO semantic analysis of any kind has been performed on the AST yet. And you wouldn't want to generate DI's after semantic analysis as the analysis fundamentally alters the AST such that you would not get the same code back out and some things would be missing from the DI files that you intended to be there. The AST at DI generation is an incredibly naive representation of the SOURCE not the SEMANTICS; which is what you would need to determine the eligibility of inlining. The answer that Walter has always given for objections about how DI files are built is that if you want anything customized about the output, you have to do it yourself. DI generation will probably NEVER be as perfect as everyone wants. But I think this solution represents a best effort to get DI files to a point where the community agrees that they would be most useful in achieving their intended purpose, which is as interface files for compiled libraries. It's not perfect, but it gets you A LOT further than the current one, if you need customization beyond that, well, D lets you do that too. :-) -- Adam Wilson Project Coordinator The Horizon Project http://www.thehorizonproject.org/
Re: Container templates and constancy of elements
On 14/03/2012 19:24, H. S. Teoh wrote: Exactly. AA keys must be immutable, no matter what. Of course, to interact nicely with existing code, methods like .opIndex or .get should also accept mutable keys for lookup purposes, and .idup them when a new entry needs to be created. This would rely on class authors making sure they define .idup. We would also need a deep version of .idup for array-of-array and array-of-class types. (d) guarantees that changes to the data that mess up the data structure will never happen, at least if the programmer doesn't bypass the const system. [...] I believe this is the best way to go. Well, at least for AA's this is needed. Otherwise there will always be the possibility that AA's will malfunction when the key changes behind the container's back. Thinking about it, if we're going to go this far, maybe we could just make the key type fully immutable whatever it is. This would enable the key variable in a foreach to be ref for efficiency (especially useful if it's a struct type) and still prevent changing of the key through it. But this precludes implementing hash slots as arrays, at least if you want to be able to delete elements. I might have to rethink my strategy here as well. A linked list would get around it but increase the memory allocation overhead - not sure if this is worth worrying about. Stewart.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Mar 14, 2012, at 1:54 PM, FeepingCreature wrote: > > I think that case is sufficiently rare that it'd have to count somewhere > between "act of god" and "outright developer malice". The assumption that the > stack frame is valid is, I'd say, safe to make in the vast majority of cases. > You pretty much have to actively try to break it, for no clearly discernible > reason. The prevalence of buffer overflow attacks might suggest otherwise.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On 03/14/12 21:08, Don Clugston wrote:
>
> I didn't realize that was possible. Very interesting.
> As it stands, though, that's got some pretty serious issues.
>
> You are on the stack of the function that was called, but you don't know for
> sure that it is a valid stack.
>
> asm {
> push EBX;
> mov EBX, ESP;
> mov ESP, 0;// Look ma, no stack!
>
> mov int ptr [ESP], 0; // segfault -- null pointer exception
>
> mov ESP, EBX;
> pop EBX;
> }
>
> Now, your user space handler will cause another segfault when it does the mov
> [ESP], 0. I think that gives you an infinite loop.
>
I think that case is sufficiently rare that it'd have to count somewhere
between "act of god" and "outright developer malice". The assumption that the
stack frame is valid is, I'd say, safe to make in the vast majority of cases.
You pretty much have to actively try to break it, for no clearly discernible
reason.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wed, 14 Mar 2012 16:45:49 -0400, Don Clugston wrote: On 14/03/12 21:31, Steven Schveighoffer wrote: On Wed, 14 Mar 2012 16:08:29 -0400, Don Clugston wrote: Now, your user space handler will cause another segfault when it does the mov [ESP], 0. I think that gives you an infinite loop. SEGFAULT inside a SEGV signal handler aborts the program (no way to turn this off IIRC). -Steve But you're not inside the signal handler when it happens. You returned. Then how does the signal handler do anything? I mean, doesn't it need a stack? Or does it just affect register variables? Most signal handlers are normal functions, and isn't there some usage of the stack to save registers? It seems there should be a way to turn off the signal handler during the time when you are suspicous of the stack being the culprit, then re-engage the signal handler before throwing the error. -Steve
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On 14/03/12 21:31, Steven Schveighoffer wrote: On Wed, 14 Mar 2012 16:08:29 -0400, Don Clugston wrote: Now, your user space handler will cause another segfault when it does the mov [ESP], 0. I think that gives you an infinite loop. SEGFAULT inside a SEGV signal handler aborts the program (no way to turn this off IIRC). -Steve But you're not inside the signal handler when it happens. You returned.
Re: DI Generation Needs your Help!
El 19/12/2011 9:11, Adam Wilson escribió: 1. Function Implementations are removed What about function inlining? would it work from a different module? I think the implementation of small functions should be kept so that client code can inline them. The current DMD does this apparently, it keeps small functions in .di files.
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 12:35:03 -0700, Andrej Mitrovic
wrote:
On 3/14/12, Andrej Mitrovic wrote:
This seems to work nicely even with the latest release. It's much
better than the current .di generation for sure. Any plans on merging
this with mainline before it goes stale?
Although I would really like to be able to keep documentation comments
in the .di files, e.g.:
class Foo {
/** Commented ctor. */
this() { }
}
->
class Foo {
/** Commented ctor. */
this();
}
I'll look into it when I get the chance. :-)
--
Adam Wilson
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/
Re: DI Generation Needs your Help!
On Wed, 14 Mar 2012 12:30:00 -0700, Andrej Mitrovic wrote: On 12/19/11, Adam Wilson wrote: If anyone wishes to test my changes against their code, you can download them from my git account here: https://lightben...@github.com/LightBender/dmd.git This seems to work nicely even with the latest release. It's much better than the current .di generation for sure. Any plans on merging this with mainline before it goes stale? I am currently maintaining this against DMD HEAD, but until this bug (http://d.puremagic.com/issues/show_bug.cgi?id=7423) gets fixed I can't open a pull request because the patch will break both the Phobos and DRuntime builds badly without it. I would fix the bug myself, but I am getting married in a month and am rather short on time. Earliest I'll likely be able to investigate a fix is May. If anyone else wants to take a shot at a fix before then, I'd happily open up a pull request once they're done.
Re: Container templates and constancy of elements
On Wed, 14 Mar 2012 14:51:54 -0400, Stewart Gordon wrote: With some containers, such as lists and trees (binary search trees aside), it doesn't matter if the elements can change state, since the data structure remains well-formed. However, with others, such as sets and AAs, it's desirable if the keys don't mutate. Any operations on them won't work right if there are keys in the wrong hash slots or out of order (depending on the underlying data structure) because they have changed since being put into the container. Of course, this doesn't apply to _values_ in an AA, since these can safely mutate. In Java and D1, it seems you just have to go on trust if you want your container to accept key types other than strict value types. But can we do better in D2 - short of forcing the key type to be immutable, which would hinder the container's usefulness? But it seems D2 has taken one step in that general direction by automatically tail-consting the key type of an AA. But it doesn't stop modifications to the key through mutable references to the data. And if the key is of class type, you can still modify the object, since D2 conflates constancy of an object reference with constancy of the object itself. This is in itself silly. Really, D should have tail-const built in for stuff like this. OK, so there's Rebindable, but I've found it to be a PITA when trying to do generic programming with it, as well as leading to this AA key anomaly because it isn't a built-in feature. I suppose there are a few possibilities for what constancy to apply to elements of a data structure to which changes might affect the structure's integrity: (a) force the type to be tail-const if it's an array, otherwise don't add any constancy (b) force the type to be tail-const if it's an array, or fully const if it's a class (c) force the type to be tail-immutable if it's an array, otherwise don't add any constancy (d) force the type to be tail-immutable if it's an array, or fully immutable if it's a class (e) don't add any constancy, but just rely on trust Currently, AAs implement (a). (d) guarantees that changes to the data that mess up the data structure will never happen, at least if the programmer doesn't bypass the const system. (e) is the route D1 and Java are forced to take. Am I right in thinking that sets and maps in the C++ STL take the same path? Anyway, what are people's thoughts on the right way to go here? IMO, @safe code should only allow d, @system/trusted should allow e. And the compiler shouldn't be "helping" you by adding const annotations. That is: int[char[]] aa; this line should either compile, and the type should be int[char[]] aa, or it should not compile. D is supposed to be able to do "at your own risk" bare-metal types of things. This seems like it unnecessarily limits code. -Steve
Re: Container templates and constancy of elements
On 03/14/2012 12:24 PM, H. S. Teoh wrote:
> On Wed, Mar 14, 2012 at 06:51:54PM +, Stewart Gordon wrote:
>> With some containers, such as lists and trees (binary search trees
>> aside), it doesn't matter if the elements can change state, since
>> the data structure remains well-formed.
>>
>> However, with others, such as sets and AAs, it's desirable if the
>> keys don't mutate. Any operations on them won't work right if there
>> are keys in the wrong hash slots or out of order (depending on the
>> underlying data structure) because they have changed since being put
>> into the container. Of course, this doesn't apply to _values_ in an
>> AA, since these can safely mutate.
>
> IMO, AA keys should be *implicitly* immutable. That is, when you declare
> something like:
>
>int[ubyte[]] x;
>
> then the key type of x should be immutable(ubyte)[], not ubyte[].
Yes, the internally stored copy of the key should be immutable(ubyte[]).
Please note the difference from yours, but I guess that extra protection
is just to protect the developers from themselves by avoiding mutating
the key in the implementation.
But the conceptual key type of the AA should be const(ubyte[]). The
reason is, a const parameter is welcoming: It says "I accept both
mutable and immutable as arguments". On the other hand, an immutable
parameter is restricting: It says "I demand only immutable as argument".
The following opIndex accepts keys of five different types of immutability:
class SingleElementAA
{
immutable(uint[]) key;
int value;
ref int opIndex(const(uint[]) key)
{
return value;
}
}
void main()
{
auto aa = new SingleElementAA();
uint[] mutable;
immutable(uint)[] element_immutable;
immutable(uint[]) all_immutable;
enum uint[] enum_immutable = [ 1 ];
aa[mutable] = 42;
aa[element_immutable] = 43;
aa[all_immutable] = 44;
aa[enum_immutable] = 45;
aa[[0, 1]] = 42; // literal key
}
Ali
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wed, 14 Mar 2012 16:08:29 -0400, Don Clugston wrote: Now, your user space handler will cause another segfault when it does the mov [ESP], 0. I think that gives you an infinite loop. SEGFAULT inside a SEGV signal handler aborts the program (no way to turn this off IIRC). -Steve
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wednesday, 14 March 2012 at 20:20:05 UTC, deadalnix wrote: The topic is *Turning a SIGSEGV into a regular function call under Linux, allowing throw*, not only Exception. I don't understand what is the problem here ? Can't we talk about how we could keep trash register clean in case we don't throw - this doesn't make much sense if we throw anyway - ? What your are mentioning here is already done. Nothing to discuss about that. This is why I try to jump into the next topic : how can we do more than just throwing. OK. But (to me, at least) you sounded like you were criticizing the implementation for solving that specific task, so it would help if you were clearer of your intentions. For example, losing the contents EAX is relativery harmless, but the contents of EBP, EGS etc. can be very important.
Re: Implicit string lit conversion to wstring/dstring
On 03/14/2012 09:18 PM, Timon Gehr wrote:
Use this for now:
void opIndexAssign(K)(in int v, scope K key)
if (!is(K==Key) && isCompatWithKey!K)
{...}
oops.
I meant:
void opIndexAssign(K)(in Value v, scope K key)
if (!is(K==Key) && isCompatWithKey!K)
{...}
'in K key'/const(K) key will only IFTI-match a const type. I don't think
that is sensible at all, you may want to file a bug report.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 21:07, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 19:48:28 UTC, deadalnix wrote: Le 14/03/2012 18:28, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 17:18:06 UTC, deadalnix wrote: Le 14/03/2012 18:00, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 16:37:45 UTC, deadalnix wrote: Le 14/03/2012 17:08, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: You are loosing EAX in the process. When would this matter? EAX is a scratch register per ABIs, no? You may want to return from the function the standard way an resume operations. To implement a moving GC using page protection for example. This doesn't have anything to do with turning signals into exceptions. No but this does, make sense to catch segfault and act according to it to implement such a functionality. This is a very close problem. You can't resume D exceptions. I'm not talking about Exception anymore. In case of Exception, this isn't a problem, but in case of regular return, this is. I don't understand how any of your posts are related to this thread at all. This thread is about turning SIGSEGV into an exception that 1) you can catch 2) will print a stack trace when uncaught. You've brought in stack overflows, moving garbage collectors, etc. I assure you, we are well-aware of the problems when using this exact code for other purposes. The topic is *Turning a SIGSEGV into a regular function call under Linux, allowing throw*, not only Exception. I don't understand what is the problem here ? Can't we talk about how we could keep trash register clean in case we don't throw - this doesn't make much sense if we throw anyway - ? What your are mentioning here is already done. Nothing to discuss about that. This is why I try to jump into the next topic : how can we do more than just throwing.
Re: Implicit string lit conversion to wstring/dstring
On 03/14/2012 08:34 PM, H. S. Teoh wrote:
On Wed, Mar 14, 2012 at 02:07:04PM -0500, Andrei Alexandrescu wrote:
On 3/14/12 2:01 PM, H. S. Teoh wrote:
However, this change broke this code:
AssociativeArray!(wstring,int) aa;
aa["abc"] = 123; // error: compiler deduces K as string,
// so isCompatWithKey!K fails: string
// can't implicitly convert to wstring
Whereas before, when opIndexAssign looked like this:
void opIndexAssign(in Value v, in Key key)
{
...
}
everything worked, because the compiler deduces the type of "abc" as
wstring since Key==wstring.
Aha! This is one of those cases in which built-in magic smells of
putrid beef soup.
+1.
I think it's possible to still make this work by beefing up the
template constraints such that the working signature is selected for
strings.
[...]
I tried the following, but it still doesn't work properly:
void opIndexAssign()(in Value v, in Key key)
{
__opIndexAssignImpl(v, key);
}
void opIndexAssign(K)(in Value v, in K key)
if (!is(K==Key)&& isCompatWithKey!K)
{
__opIndexAssignImpl(v, key);
}
This causes this case to fail:
AssociativeArray!(wstring,int) aa;
wchar[] key = "abc"w.dup;
aa[key] = 123; // Error: template
newAA.AA!(immutable(wchar)[],int).AssociativeArray.opIndexAssign() cannot
deduce template function from argument types !()(int,wchar[])
I don't understand what's happening here. The condition "!is(K==Key)"
is necessary because otherwise the compiler complains that more than one
template matches the given call, but for some weird reason both
templates vanish from consideration when the condition is put in.
T
Use this for now:
void opIndexAssign(K)(in int v, scope K key)
if (!is(K==Key) && isCompatWithKey!K)
{...}
'in K key'/const(K) key will only IFTI-match a const type. I don't think
that is sensible at all, you may want to file a bug report.
Re: Multiple return values...
On 3/14/12 5:00 PM, Simen Kjærås wrote:
On Wed, 14 Mar 2012 20:02:50 +0100, Ary Manzana
wrote:
Here's what you can do in Ruby:
a = 1
b = 2
# Swap the contents
a, b = b, a
Can you do something like that with templates in D, with a nice syntax?
template to(T...) {
alias T to;
}
auto from(T...)(T t) {
struct Result { T t; alias t this; }
return Result( t );
}
void main( ) {
int a = 3;
int b = 4;
to!(a, b) = from(b, a);
assert( a == 4 );
assert( b == 3 );
}
Awesome! :-)
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On 13/03/12 11:09, FeepingCreature wrote:
Note: I worked out this method for my own language, Neat, but the basic
approach should be portable to D's exceptions as well.
I've seen it argued a lot over the years (even argued it myself) that it's
impossible to throw from Linux signal handlers. This is basically correct,
because they constitute an interruption in the stack that breaks exceptions'
ability to unroll properly.
However, there is a method to turn a signal handler into a regular function
call that you can throw from.
Basically, what we need to do is similar to a stack buffer overflow exploit.
Under Linux, the extended signal handler that is set with sigaction is called
with three arguments: the signal, a siginfo_t* and a ucontext_t* as the third.
The third parameter is what we're interested in. Deep inside the ucontext_t
struct is uc.mcontext.gregs[REG_EIP], the address of the instruction that
caused the segfault. This is the location that execution returns to when the
signal handler returns. By overwriting this location, we can turn a return into
a function call.
First, gregs[REG_EAX] = gregs[REG_EIP];
We can safely assume that the function that caused the segfault doesn't really
need its EAX anymore, so we can reuse it to reconstruct a proper stackframe to
throw from later.
Second, gregs[REG_EIP] = cast(void*)&sigsegv_userspace_handler;
Note that the naked attribute was not used. If used, it can make this code
slightly easier.
extern(C) void sigsegv_userspace_handler() {
// done implicitly
// asm { push ebp; }
// asm { mov ebp, esp; }
asm { mov ebx, [esp]; } // backup the pushed ebp
asm { mov [esp], eax; } // replace it with the correct return address
// which was originally left out due to the
// irregular way we entered this function (via a
ret).
asm { push ebx; } // recreate the pushed ebp
asm { mov ebp, esp; } // complete stackframe.
// originally, our stackframe (because we entered this function via a ret)
// was [ebp]. Now, it's [return address][ebp], as is proper for cdecl.
// at this point, we can safely throw
// (or invoke any other non-handler-safe function).
throw new SignalException("SIGSEGV");
}
I didn't realize that was possible. Very interesting.
As it stands, though, that's got some pretty serious issues.
You are on the stack of the function that was called, but you don't know
for sure that it is a valid stack.
asm {
push EBX;
mov EBX, ESP;
mov ESP, 0;// Look ma, no stack!
mov int ptr [ESP], 0; // segfault -- null pointer exception
mov ESP, EBX;
pop EBX;
}
Now, your user space handler will cause another segfault when it does
the mov [ESP], 0. I think that gives you an infinite loop.
I think the idea would work, if you had some guarantee that the stack
pointer was valid. Then, call a separate handler if it is not.
The primary 'trick' in Windows SEH is that it goes to great lengths to
verify that the stack is valid. I'm not sure that in Linux user space
you have enough information to verify it. But maybe you do. At least,
you should be able to check that it's in memory which is owned by your
process.
Would be awesome if it is possible.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wednesday, 14 March 2012 at 19:48:28 UTC, deadalnix wrote: Le 14/03/2012 18:28, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 17:18:06 UTC, deadalnix wrote: Le 14/03/2012 18:00, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 16:37:45 UTC, deadalnix wrote: Le 14/03/2012 17:08, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: You are loosing EAX in the process. When would this matter? EAX is a scratch register per ABIs, no? You may want to return from the function the standard way an resume operations. To implement a moving GC using page protection for example. This doesn't have anything to do with turning signals into exceptions. No but this does, make sense to catch segfault and act according to it to implement such a functionality. This is a very close problem. You can't resume D exceptions. I'm not talking about Exception anymore. In case of Exception, this isn't a problem, but in case of regular return, this is. I don't understand how any of your posts are related to this thread at all. This thread is about turning SIGSEGV into an exception that 1) you can catch 2) will print a stack trace when uncaught. You've brought in stack overflows, moving garbage collectors, etc. I assure you, we are well-aware of the problems when using this exact code for other purposes.
Re: Multiple return values...
On Wed, 14 Mar 2012 20:02:50 +0100, Ary Manzana
wrote:
Here's what you can do in Ruby:
a = 1
b = 2
# Swap the contents
a, b = b, a
Can you do something like that with templates in D, with a nice syntax?
template to(T...) {
alias T to;
}
auto from(T...)(T t) {
struct Result { T t; alias t this; }
return Result( t );
}
void main( ) {
int a = 3;
int b = 4;
to!(a, b) = from(b, a);
assert( a == 4 );
assert( b == 3 );
}
Re: Implicit string lit conversion to wstring/dstring
On 3/14/12 2:34 PM, H. S. Teoh wrote:
I tried the following, but it still doesn't work properly:
void opIndexAssign()(in Value v, in Key key)
{
__opIndexAssignImpl(v, key);
}
void opIndexAssign(K)(in Value v, in K key)
if (!is(K==Key)&& isCompatWithKey!K)
{
__opIndexAssignImpl(v, key);
}
Try special casing for the exact types (string, char[] etc), get that
working, and then generalize from there.
Andrei
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 18:01, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 16:39:29 UTC, deadalnix wrote: Le 14/03/2012 17:34, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 07:35:50 UTC, FeepingCreature wrote: Sweet. Yeah, I think you need to use naked and reconstruct the stackframe. Not sure how it'd look; I'm not familiar with the x86_64 ABI. I think it might be safe to just reconstruct the stack frame in the signal handler, and set gregs[REG_EIP] to &_d_throw directly. It should also use a pre-allocated exception object (like how it's done with OutofMemoryError and InvalidMemoryOperationError), in case there's data corruption in the GC. Especially if the signal is sent because of stack overflow ! Not sure if sarcasm..? In case of a stack overflow, you can't call _d_throwc (or use the "throw" statement) anyway. You can page protect the last segment of the stack, and unprotect it before throwing.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 18:28, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 17:18:06 UTC, deadalnix wrote: Le 14/03/2012 18:00, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 16:37:45 UTC, deadalnix wrote: Le 14/03/2012 17:08, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: You are loosing EAX in the process. When would this matter? EAX is a scratch register per ABIs, no? You may want to return from the function the standard way an resume operations. To implement a moving GC using page protection for example. This doesn't have anything to do with turning signals into exceptions. No but this does, make sense to catch segfault and act according to it to implement such a functionality. This is a very close problem. You can't resume D exceptions. I'm not talking about Exception anymore. In case of Exception, this isn't a problem, but in case of regular return, this is.
Re: Implicit string lit conversion to wstring/dstring
On Wed, Mar 14, 2012 at 02:07:04PM -0500, Andrei Alexandrescu wrote:
> On 3/14/12 2:01 PM, H. S. Teoh wrote:
> >However, this change broke this code:
> >
> > AssociativeArray!(wstring,int) aa;
> > aa["abc"] = 123;// error: compiler deduces K as string,
> > // so isCompatWithKey!K fails: string
> > // can't implicitly convert to wstring
> >
> >Whereas before, when opIndexAssign looked like this:
> >
> > void opIndexAssign(in Value v, in Key key)
> > {
> > ...
> > }
> >
> >everything worked, because the compiler deduces the type of "abc" as
> >wstring since Key==wstring.
>
> Aha! This is one of those cases in which built-in magic smells of
> putrid beef soup.
>
> I think it's possible to still make this work by beefing up the
> template constraints such that the working signature is selected for
> strings.
[...]
Also, IMHO, this needs to work for array literals in general, not just
strings. For example, this should work:
int[ubyte[]] aa;
aa[[1,2,3]] = 123;
The [1,2,3] should be deduced as ubyte[] instead of int[].
T
--
Ignorance is bliss... but only until you suffer the consequences!
Re: DI Generation Needs your Help!
On 3/14/12, Andrej Mitrovic wrote:
> This seems to work nicely even with the latest release. It's much
> better than the current .di generation for sure. Any plans on merging
> this with mainline before it goes stale?
Although I would really like to be able to keep documentation comments
in the .di files, e.g.:
class Foo {
/** Commented ctor. */
this() { }
}
->
class Foo {
/** Commented ctor. */
this();
}
Re: Implicit string lit conversion to wstring/dstring
On Wed, Mar 14, 2012 at 02:07:04PM -0500, Andrei Alexandrescu wrote:
> On 3/14/12 2:01 PM, H. S. Teoh wrote:
> >However, this change broke this code:
> >
> > AssociativeArray!(wstring,int) aa;
> > aa["abc"] = 123;// error: compiler deduces K as string,
> > // so isCompatWithKey!K fails: string
> > // can't implicitly convert to wstring
> >
> >Whereas before, when opIndexAssign looked like this:
> >
> > void opIndexAssign(in Value v, in Key key)
> > {
> > ...
> > }
> >
> >everything worked, because the compiler deduces the type of "abc" as
> >wstring since Key==wstring.
>
> Aha! This is one of those cases in which built-in magic smells of
> putrid beef soup.
+1.
> I think it's possible to still make this work by beefing up the
> template constraints such that the working signature is selected for
> strings.
[...]
I tried the following, but it still doesn't work properly:
void opIndexAssign()(in Value v, in Key key)
{
__opIndexAssignImpl(v, key);
}
void opIndexAssign(K)(in Value v, in K key)
if (!is(K==Key) && isCompatWithKey!K)
{
__opIndexAssignImpl(v, key);
}
This causes this case to fail:
AssociativeArray!(wstring,int) aa;
wchar[] key = "abc"w.dup;
aa[key] = 123; // Error: template
newAA.AA!(immutable(wchar)[],int).AssociativeArray.opIndexAssign() cannot
deduce template function from argument types !()(int,wchar[])
I don't understand what's happening here. The condition "!is(K==Key)"
is necessary because otherwise the compiler complains that more than one
template matches the given call, but for some weird reason both
templates vanish from consideration when the condition is put in.
T
--
It is the quality rather than the quantity that matters. -- Lucius Annaeus
Seneca
Re: DI Generation Needs your Help!
On 12/19/11, Adam Wilson wrote: > If anyone wishes to test my changes against their code, you can download > them from my git account here: > https://lightben...@github.com/LightBender/dmd.git This seems to work nicely even with the latest release. It's much better than the current .di generation for sure. Any plans on merging this with mainline before it goes stale?
Re: Container templates and constancy of elements
On Wed, Mar 14, 2012 at 06:51:54PM +, Stewart Gordon wrote: > With some containers, such as lists and trees (binary search trees > aside), it doesn't matter if the elements can change state, since > the data structure remains well-formed. > > However, with others, such as sets and AAs, it's desirable if the > keys don't mutate. Any operations on them won't work right if there > are keys in the wrong hash slots or out of order (depending on the > underlying data structure) because they have changed since being put > into the container. Of course, this doesn't apply to _values_ in an > AA, since these can safely mutate. IMO, AA keys should be *implicitly* immutable. That is, when you declare something like: int[ubyte[]] x; then the key type of x should be immutable(ubyte)[], not ubyte[]. Otherwise, it just doesn't make any sense, and causes several of the AA-related bugs currently on the bugtracker. > In Java and D1, it seems you just have to go on trust if you want > your container to accept key types other than strict value types. > But can we do better in D2 - short of forcing the key type to be > immutable, which would hinder the container's usefulness? > > But it seems D2 has taken one step in that general direction by > automatically tail-consting the key type of an AA. But it doesn't > stop modifications to the key through mutable references to the > data. Exactly. AA keys must be immutable, no matter what. Of course, to interact nicely with existing code, methods like .opIndex or .get should also accept mutable keys for lookup purposes, and .idup them when a new entry needs to be created. > And if the key is of class type, you can still modify the object, > since D2 conflates constancy of an object reference with constancy of > the object itself. This is in itself silly. Really, D should have > tail-const built in for stuff like this. This is a major PITA. Especially if you're trying to be const-correct in your code, then it leads to nonsense like being unable to traverse a linked list inside a const method, because the iteration pointer itself is const (whereas it's the *data* it's pointing to that's const) so you can't overwrite the loop variable. However, you *can* recursively search the list. I find this to be a major WAT in D. [...] > I suppose there are a few possibilities for what constancy to apply > to elements of a data structure to which changes might affect the > structure's integrity: > > (a) force the type to be tail-const if it's an array, otherwise don't add any > constancy > (b) force the type to be tail-const if it's an array, or fully const if it's > a class > (c) force the type to be tail-immutable if it's an array, otherwise don't add > any constancy > (d) force the type to be tail-immutable if it's an array, or fully immutable > if it's a class > (e) don't add any constancy, but just rely on trust > > > Currently, AAs implement (a). Which is prone to bugs. > (d) guarantees that changes to the data that mess up the data > structure will never happen, at least if the programmer doesn't bypass > the const system. [...] I believe this is the best way to go. Well, at least for AA's this is needed. Otherwise there will always be the possibility that AA's will malfunction when the key changes behind the container's back. T -- Just because you survived after you did it, doesn't mean it wasn't stupid!
Re: Implicit string lit conversion to wstring/dstring
On 3/14/12 2:01 PM, H. S. Teoh wrote:
However, this change broke this code:
AssociativeArray!(wstring,int) aa;
aa["abc"] = 123; // error: compiler deduces K as string,
// so isCompatWithKey!K fails: string
// can't implicitly convert to wstring
Whereas before, when opIndexAssign looked like this:
void opIndexAssign(in Value v, in Key key)
{
...
}
everything worked, because the compiler deduces the type of "abc" as
wstring since Key==wstring.
Aha! This is one of those cases in which built-in magic smells of putrid
beef soup.
I think it's possible to still make this work by beefing up the template
constraints such that the working signature is selected for strings.
Andrei
Re: Multiple return values...
On 3/14/12 2:02 PM, Ary Manzana wrote: On 3/13/12 6:12 PM, Andrei Alexandrescu wrote: On 3/13/12 2:57 PM, Manu wrote: And you think that's more readable and intuitive than: (v1, v2, v3) = fun(); ? Yes (e.g. when I see the commas my mind starts running in all directions because that's valid code nowadays that ignores v1 and v2 and keeps v3 as an lvalue). Who uses that, except code generators? I'd like D to deprecate the comma so it can be used for other things, like tuple assignment. I'd like that, too, but we need to worry about breaking code, too. Let me put it another way: I don't see one syntax over another a deal maker or deal breaker. At all. Well, it's sad that syntax is not very important in D. That's an undue generalization of what I said. If you have to write less code there will be less chance for bugs and it will be more understandable (unless you obfuscate the code, obviously). I agree. Here's what you can do in Ruby: a = 1 b = 2 # Swap the contents a, b = b, a Can you do something like that with templates in D, with a nice syntax? swap(a, b) Andrei
Re: Multiple return values...
On 3/13/12 6:12 PM, Andrei Alexandrescu wrote: On 3/13/12 2:57 PM, Manu wrote: And you think that's more readable and intuitive than: (v1, v2, v3) = fun(); ? Yes (e.g. when I see the commas my mind starts running in all directions because that's valid code nowadays that ignores v1 and v2 and keeps v3 as an lvalue). Who uses that, except code generators? I'd like D to deprecate the comma so it can be used for other things, like tuple assignment. Let me put it another way: I don't see one syntax over another a deal maker or deal breaker. At all. Well, it's sad that syntax is not very important in D. If you have to write less code there will be less chance for bugs and it will be more understandable (unless you obfuscate the code, obviously). Here's what you can do in Ruby: a = 1 b = 2 # Swap the contents a, b = b, a Can you do something like that with templates in D, with a nice syntax?
Re: Implicit string lit conversion to wstring/dstring
On Wed, Mar 14, 2012 at 02:44:40PM -0400, Steven Schveighoffer wrote:
> On Wed, 14 Mar 2012 14:16:11 -0400, H. S. Teoh
> wrote:
>
> >What I want is to force the compiler to deduce S=dstring when I
> >declare func(S)(S) and call it as func("abc").
>
> http://d.puremagic.com/issues/show_bug.cgi?id=4998
>
> Please vote or contribute your thoughts.
[...]
Ahhh, thanks for pointing this out. That is exactly the problem I'm
struggling with.
I guess the motivation of my original question wasn't clear. I should
say that I ran into this problem in the context of my AA implementation.
I've successfully implemented Andrei's suggestion: int[wstring] will now
accept wchar[], const(wchar)[], in addition to wstring in .get,
.opIndex, etc.. It will implicitly call wchar[].idup when it needs to
create a new hash entry. (Ditto with dstring, and with any immutable
array key type.)
To implement this change, opIndexAssign now looks like this:
struct AssociativeArray(Key,Value) {
...
void opIndexAssign(K)(in Value v, in K key)
if (isCompatWithKey!K)
{
...
}
}
The template isCompatWithKey basically checks if K can be implicitly
converted to Key, or has an .idup method that returns something that can
be implicitly converted to Key.
However, this change broke this code:
AssociativeArray!(wstring,int) aa;
aa["abc"] = 123;// error: compiler deduces K as string,
// so isCompatWithKey!K fails: string
// can't implicitly convert to wstring
Whereas before, when opIndexAssign looked like this:
void opIndexAssign(in Value v, in Key key)
{
...
}
everything worked, because the compiler deduces the type of "abc" as
wstring since Key==wstring.
Now you have to write:
aa["abc"w] = 123;
which isn't the end of the world, I suppose, but it's not very nice, and
also breaks existing code that depended on the compiler automatically
deducing that typeof("abc")==wstring.
T
--
There are three kinds of people in the world: those who can count, and those
who can't.
Container templates and constancy of elements
With some containers, such as lists and trees (binary search trees aside), it doesn't matter if the elements can change state, since the data structure remains well-formed. However, with others, such as sets and AAs, it's desirable if the keys don't mutate. Any operations on them won't work right if there are keys in the wrong hash slots or out of order (depending on the underlying data structure) because they have changed since being put into the container. Of course, this doesn't apply to _values_ in an AA, since these can safely mutate. In Java and D1, it seems you just have to go on trust if you want your container to accept key types other than strict value types. But can we do better in D2 - short of forcing the key type to be immutable, which would hinder the container's usefulness? But it seems D2 has taken one step in that general direction by automatically tail-consting the key type of an AA. But it doesn't stop modifications to the key through mutable references to the data. And if the key is of class type, you can still modify the object, since D2 conflates constancy of an object reference with constancy of the object itself. This is in itself silly. Really, D should have tail-const built in for stuff like this. OK, so there's Rebindable, but I've found it to be a PITA when trying to do generic programming with it, as well as leading to this AA key anomaly because it isn't a built-in feature. I suppose there are a few possibilities for what constancy to apply to elements of a data structure to which changes might affect the structure's integrity: (a) force the type to be tail-const if it's an array, otherwise don't add any constancy (b) force the type to be tail-const if it's an array, or fully const if it's a class (c) force the type to be tail-immutable if it's an array, otherwise don't add any constancy (d) force the type to be tail-immutable if it's an array, or fully immutable if it's a class (e) don't add any constancy, but just rely on trust Currently, AAs implement (a). (d) guarantees that changes to the data that mess up the data structure will never happen, at least if the programmer doesn't bypass the const system. (e) is the route D1 and Java are forced to take. Am I right in thinking that sets and maps in the C++ STL take the same path? Anyway, what are people's thoughts on the right way to go here? Stewart.
Re: Implicit string lit conversion to wstring/dstring
On Wed, 14 Mar 2012 14:16:11 -0400, H. S. Teoh
wrote:
What I want is to force the compiler to deduce S=dstring when I declare
func(S)(S) and call it as func("abc").
http://d.puremagic.com/issues/show_bug.cgi?id=4998
Please vote or contribute your thoughts.
-Steve
Re: Implicit string lit conversion to wstring/dstring
"Dmitry Olshansky" wrote in message
news:jjqnnj$1j86$1...@digitalmars.com...
>
> How about this (untested)?
>
> void func()(dstring s) {
> dstring t = s;
> //... funcImpl(t);
> }
> void func(S)(S s) {
> dstring t = to!dstring(s);
> //... funcImpl(t);
> }
>
That fails to compile when passed a dstring because it matches both
functions. Adding an "if(!is(S==dstring))" constraint to the second makes it
compile, but then func("abc") just calls the second one instead of the
first.
Re: Implicit string lit conversion to wstring/dstring
"Alex Rønne Petersen" wrote in message news:jjqn9f$1iht$1...@digitalmars.com... > > But then... why not just make it take a dstring? Maybe I'm not following > your intent here... > Probably because then this becomes an error: string s = "abc"; func(s); It's an interesting problem. I'm not aware of a solution. I suppose you'd just have to make it take only dstring and then expect string/wstring vars to be handled by the caller like "func(to!dstring(s))"
Re: Implicit string lit conversion to wstring/dstring
On 14.03.2012 22:00, H. S. Teoh wrote:
Code:
import std.stdio;
version(explicit) {
void func(dstring s) {
dstring t = s;
writeln(t);
}
} else {
void func(S)(S s) {
dstring t = s; // line 10
writeln(t);
}
}
void main() {
func("abc");
}
If version=explicit is set, the program compiles fine. But if not, then
the compiler complains:
test.d:10: Error: cannot implicitly convert expression (s) of type
string to immutable(dchar)[]
What do I need to do to make the template version of func trigger
implicit conversion of the string lit to dstring? What I'm trying to do
is to templatize dstring as well, but still benefit from the
string->dstring conversion when instantiated with dstring.
(Ditto with implicit conversion to wstring.)
T
How about this (untested)?
void func()(dstring s) {
dstring t = s;
//... funcImpl(t);
}
void func(S)(S s) {
dstring t = to!dstring(s);
//... funcImpl(t);
}
--
Dmitry Olshansky
Re: Wanted: 128 bit integers
On Wednesday, March 14, 2012 17:51:49 Alex Rønne Petersen wrote: > On 14-03-2012 17:49, Jonathan M Davis wrote: > > On Wednesday, March 14, 2012 10:57:03 Manu wrote: > >> On 14 March 2012 02:37, Alex Rønne Petersen wrote: > >>> On 14-03-2012 01:34, Paul D. Anderson wrote: > >>> The D language specifies 128-bit integers: cent and ucent. They just > >>> aren't implemented yet... > >> > >> Why aren't they implemented in a library for the time being at least, so > >> code can compile and work? > > > > I believe that what it really comes down to is that cent and ucent were > > set > > aside just in case they would be needed with no specific plans to do > > anything with them. So, they'll probably be implemented eventually, but > > they're definitely not a priority. And if you really need larger > > integers, then there's BigInt. Prior to the 64-bit ports of dmd (which > > are fairly recent), dmd wasn't even on any architectures that supported > > 128 bit integers anyway. > > > > - Jonathan M Davis > > There aren't really any platforms that natively support 128-bit > integers, even today. The most "support" you'll get is SIMD-like > extensions, but those aren't necessarily useful for implementing 128-bit > integers. > > So, most likely, the compiler would just have to unroll 128-bit > operations just as it does for 64-bit operations on 32-bit targets. long long is 128 bits on 64-bit Linux. That's what I meant by support. 32-bit doesn't have that with any C type on any platform that I know of. I have no idea how it's implemented though. - Jonathan M Davis
Re: Implicit string lit conversion to wstring/dstring
On 14-03-2012 19:16, H. S. Teoh wrote:
On Wed, Mar 14, 2012 at 07:00:35PM +0100, Alex Rønne Petersen wrote:
On 14-03-2012 19:00, H. S. Teoh wrote:
Code:
import std.stdio;
version(explicit) {
void func(dstring s) {
dstring t = s;
writeln(t);
}
} else {
void func(S)(S s) {
dstring t = s; // line 10
writeln(t);
}
}
void main() {
func("abc");
}
If version=explicit is set, the program compiles fine. But if not, then
the compiler complains:
test.d:10: Error: cannot implicitly convert expression (s) of type
string to immutable(dchar)[]
What do I need to do to make the template version of func trigger
implicit conversion of the string lit to dstring? What I'm trying to do
is to templatize dstring as well, but still benefit from the
string->dstring conversion when instantiated with dstring.
(Ditto with implicit conversion to wstring.)
T
I doubt that such an implicit conversion even exists. You have to
prefix the string appropriately, e.g.:
OK, maybe implicit conversion is the wrong word. The compiler is
obviously interpreting func("abc") as func("abc"d) when we declare
func(dstring). But when we declare func(S)(S), the compiler deduces
"abc" as string and sets S=string.
What I want is to force the compiler to deduce S=dstring when I declare
func(S)(S) and call it as func("abc").
T
But then... why not just make it take a dstring? Maybe I'm not following
your intent here...
Anyway, what Phobos functions tend to do is to set S = string by default
and otherwise force people to either pass the string type *or* use the
string suffices.
--
- Alex
Re: Implicit string lit conversion to wstring/dstring
On Wed, Mar 14, 2012 at 07:00:35PM +0100, Alex Rønne Petersen wrote:
> On 14-03-2012 19:00, H. S. Teoh wrote:
> >Code:
> > import std.stdio;
> > version(explicit) {
> > void func(dstring s) {
> > dstring t = s;
> > writeln(t);
> > }
> > } else {
> > void func(S)(S s) {
> > dstring t = s; // line 10
> > writeln(t);
> > }
> > }
> > void main() {
> > func("abc");
> > }
> >
> >If version=explicit is set, the program compiles fine. But if not, then
> >the compiler complains:
> >
> > test.d:10: Error: cannot implicitly convert expression (s) of type
> > string to immutable(dchar)[]
> >
> >What do I need to do to make the template version of func trigger
> >implicit conversion of the string lit to dstring? What I'm trying to do
> >is to templatize dstring as well, but still benefit from the
> >string->dstring conversion when instantiated with dstring.
> >
> >(Ditto with implicit conversion to wstring.)
> >
> >
> >T
> >
>
> I doubt that such an implicit conversion even exists. You have to
> prefix the string appropriately, e.g.:
OK, maybe implicit conversion is the wrong word. The compiler is
obviously interpreting func("abc") as func("abc"d) when we declare
func(dstring). But when we declare func(S)(S), the compiler deduces
"abc" as string and sets S=string.
What I want is to force the compiler to deduce S=dstring when I declare
func(S)(S) and call it as func("abc").
T
--
Дерево держится корнями, а человек - друзьями.
Re: Implicit string lit conversion to wstring/dstring
On 14-03-2012 19:00, Alex Rønne Petersen wrote:
On 14-03-2012 19:00, H. S. Teoh wrote:
Code:
import std.stdio;
version(explicit) {
void func(dstring s) {
dstring t = s;
writeln(t);
}
} else {
void func(S)(S s) {
dstring t = s; // line 10
writeln(t);
}
}
void main() {
func("abc");
}
If version=explicit is set, the program compiles fine. But if not, then
the compiler complains:
test.d:10: Error: cannot implicitly convert expression (s) of type
string to immutable(dchar)[]
What do I need to do to make the template version of func trigger
implicit conversion of the string lit to dstring? What I'm trying to do
is to templatize dstring as well, but still benefit from the
string->dstring conversion when instantiated with dstring.
(Ditto with implicit conversion to wstring.)
T
I doubt that such an implicit conversion even exists. You have to prefix
the string appropriately, e.g.:
string s = "foo";
wstring w = w"bar";
dstring d = d"baz";
Sorry, I lied. Like this:
string s = "foo"c; (the c is optional)
wstring w = "bar"w;
dstring d = "baz"d;
See: http://dlang.org/lex.html
--
- Alex
Re: Implicit string lit conversion to wstring/dstring
On 14-03-2012 19:00, H. S. Teoh wrote:
Code:
import std.stdio;
version(explicit) {
void func(dstring s) {
dstring t = s;
writeln(t);
}
} else {
void func(S)(S s) {
dstring t = s; // line 10
writeln(t);
}
}
void main() {
func("abc");
}
If version=explicit is set, the program compiles fine. But if not, then
the compiler complains:
test.d:10: Error: cannot implicitly convert expression (s) of type
string to immutable(dchar)[]
What do I need to do to make the template version of func trigger
implicit conversion of the string lit to dstring? What I'm trying to do
is to templatize dstring as well, but still benefit from the
string->dstring conversion when instantiated with dstring.
(Ditto with implicit conversion to wstring.)
T
I doubt that such an implicit conversion even exists. You have to prefix
the string appropriately, e.g.:
string s = "foo";
wstring w = w"bar";
dstring d = d"baz";
--
- Alex
Implicit string lit conversion to wstring/dstring
Code:
import std.stdio;
version(explicit) {
void func(dstring s) {
dstring t = s;
writeln(t);
}
} else {
void func(S)(S s) {
dstring t = s; // line 10
writeln(t);
}
}
void main() {
func("abc");
}
If version=explicit is set, the program compiles fine. But if not, then
the compiler complains:
test.d:10: Error: cannot implicitly convert expression (s) of type
string to immutable(dchar)[]
What do I need to do to make the template version of func trigger
implicit conversion of the string lit to dstring? What I'm trying to do
is to templatize dstring as well, but still benefit from the
string->dstring conversion when instantiated with dstring.
(Ditto with implicit conversion to wstring.)
T
--
He who laughs last thinks slowest.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wednesday, 14 March 2012 at 17:18:06 UTC, deadalnix wrote: Le 14/03/2012 18:00, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 16:37:45 UTC, deadalnix wrote: Le 14/03/2012 17:08, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: You are loosing EAX in the process. When would this matter? EAX is a scratch register per ABIs, no? You may want to return from the function the standard way an resume operations. To implement a moving GC using page protection for example. This doesn't have anything to do with turning signals into exceptions. No but this does, make sense to catch segfault and act according to it to implement such a functionality. This is a very close problem. You can't resume D exceptions.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 18:00, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 16:37:45 UTC, deadalnix wrote: Le 14/03/2012 17:08, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: You are loosing EAX in the process. When would this matter? EAX is a scratch register per ABIs, no? You may want to return from the function the standard way an resume operations. To implement a moving GC using page protection for example. This doesn't have anything to do with turning signals into exceptions. No but this does, make sense to catch segfault and act according to it to implement such a functionality. This is a very close problem.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wed, Mar 14, 2012 at 05:39:38PM +0100, deadalnix wrote: > Le 14/03/2012 17:08, Vladimir Panteleev a écrit : > >On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: > >>You are loosing EAX in the process. > > > >When would this matter? EAX is a scratch register per ABIs, no? > > You may want to return from the function the standard way an resume > operations. To implement a moving GC using page protection for > example. I believe the original purpose of this was to catch SIGSEGV and turn it into a thrown Error. So we don't care whether EAX is overwritten since we're never going to return to the code that caused the SEGV; we're just reconstructing the stack frame so that stack unwinding will work correctly when we throw the Error. T -- People tell me that I'm skeptical, but I don't believe it.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wednesday, 14 March 2012 at 16:37:45 UTC, deadalnix wrote: Le 14/03/2012 17:08, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: You are loosing EAX in the process. When would this matter? EAX is a scratch register per ABIs, no? You may want to return from the function the standard way an resume operations. To implement a moving GC using page protection for example. This doesn't have anything to do with turning signals into exceptions.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wednesday, 14 March 2012 at 16:39:29 UTC, deadalnix wrote: Le 14/03/2012 17:34, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 07:35:50 UTC, FeepingCreature wrote: Sweet. Yeah, I think you need to use naked and reconstruct the stackframe. Not sure how it'd look; I'm not familiar with the x86_64 ABI. I think it might be safe to just reconstruct the stack frame in the signal handler, and set gregs[REG_EIP] to &_d_throw directly. It should also use a pre-allocated exception object (like how it's done with OutofMemoryError and InvalidMemoryOperationError), in case there's data corruption in the GC. Especially if the signal is sent because of stack overflow ! Not sure if sarcasm..? In case of a stack overflow, you can't call _d_throwc (or use the "throw" statement) anyway.
Re: Wanted: 128 bit integers
Jonathan M Davis: Prior to the 64-bit ports of dmd (which are fairly recent), dmd wasn't even on any architectures that supported 128 bit integers anyway. You are able to support 128 bit numbers even on a 16 bit system :-) Bye, bearophile
Re: Wanted: 128 bit integers
On 14-03-2012 17:49, Jonathan M Davis wrote: On Wednesday, March 14, 2012 10:57:03 Manu wrote: On 14 March 2012 02:37, Alex Rønne Petersen wrote: On 14-03-2012 01:34, Paul D. Anderson wrote: The D language specifies 128-bit integers: cent and ucent. They just aren't implemented yet... Why aren't they implemented in a library for the time being at least, so code can compile and work? I believe that what it really comes down to is that cent and ucent were set aside just in case they would be needed with no specific plans to do anything with them. So, they'll probably be implemented eventually, but they're definitely not a priority. And if you really need larger integers, then there's BigInt. Prior to the 64-bit ports of dmd (which are fairly recent), dmd wasn't even on any architectures that supported 128 bit integers anyway. - Jonathan M Davis There aren't really any platforms that natively support 128-bit integers, even today. The most "support" you'll get is SIMD-like extensions, but those aren't necessarily useful for implementing 128-bit integers. So, most likely, the compiler would just have to unroll 128-bit operations just as it does for 64-bit operations on 32-bit targets. -- - Alex
Re: Wanted: 128 bit integers
On Wednesday, March 14, 2012 10:57:03 Manu wrote: > On 14 March 2012 02:37, Alex Rønne Petersen wrote: > > On 14-03-2012 01:34, Paul D. Anderson wrote: > > The D language specifies 128-bit integers: cent and ucent. They just > > aren't implemented yet... > > Why aren't they implemented in a library for the time being at least, so > code can compile and work? I believe that what it really comes down to is that cent and ucent were set aside just in case they would be needed with no specific plans to do anything with them. So, they'll probably be implemented eventually, but they're definitely not a priority. And if you really need larger integers, then there's BigInt. Prior to the 64-bit ports of dmd (which are fairly recent), dmd wasn't even on any architectures that supported 128 bit integers anyway. - Jonathan M Davis
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 14:43, FeepingCreature a écrit : On 03/14/12 12:13, deadalnix wrote: Le 13/03/2012 23:24, Vladimir Panteleev a écrit : I think something like this needs to end up in Druntime, at least for Linux x86 and x64. You are loosing EAX in the process. It's somewhat unavoidable. One way or another, you need to find _some_ threadlocal spot to stick those extra size_t.sizeof bytes, since you mustn't lose data, but the hack works by _overwriting_ the return address. Thread local storage is a very easy thing in D. Can't we just use a static variable and set from within the signal handler ?
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 17:34, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 07:35:50 UTC, FeepingCreature wrote: Sweet. Yeah, I think you need to use naked and reconstruct the stackframe. Not sure how it'd look; I'm not familiar with the x86_64 ABI. I think it might be safe to just reconstruct the stack frame in the signal handler, and set gregs[REG_EIP] to &_d_throw directly. It should also use a pre-allocated exception object (like how it's done with OutofMemoryError and InvalidMemoryOperationError), in case there's data corruption in the GC. Especially if the signal is sent because of stack overflow !
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
Le 14/03/2012 17:08, Vladimir Panteleev a écrit : On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: You are loosing EAX in the process. When would this matter? EAX is a scratch register per ABIs, no? You may want to return from the function the standard way an resume operations. To implement a moving GC using page protection for example.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wednesday, 14 March 2012 at 07:35:50 UTC, FeepingCreature wrote: Sweet. Yeah, I think you need to use naked and reconstruct the stackframe. Not sure how it'd look; I'm not familiar with the x86_64 ABI. I think it might be safe to just reconstruct the stack frame in the signal handler, and set gregs[REG_EIP] to &_d_throw directly. It should also use a pre-allocated exception object (like how it's done with OutofMemoryError and InvalidMemoryOperationError), in case there's data corruption in the GC.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On Wednesday, 14 March 2012 at 11:11:54 UTC, deadalnix wrote: You are loosing EAX in the process. When would this matter? EAX is a scratch register per ABIs, no?
Re: Replacing AA's in druntime
On Tue, 13 Mar 2012 22:39:25 -0400, Jakob Bornecrantz wrote: On Wednesday, 14 March 2012 at 00:52:32 UTC, H. S. Teoh wrote: Hi all, My AA implementation is slowly inching closer to being ready to replace aaA.d. So far I've been writing the implementation outside of object_.d for ease of testing & development; now I'm ready to start moving stuff into object_.d to start working on integration with druntime. Hi, If I'm understanding this correctly you are moving the entire implementation of the AA into object.d and as such letting programs be purview to its inner working? In sort meaning you are making the entire AA implementation D ABI locked. This will make it impossible to either change the AA implementation in any ABI breaking fashion or make it impossible to pass AA's between libraries compiled against different versions of druntime. Is this what we really want? This is unavoidable, whether it's a template or not. What changes do you envision would be transparent using an opaque pImpl model (as was done in previous versions of phobos), but would break using templates? -Steve
Re: Replacing AA's in druntime
On 14/03/12 03:39, Jakob Bornecrantz wrote: On Wednesday, 14 March 2012 at 00:52:32 UTC, H. S. Teoh wrote: Hi all, My AA implementation is slowly inching closer to being ready to replace aaA.d. So far I've been writing the implementation outside of object_.d for ease of testing & development; now I'm ready to start moving stuff into object_.d to start working on integration with druntime. Hi, If I'm understanding this correctly you are moving the entire implementation of the AA into object.d and as such letting programs be purview to its inner working? In sort meaning you are making the entire AA implementation D ABI locked. This will make it impossible to either change the AA implementation in any ABI breaking fashion or make it impossible to pass AA's between libraries compiled against different versions of druntime. Much less so than the existing AA implementation.
Re: Turning a SIGSEGV into a regular function call under Linux, allowing throw
On 03/14/12 12:13, deadalnix wrote: > Le 13/03/2012 23:24, Vladimir Panteleev a écrit : >> I think something like this needs to end up in Druntime, at least for >> Linux x86 and x64. > > You are loosing EAX in the process. It's somewhat unavoidable. One way or another, you need to find _some_ threadlocal spot to stick those extra size_t.sizeof bytes, since you mustn't lose data, but the hack works by _overwriting_ the return address.
Re: toHash => pure, nothrow, const, @safe
On Wed, 14 Mar 2012 09:27:08 -0400, Martin Nowak wrote: In essence, when @type is encountered the compiler looks at TypeInfo_Struct (in object.di) for the equivalent xfuncname. Then uses the attributes of that function pointer (and also the parameter types/count) to compile the given method. Why would you want to add explicit annotation for implicit TypeInfo_Struct methods? Because right now, it's a guessing game of whether you wanted an operation to be part of the typeinfo's interface. And many times, the compiler guesses wrong. I've seen countless posts on d.learn saying "why won't AA's call my opEquals or opHash function?" With explicit annotation, you have instructed the compiler "I expect this to be in TypeInfo," so it can take the appropriate actions if it doesn't match. I think @type is a very interesting idea if combined with a string->method lookup in TypeInfo_Struct, but this wouldn't allow for static type checking. Yes it would. It has access to TypeInfo_Struct in object.di, so it can figure out what the signature should be. -Steve
Re: Fortran DLL and D
On 03/13/2012 05:15 PM, Michael wrote:
Hi everyone)
dmd 2.058
os: win 7 64 bit
fortran compilers: gfortran, ftn95
Here's apples to oranges, since I'm on linux 64 bit, but with your code
and this:
pragma(lib, "flib");
extern(C) void fsu_(int*i);
void main(){
int i = 1;
fsu_(&i);
}
when compiled on my box gives
The answer is x = 2.5004E-02 1
If something similar doesn't work for you, can you post disassembly
dumps of your dll function and calling function?
