Re: The Case Against Autodecode
On Friday, 3 June 2016 at 20:53:32 UTC, H. S. Teoh wrote: Even the Greek sigma has two forms depending on whether it's at the end of a word or not -- so should it be two code points or one? If you say two, then you'd have a problem with how to search for sigma in Greek text, and you'd have to search for either medial sigma or final sigma. But if you say one, then you'd have a problem with having two different letterforms for a single codepoint. In Unicode there are 2 different codepoints for lower case sigma ς U+03C2 and σ U+3C3 but only one uppercase Σ U+3A3 sigma. Codepoint U+3A2 is undefined. So your objection is not hypothetic, it is actually an issue for uppercase() and lowercase() functions. Another difficulty besides dotless and dotted i of Turkic, the double letters used in latin transcription of cyrillic text in east and south europe dž, lj, nj and dz, which have an uppercase forme (DŽ, LJ, NJ, DZ) and a titlecase form (Dž, Lj, Nj, Dz). Besides, that still doesn't solve the problem of what "i".uppercase() should return. In most languages, it should return "I", but in Turkish it should not. And if we really went the route of encoding Cyrillic letters the same as their Latin lookalikes, we'd have a problem with what "m".uppercase() should return, because now it depends on which font is in effect (if it's a Cyrillic cursive font, the correct answer is "Т", if it's a Latin font, the correct answer is "M" -- the other combinations: who knows). That sounds far worse than what we have today. As an anecdote I can tell the story of the accession to the European Union of Romania and Bulgaria in 2007. The issue was that 3 letters used by Romanian and Bulgarian had been forgotten by the Unicode consortium (Ș U+0218, ș U+219, Ț U+21A, ț U+21B and 2 Cyrillic letters that I do not remember). The Romanian used as a replacement Ş, ş, Ţ and ţ (U+15D, U+15E and U+161 and U+162), which look a little bit alike. When the Commission finally managed to force Mirosoft to correct the fonts to include them, we could start to correct the data. The transition was finished in 2012 and was only possible because no other language we deal with uses the "wrong" codepoints (Turkish but fortunately we only have a handful of them in our db's). So 5 years of ad hoc processing for the substicion of 4 codepoints. BTW: using combining diacritics was out of the question at that time simply because Microsoft Word didn't support it at that time and many documents we encountered still only used codepages (one has also to remember that in big institution like the EC, the IT is always several years behind the open market, which means that when product is in release X, the Institution still might use release X-5 years).
Re: The Case Against Autodecode
One has also to take into consideration that Unicode is the way it is because it was not invented in an empty space. It had to take consideration of the existing and find compromisses allowing its adoption. Even if they had invented the perfect encoding, NO ONE WOULD HAVE USED IT, as it would have fubar the existing. As it was invented it allowed a (relatively smooth) transition. Here some points that made it even possible that Unicode could be adopted at all: - 16 bits: while that choice was a bit shortsighted, 16 bits is a good compromice between compactness and richness (BMP suffice to express nearly all living languages). - Using more or less the same arrangement of codepoints as in the different codepages. This allowed to transform legacy documents with simple scripts (matter of fact I wrote a script to repair misencoded Greek documents, it consisted mainly of unich = ch>0x80 ? ch+0x2D0 : ch; - Utf-8: this was the genious stroke encoding that allowed to mix it all without requiring awful acrobatics (Joakim is completely out to lunch on that one, shifting encoding without self-synchronisation are hellish, that's why Chinese and Japanese adopted Unicode without hesitation, they had enough experience with their legacy encodings. - Letting time for the transition. So all the points that people here criticize, were in fact the reason why Unicode could even be become the standard it is now.
Re: The Case Against Autodecode
On 6/3/2016 11:17 PM, H. S. Teoh via Digitalmars-d wrote:
On Fri, Jun 03, 2016 at 08:03:16PM -0700, Walter Bright via Digitalmars-d wrote:
It works for books.
Because books don't allow their readers to change the font.
Unicode is not the font.
This madness already exists *without* Unicode. If you have a page with a
single glyph 'm' printed on it and show it to an English speaker, he
will say it's lowercase M. Show it to a Russian speaker, and he will say
it's lowercase Т. So which letter is it, M or Т?
It's not a problem that Unicode can solve. As you said, the meaning is in the
context. Unicode has no context, and tries to solve something it cannot.
('m' doesn't always mean m in english, either. It depends on the context.)
Ya know, if Unicode actually solved these problems, you'd have a case. But it
doesn't, and so you don't :-)
If you're going to represent both languages, you cannot get away from
needing to represent letters abstractly, rather than visually.
Books do visually just fine!
So should O and 0 share the same glyph or not? They're visually the same
thing,
No, they're not. Not even on old typewriters where every key was expensive. Even
without the slash, the O tends to be fatter than the 0.
The very fact that we distinguish between O and 0, independently of what
Unicode did/does, is already proof enough that going by visual
representation is inadequate.
Except that you right now are using a font where they are different enough that
you have no trouble at all distinguishing them without bothering to look it up.
And so am I.
In other words toUpper and toLower does not belong in the standard
library. Great.
Unicode and the standard library are two different things.
Re: The Case For Autodecode
On Friday, 3 June 2016 at 20:18:31 UTC, Steven Schveighoffer
wrote:
On 6/3/16 3:52 PM, ag0aep6g wrote:
On 06/03/2016 09:09 PM, Steven Schveighoffer wrote:
Except many chars *do* properly convert. This should work:
char c = 'a';
dchar d = c;
assert(d == 'a');
Yeah, that's what I meant by "standalone code unit". Code
units that on
their own represent a code point would not be touched.
But you can get a standalone code unit that is part of a coded
sequence quite easily
foo(string s)
{
auto x = s[0];
dchar d = x;
}
As I mentioned in my earlier reply, some kind of "bounds
checking" for
the conversion could be a possibility.
Hm... an interesting possiblity:
dchar _dchar_convert(char c)
{
return cast(int)cast(byte)c; // get sign extension for
non-ASCII
}
So when the char's most significant bit is set, this fills the
upper
bits of the dchar with 1s, right? And a set most significant
bit in a
char means it's part of a multibyte sequence, while in a dchar
it means
that the dchar is invalid, because they only go up to
U+10. Huh. Neat.
An interesting thing is that I think the CPU can do this for us.
Does it work for for char -> wchar, too?
It does not. 0x is a valid code point, and I think so are
all the other values that would result. In fact, I think there
are no invalid code units for wchar.
https://codepoints.net/specials
U+ would be fine, better at least than a surrogate.
Re: Lifetime tracking
On 6/3/2016 4:38 AM, Timon Gehr wrote: I'm sure there is some syntax that will feel natural to D programmers. Aye, there's the rub. Nobody has found one in over 10 years, despite a lot of discussion.
Re: Lifetime tracking
On 6/4/2016 2:14 AM, Walter Bright wrote: On 6/3/2016 4:38 AM, Timon Gehr wrote: I'm sure there is some syntax that will feel natural to D programmers. Aye, there's the rub. Nobody has found one in over 10 years, despite a lot of discussion. No one found one for Rust, either.
Re: Free the DMD backend
On Fri, 2016-06-03 at 07:12 +, Iain Buclaw via Digitalmars-d wrote: > On Friday, 3 June 2016 at 03:17:56 UTC, Eugene Wissner wrote: > > On Thursday, 2 June 2016 at 18:16:33 UTC, Basile B. wrote: > > > It's also that LDC is at front end 2.070 and GDC 2.067 ;););) > > > > > > GDC is actively maintained and it would have the latest > > features if more developers come, what would happen if it would > > be the reference compiler. > > LDC is also no more open than DMD. Go figure! Is that true. Debian and Fedora package LDC but they will not package DMD. GDC has to work within the GCC release framework so I think probably not a good context for the D reference compiler. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: The Case Against Autodecode
On 03/06/2016 20:12, Dmitry Olshansky wrote: On 02-Jun-2016 23:27, Walter Bright wrote: I wonder what rationale there is for Unicode to have two different sequences of codepoints be treated as the same. It's madness. Yeah, Unicode was not meant to be easy it seems. Or this is whatever happens with evolutionary design that started with "everything is a 16-bit character". Typing as someone who as spent some time creating typefaces, having two representations makes sense, and it didn't start with Unicode, it started with movable type. It is much easier for a font designer to create the two codepoint versions of characters for most instances, i.e. make the base letters and the diacritics once. Then what I often do is make single codepoint versions of the ones I'm likely to use, but only if they need more tweaking than the kerning options of the font format allow. I'll omit the history lesson on how this was similar in the case of movable type. Keyboards for different languages mean that a character that is a single keystroke in one case is two together or in sequence in another. This means that Unicode not only represents completed strings, but also those that are mid composition. The ordering that it uses to ensure that graphemes have a single canonical representation is based on the order that those multi-key characters are entered. I wouldn't call it elegant, but its not inelegant either. Trying to represent all sufficiently similar glyphs with the same codepoint would lead to a layout problem. How would you order them so that strings of any language can be sorted by their local sorting rules, without having to special case algorithms? Also consider ligatures, such as those for "ff", "fi", "ffi", "fl", "ffl" and many, many more. Typographers create these glyphs whenever available kerning tools do a poor job of combining them from the individual glyphs. From the point of view of meaning they should still be represented as individual codepoints, but for display (electronic or print) that sequence needs to be replaced with the single codepoint for the ligature. I think that in order to understand the decisions of the Unicode committee, one has to consider that they are trying to unify the concerns of representing written information from two sides. One side prioritises storage and manipulation, while the other considers aesthetics and design workflow more important. My experience of using Unicode from both sides gives me a different appreciation for the difficulties of reconciling the two. A... P.S. Then they started adding emojis, and I lost all faith in humanity ;)
Re: Cool D tricks resources?
On Saturday, 4 June 2016 at 03:05:59 UTC, Seb wrote: On Saturday, 4 June 2016 at 01:27:46 UTC, Mike Parker wrote: On Saturday, 4 June 2016 at 00:55:33 UTC, Pie? wrote: Has anyone gathered up all the cool tricks one can do in D? There are a ton of things D can do differently that are optimal in some cases but I never remember them. A sort of cook book of D tricks for different applications would be nice Take a look at http://p0nce.github.io/d-idioms/. and the gems from our DLang Tour - tour.dlang.io @Audience: We are still actively looking for people who want to add a "Gem" to the tour. So if you have a favorite feature in D that you miss there, ping us! why not using gitbook and a domain like dlangbyexample.com (mine. :)) that I would lovely donate it to foundation. example: http://rustbyexample.com/
Re: The Case For Autodecode
On Friday, 3 June 2016 at 11:24:40 UTC, ag0aep6g wrote: Finally, this is not the only argument in favor of *keeping* autodecoding, of course. Not wanting to break user code is the big one there, I guess. I'm not familiar with the details of autodecoding, but one thing strikes me about this whole discussion. It seems to me that it is just nibbling around the edges of how one should implement full Unicode support. And it seems to me that that topic, and how autodecoding plays into it, won't be properly understood except by comparison with mature software that has undergone many years of testing and revision. Two examples stand out to me: * Perl 5 has undergone a gradual evolution, over many releases, to get this right. It might also be the case that Perl 6 is even cleaner. * The International Components for Unicode (ICU) package, with supported libraries for C, C++, and Java. This is the industry- standard definition of what it means to handle Unicode in these languages. See http://site.icu-project.org/ for details. Both of these implementations have seen many years of real-world use, so I would tend to look to them for guidance over trying to develop my own opinion based on some small set of particular use cases I might happen to have encountered.
Re: [OT] Things I like about Andrei
Thank you folks for the kind words. It has been an honor to work with you all on the D language. -- Andrei
Re: Blocking points for further D adoption
The largest blocking point to me is the community attitude. D constantly wants to 'rule them all' instead of integrating with other language ecosystems. This only recently started to change, but only towards C/C++ and not in the other direction, which is dynamic languages. PyD is only barely alive, and nobody seems to be interested to take it to the next level—of making it easy to distribute the created packages. I'm speaking here from a researcher's perspective. One must realize that in our universe, there is often no time to learn yet another language, so people consolidate around Python so that everyone stays productive, and this situation will not change until someone rolls out a complete replacement for numpy, scipy, pandas, and scikit-learn at the very least. (and that won't happen any time soon) A fancy custom Jupyter kernel is nice but often half-baked and not really necessary. But solving distribution of shared libraries is a must if you (still) want to become a C++ replacement. To me it seems that D currently has a unique advantage of being able to easily generate in compile time all the boilerplate binding code that everybody hates to write in C++ (or if one uses boost::python, hates to wait to compile). Combine that with the fact that many are terrified of C/C++ insomuch that Cython was invented, and D offers a much nicer language with GC for those who don't want to even know about memory management. Research people would love this, but only if it's a production-ready solution that needs no extra time investment.
Re: Lifetime tracking
On 3 June 2016 at 10:40, Stefan Koch via Digitalmars-d wrote: > On Friday, 3 June 2016 at 00:31:31 UTC, Walter Bright wrote: > >> If they cover the cases that matter, it's good. Rust has the type system >> annotations you want, but Rust has a reputation for being difficult to write >> code for. > > > I think we can incorporate typesafe borrowing without making it difficult to > write. I also haven't heard any reports from colleagues who have tried out rust to a decent extent that borrow pointers are among the things that make rust code hard to write. 'rust is hard' != 'borrowing is hard' .. also, I don't necessarily think 'scope' would be identical to rust, it would be designed for D, obviously.
Re: Lifetime tracking
On 4 June 2016 at 19:14, Walter Bright via Digitalmars-d wrote: > On 6/3/2016 4:38 AM, Timon Gehr wrote: >> >> I'm sure there is some syntax that >> will feel natural to D programmers. > > > Aye, there's the rub. Nobody has found one in over 10 years, despite a lot > of discussion. What is unnatural about 'scope' as has been proposed by heaps of people? I find DIP25 much more awkward and unnatural than 'scope'.
Re: The Case For Autodecode
On 6/4/16 4:57 AM, Patrick Schluter wrote: On Friday, 3 June 2016 at 20:18:31 UTC, Steven Schveighoffer wrote: On 6/3/16 3:52 PM, ag0aep6g wrote: Does it work for for char -> wchar, too? It does not. 0x is a valid code point, and I think so are all the other values that would result. In fact, I think there are no invalid code units for wchar. https://codepoints.net/specials U+ would be fine, better at least than a surrogate. U+ is still a valid code point, even if it's not assigned any unicode character. But the result would be U+ff80 to U+, and I'm sure some of those are valid code points. -Steve
Re: Request for approval: deploying dlang.org on merge
On 06/03/2016 05:43 PM, Vladimir Panteleev wrote: On Thursday, 26 May 2016 at 16:38:16 UTC, Andrei Alexandrescu wrote: If Vladimir is okay with auto-deployment I'm okay too. Move fast! This is now live. (Perhaps it should be posted in Announce?) This is awesome, thank you guys for making it happen! The only question is, how do we avoid cases in which a deployment breaks something important? -- Andrei 1. Check the documentation tester results before merging anything. This also applies to other components (Phobos, Druntime, DMD) though most changes there are unlikely to break things. 2. If something breaks, make and merge a PR which reverts or fixes it. Pushing directly to master would also work, but wouldn't make a big difference in practice. 3. If that doesn't work, people with access can always ssh to dl...@k3.1azy.net or dlang.org to fix things by hand. Sounds like a good seed for a wiki page. Andrei
Re: Blocking points for further D adoption
On Saturday, 4 June 2016 at 13:18:02 UTC, Artem Tarasov wrote: The largest blocking point to me is the community attitude. D constantly wants to 'rule them all' instead of integrating with other language ecosystems. This only recently started to change, but only towards C/C++ and not in the other direction, which is dynamic languages. PyD is only barely alive, and nobody seems to be interested to take it to the next level—of making it easy to distribute the created packages. Fair point :/ I'm speaking here from a researcher's perspective. One must realize that in our universe, there is often no time to learn yet another language, so people consolidate around Python so that everyone stays productive, and this situation will not change until someone rolls out a complete replacement for numpy, scipy, pandas, and scikit-learn at the very least. (and that won't happen any time soon) We are working on it - see mir.dlang.io The more people help, the faster it will happen!
Shared, but synchronisation is not desired for multithreading
If you ignore the discouraged __gshared keyword, to be able to
share a variable between threads, you need to be using "shared"
keyword.
While designing your class with "shared" methods, the compiler
directly assumes that objects of this class must be protected
against threading problems.
There can be three USAGEs of a class object:
1. It will be non-shared. So, it is stored in TLS, and only one
thread can access it.
2. It will be shared. But programmer knows that the object is
designed as "shared" with the purpose of reading its value from
multiple threads.
3. It will be shared. But the object must be synchronised.
Because programmer knows that multiple threads will be reading
from and writing to object.
Currently, in a normal coding environment (I am not talking about
using extra parameters, increasing complexity etc),
distinguishing between 2 and 3 does not seem like possible. You
prepare your shared class, and its methods are designed to be
either sycnhronised or not synchronised. There is no middle point
unless you define the same method with different names, or use a
flag like "bool run_this_method_synchronised_please".
So, what I did is using UDA for this with the name @ThreadSafe.
e.g.
@ThreadSafe auto myObject = new shared MyClass();
In a method of the class, I make the declaration as following:
public void foo() shared{
static if( std.traits.hasUDA!( this, ThreadSafe ) ){
// lock mutex
scope(exit){
// unlock mutex
}
}
// do your normal operations
}
This way, if the object is desired to be doing synchronisation,
you only add an attribute to it.
There are some small problems here, those are related to D's
implementation right now:
1. There is no standard way of saying @ThreadSafe. You are
supposed to be defining it. If language was to be defining a
standard attribute as @ThreadSafe, it could be used everywhere
for this purpose.
2. If a method is defined as shared, compiler immediately warns
the programmer to use core.atomic.atomicOp. If codes are already
being designed as thread-safe by the programmer, normal variable
operations could be used without any concern.
3. As far as I remember, there were some talks about synchronized
keyword not being used much. Maybe its usage could be changed to
support this @ThreadSafe system.
Re: Cool D tricks resources?
On Saturday, 4 June 2016 at 10:02:44 UTC, bob belcher wrote: On Saturday, 4 June 2016 at 03:05:59 UTC, Seb wrote: On Saturday, 4 June 2016 at 01:27:46 UTC, Mike Parker wrote: On Saturday, 4 June 2016 at 00:55:33 UTC, Pie? wrote: Has anyone gathered up all the cool tricks one can do in D? There are a ton of things D can do differently that are optimal in some cases but I never remember them. A sort of cook book of D tricks for different applications would be nice Take a look at http://p0nce.github.io/d-idioms/. and the gems from our DLang Tour - tour.dlang.io @Audience: We are still actively looking for people who want to add a "Gem" to the tour. So if you have a favorite feature in D that you miss there, ping us! why not using gitbook and a domain like dlangbyexample.com (mine. :)) that I would lovely donate it to foundation. example: http://rustbyexample.com/ We could do that too! For the Dlang Tour (which is written entirely in D) being independent (GitBook is a company) and able to customize it, is important. For more infos and discussion, I created an issue: https://github.com/stonemaster/dlang-tour/issues/209
Re: Shared, but synchronisation is not desired for multithreading
On Saturday, 4 June 2016 at 15:11:51 UTC, tcak wrote:
If you ignore the discouraged __gshared keyword, to be able to
share a variable between threads, you need to be using "shared"
keyword.
While designing your class with "shared" methods, the compiler
directly assumes that objects of this class must be protected
against threading problems.
There can be three USAGEs of a class object:
1. It will be non-shared. So, it is stored in TLS, and only one
thread can access it.
2. It will be shared. But programmer knows that the object is
designed as "shared" with the purpose of reading its value from
multiple threads.
3. It will be shared. But the object must be synchronised.
Because programmer knows that multiple threads will be reading
from and writing to object.
Currently, in a normal coding environment (I am not talking
about using extra parameters, increasing complexity etc),
distinguishing between 2 and 3 does not seem like possible. You
prepare your shared class, and its methods are designed to be
either sycnhronised or not synchronised. There is no middle
point unless you define the same method with different names,
or use a flag like "bool run_this_method_synchronised_please".
So, what I did is using UDA for this with the name @ThreadSafe.
e.g.
@ThreadSafe auto myObject = new shared MyClass();
In a method of the class, I make the declaration as following:
public void foo() shared{
static if( std.traits.hasUDA!( this, ThreadSafe ) ){
// lock mutex
scope(exit){
// unlock mutex
}
}
// do your normal operations
}
This way, if the object is desired to be doing synchronisation,
you only add an attribute to it.
There are some small problems here, those are related to D's
implementation right now:
1. There is no standard way of saying @ThreadSafe. You are
supposed to be defining it. If language was to be defining a
standard attribute as @ThreadSafe, it could be used everywhere
for this purpose.
2. If a method is defined as shared, compiler immediately warns
the programmer to use core.atomic.atomicOp. If codes are
already being designed as thread-safe by the programmer, normal
variable operations could be used without any concern.
3. As far as I remember, there were some talks about
synchronized keyword not being used much. Maybe its usage could
be changed to support this @ThreadSafe system.
If the method is thread-safe, it should be marked as shared.
Otherwise, it should not be shared. I think you're trying to mark
functions that aren't actually thread safe but you call in a
`synchronized` context as shared, when they should not be.
If you've made guarantees that the shared object you are
modifying can only be accessed by one thread, you can cast it to
unshared, and call its thread-unsafe methods (which are now
safe). I think there were plans to get `synchronized` to do this
for you, but it doesn't, which makes it fairly unwieldy.
(It also doesn't help that many "thread-safe" functions in D
aren't marked as shared where they really ought to be, ex. all
the functions in core.sync.mutex)
Re: Blocking points for further D adoption
On Saturday, 4 June 2016 at 13:18:02 UTC, Artem Tarasov wrote: which is dynamic languages. PyD is only barely alive, and nobody seems to be interested to take it to the next level—of making it easy to distribute the created packages. D integrates quite easily with R. I speak from experience, regularly using the two together. You can embed R inside your D program and pass data trivially between them. The technical side is not an issue. Distribution is not an issue - it is the same as any D program calling into a C library. All the other stuff (support for three OSes, documentation, etc.) takes a lot of time and is no fun. Research people would love this, but only if it's a production-ready solution that needs no extra time investment. Yep. And if nobody wants to do the work, it will never happen. I originally did it as a fun way to pass the time while waiting for my son at his many events. I may have been the only one in the world with that particular set of circumstances though. You'll probably be waiting a long time for that production-ready solution that needs no extra time investment.
Fixed date to move to ddox for Phobos documentation
More than two and half years ago, Sönke added ddox builds for the Phobos documentation. We all know that there are many reasons for ddox - being able to generate single pages for methods is just one, it also eliminates all the JavaScript hacks (e.g. the quickindex menu, anchors, ...) that we have added over time to deal with the shortcomings of ddoc. This post originates from a recent discussion [2] that showed the higher ranking of the ddox pages in search engines because of those single pages, more static content and meta information. To quote Adam [3]: ddox got a decent go up to here. But then we need to decide what's next - a clear goal, including a due date, gets us all aligned and removes a lot of the uncertainty on the author's side; it is some reassurance that they aren't wasting their time, and encourages outside teams to get onboard. We got the MREF change into Phobos a month ago and Sönke has fixed the last blocking bug with ddox (broken source code links) a couple of days ago. Imho it's quite impressive that he still pushes the project and as Adam correctly said - we need to make a decision and have a clear deadline like 2.072 will be the last documentation build with ddoc, once it's released we will remove the ddoc Phobos build and make ddox (/library) the standard (with redirect, of course). This gives us also two to three months to test it properly again (it has been tested now for 2.5 years!!) and resolve issues if occurring. Does this sound reasonable to everyone? [1] https://github.com/dlang/dlang.org/pull/267 [2] http://forum.dlang.org/post/575079de.2040...@erdani.org [3] http://forum.dlang.org/post/zuzlvgqposlmtgdnx...@forum.dlang.org
clipboard button to copy selective import from documentation?
I'm finding myself writing a lot of selective imports nowaday, while also browsing the documentation (sometimes even using dman). I though it might be a nice to have a "copy-as-selective-import" button next to the anchor link. Then once you find what you've been looking for, e.g. http://dlang.org/phobos/std_math.html#.approxEqual, you can click on it and paste import std.math : approxEqual; into your code (and edit it accordingly). Useful? What do you think? Anyone volunteering? -Martin NB: We use clipboard.js on the dub registry, a JS only clipboard solution.
Re: Blocking points for further D adoption
On Sat, Jun 4, 2016 at 5:56 PM, bachmeier via Digitalmars-d < digitalmars-d@puremagic.com> wrote: > > > D integrates quite easily with R. I speak from experience, regularly using > the two together. You can embed R inside your D program and pass data > trivially between them. The technical side is not an issue. Distribution is > not an issue - it is the same as any D program calling into a C library. > All the other stuff (support for three OSes, documentation, etc.) takes a > lot of time and is no fun. Embedding is easy indeed, and it is rather fun (also speaking from experience), but it again shifts the focus towards D, while most people would like to call D from R, and that's where things start to go awry, especially once you start to think about all the complicacies of cross-platform/multithreaded/GC-reliant extensions, possibly several loaded at the same time. That requires quite a bit of enthusiasm that I admittedly no longer have.
Re: Shared, but synchronisation is not desired for multithreading
On Saturday, 4 June 2016 at 15:51:22 UTC, Alex Parrill wrote: (It also doesn't help that many "thread-safe" functions in D aren't marked as shared where they really ought to be, ex. all the functions in core.sync.mutex) And you have to be continuously casting the methods of Mutex, Thread, Condition, and others wherever you use them in a shared class which makes a mess everywhere. Only a person who uses those classes in many occasions can understand this situation. Thus, I have started this topic.
Re: Fixed date to move to ddox for Phobos documentation
On 06/04/2016 12:10 PM, Seb wrote: More than two and half years ago, Sönke added ddox builds for the Phobos documentation. We all know that there are many reasons for ddox - being able to generate single pages for methods is just one, it also eliminates all the JavaScript hacks (e.g. the quickindex menu, anchors, ...) that we have added over time to deal with the shortcomings of ddoc. This post originates from a recent discussion [2] that showed the higher ranking of the ddox pages in search engines because of those single pages, more static content and meta information. To quote Adam [3]: ddox got a decent go up to here. But then we need to decide what's next - a clear goal, including a due date, gets us all aligned and removes a lot of the uncertainty on the author's side; it is some reassurance that they aren't wasting their time, and encourages outside teams to get onboard. We got the MREF change into Phobos a month ago and Sönke has fixed the last blocking bug with ddox (broken source code links) a couple of days ago. Imho it's quite impressive that he still pushes the project and as Adam correctly said - we need to make a decision and have a clear deadline like 2.072 will be the last documentation build with ddoc, once it's released we will remove the ddoc Phobos build and make ddox (/library) the standard (with redirect, of course). This gives us also two to three months to test it properly again (it has been tested now for 2.5 years!!) and resolve issues if occurring. Does this sound reasonable to everyone? [1] https://github.com/dlang/dlang.org/pull/267 [2] http://forum.dlang.org/post/575079de.2040...@erdani.org [3] http://forum.dlang.org/post/zuzlvgqposlmtgdnx...@forum.dlang.org I recall there were a few issues with ddox rendering up until relatively recently, have all been fixed? I don't see these options mutually exclusive, though I do want to make ddox the default/preferred rendering if ready. Consider e.g. https://www.gnu.org/software/make/manual/ which offers the documentation in various formats, including fewer/multiple pages. Now, I gave myself a few minutes to take a look at ddox: * http://dlang.org/phobos/ looks nicer than http://dlang.org/library/. The intro text is not very important, but the grouping is nice. The table vertical split ratio is also nicer. * I assume the dead links matter will be solved. * The cheat sheet made sense for the single-page docs but not for the new ones. Consider e.g. http://dlang.org/library/std/algorithm/comparison.html - it's two tables with the same row headings one after another. The information should be consolidated in one table (I'm speaking from the user's perspective; I know it's hard). * There are a few formatting issues that I like less in the new docs. These are of course fixable with relative ease. Looking at e.g. http://dlang.org/library/std/algorithm/comparison/among.html: ** the vertical spacing is generally too fluffy (do we really need 25% of the page occupied by Authors and License for _every_ function?) ** Do we need the actual "Prototypes" heading or is it implicit? ** The booktable style is nice for tables that span the entire page width; for parameters the boxes at http://dlang.org/phobos/std_algorithm_comparison.html#among seem nicer to me. ** Do we need the actual heading "Parameters", or it suffices to put "Parameter" instead of "Name" in the parameter box heading? ** Examples are nice but again vertical spacing overall and between consecutive examples is too large. ** Generally the sentiment of the formatting is "too much foreplay". Get to the point already, don't have people scroll through fluff. Relegate Authors and License to a small footnote below everything - even after disqus. * The hotlinks to parameters (e.g. "value", "values") seem unnecessary unless the synopsis is very long which shouldn't be the case. It's also inconsistent - pred is not hotlinked. * How are we going to deal with links (possibly with hashes) pointing to the existing artifacts? Again the simplest way is to keep the old docs but just deemphasize them on dlang.org. * I'd like a "Show entire module documentation as single page" link for e.g. http://dlang.org/library/std/algorithm/comparison.html. * How do we address the ebook/pdf production? Do we continue using ddoc or look into a ddox approach? * Any other extant issues with /library/ (I recall there were a few cases in which not all text was rendered)? Andrei
Re: Fixed date to move to ddox for Phobos documentation
On 06/04/2016 07:01 PM, Andrei Alexandrescu wrote: * The cheat sheet made sense for the single-page docs but not for the new ones. Consider e.g. http://dlang.org/library/std/algorithm/comparison.html - it's two tables with the same row headings one after another. The information should be consolidated in one table (I'm speaking from the user's perspective; I know it's hard). I have a pull request open that addresses the issue: https://github.com/dlang/phobos/pull/4397 It hides the handcrafted cheat sheets. It's not been pulled yet, because the handcrafted cheat sheets may be better than the generated ones. I haven't been able to figure out how to hide the generated ones. Also, the handcrafted ones tend to get outdated which is especially bad with DDOX, because you can't just scroll down to it.
Re: Fixed date to move to ddox for Phobos documentation
Am 04.06.2016 um 19:18 schrieb ag0aep6g: On 06/04/2016 07:01 PM, Andrei Alexandrescu wrote: * The cheat sheet made sense for the single-page docs but not for the new ones. Consider e.g. http://dlang.org/library/std/algorithm/comparison.html - it's two tables with the same row headings one after another. The information should be consolidated in one table (I'm speaking from the user's perspective; I know it's hard). I have a pull request open that addresses the issue: https://github.com/dlang/phobos/pull/4397 It hides the handcrafted cheat sheets. It's not been pulled yet, because the handcrafted cheat sheets may be better than the generated ones. I haven't been able to figure out how to hide the generated ones. Also, the handcrafted ones tend to get outdated which is especially bad with DDOX, because you can't just scroll down to it. I think ideally we could do something like introducing a special "Cheatsheet" section or macro for each function that DDOX could aggregate or use instead of the normal short description. Also interesting (although less so since we started using sub modules) would be to do the same for a group/category that would be used to group symbols by other means that simply their type category. The downside is that the normal Ddoc generated docs couldn't use that information to display the overview tables like it does now, so it would make most sense to do this after making the switch.
Let's avoid range specializations - make ranges great again!
In Phobos (especially in std.algorithm) a lot of specialization
between a RandomAccessRange and an InputRange are used even
though only an InputRange is required.
The reason for this is solely performance - imho we should start
to tackle this and remove such specializations. Let's create
better code!
To quote David from the recent PR
(https://github.com/dlang/phobos/pull/4265) where adding a
identity specialization achieves a 5-10x speedup for
{min,max}Element.
I think that we have a larger issue at hand here. Ranges are of
prime strategic importance for D as an accessible way of
composing complex operations out of reusable primitives. A
large part of this comes from the fact that they are supposed
to be zero-cost abstractions – after the compiler optimiser has
had its way with a piece of code, we expect it to be equivalent
to a hand-written loop.
1) Should it matter whether generic range looping or
"specialized" r[i] access is used?
Answer for dmd it does (7x slower), ldc does the job (nearly)
correctly.
```
dmd -release -O -boundscheck=off test_looping.d &&
./test_looping
random_access 685 ms, 351 μs, and 9 hnsecs
foreach 685 ms, 888 μs, and 4 hnsecs
foreach_ref 685 ms, 654 μs, and 4 hnsecs
for_range_api 7 secs, 211 ms, 530 μs, and 3 hnsecs
```
ldc -release -O3 -boundscheck=off test_looping.d &&
./test_looping
random_access 142 ms and 88 μs
foreach 142 ms, 8 μs, and 3 hnsecs
foreach_ref 142 ms, 400 μs, and 9 hnsecs
for_range_api 143 ms, 394 μs, and 1 hnsec
```
Benchmarking code:
https://github.com/wilzbach/perf-ranges/blob/master/test_looping.d
Bug report for DMD: https://issues.dlang.org/show_bug.cgi?id=16120
(Tested with both ldc 0.17 and 1.0)
2) Should it matter whether I use r.save or r[i..$]?
Answer: it does for dmd and ldc.
```
dmd -release -O -boundscheck=off test_save.d && ./test_save
random_access 424 ms and 72 μs
foreach_random 420 ms, 754 μs, and 5 hnsecs
for_range_api 2 secs, 562 ms, 415 μs, and 5 hnsecs
ldc -release -O3 -boundscheck=off test_save.d && ./test_save
random_access 245 ms, 713 μs, and 4 hnsecs
foreach_random 241 ms, 533 μs, and 1 hnsec
for_range_api 349 ms, 582 μs, and 9 hnsecs
```
Benchmarking code:
https://github.com/wilzbach/perf-ranges/blob/master/test_save.d
Open question: how can we make DMD/LDC smarter, so that they
inline range primitives and the promise about the zero-cost
abstractions is full-filled?
Re: Fixed date to move to ddox for Phobos documentation
On 06/04/2016 07:25 PM, Sönke Ludwig wrote: I think ideally we could do something like introducing a special "Cheatsheet" section or macro for each function that DDOX could aggregate or use instead of the normal short description. Also interesting (although less so since we started using sub modules) would be to do the same for a group/category that would be used to group symbols by other means that simply their type category. The downside is that the normal Ddoc generated docs couldn't use that information to display the overview tables like it does now, so it would make most sense to do this after making the switch. Thinking about this, if the information in the cheat sheet is the most helpful, maybe we should be putting exactly that in the summaries of things. Meaning, maybe we should change summaries to be one short sentence and/or one short example.
Re: clipboard button to copy selective import from documentation?
On Saturday, 4 June 2016 at 16:10:26 UTC, Martin Nowak wrote: I'm finding myself writing a lot of selective imports nowaday, while also browsing the documentation (sometimes even using dman). I though it might be a nice to have a "copy-as-selective-import" button next to the anchor link. Then once you find what you've been looking for, e.g. http://dlang.org/phobos/std_math.html#.approxEqual, you can click on it and paste import std.math : approxEqual; into your code (and edit it accordingly). Useful? What do you think? Yes. Anyone volunteering? How about https://github.com/dlang/dlang.org/pull/1330?
Re: clipboard button to copy selective import from documentation?
On Saturday, 4 June 2016 at 16:10:26 UTC, Martin Nowak wrote: Useful? What do you think? Anyone volunteering? Generally I'm not a fan of bloating things up with JavaScript widgets. Perhaps only if there is a lot of demand for this? Perhaps we should improve editor support instead. For my previous editor I had this nice setup: http://dump.thecybershadow.net/e91be687ebaeb0171d830025adf82848/autofix.gif I've added a json target to the Phobos posix.mak a while ago. Care to beat me to writing the Emacs Lisp port? :) DCD would also be one way of achieving this (one advantage of its current gobble-up-everything model).
Re: Fixed date to move to ddox for Phobos documentation
Am 04.06.2016 um 19:01 schrieb Andrei Alexandrescu: I recall there were a few issues with ddox rendering up until relatively recently, have all been fixed? I think they have. Vladimir has reported a bunch of them over time and all of those have been fixed. I don't see these options mutually exclusive, though I do want to make ddox the default/preferred rendering if ready. Consider e.g. https://www.gnu.org/software/make/manual/ which offers the documentation in various formats, including fewer/multiple pages. Now, I gave myself a few minutes to take a look at ddox: * http://dlang.org/phobos/ looks nicer than http://dlang.org/library/. The intro text is not very important, but the grouping is nice. The table vertical split ratio is also nicer. This is currently just the generic module overview. We could simply replace it by a handcrafted version. Shall I copy the one from /phobos/? ...or I could also just read the corresponding .ddoc file and render that. * I assume the dead links matter will be solved. * The cheat sheet made sense for the single-page docs but not for the new ones. Consider e.g. http://dlang.org/library/std/algorithm/comparison.html - it's two tables with the same row headings one after another. The information should be consolidated in one table (I'm speaking from the user's perspective; I know it's hard). Leaving that to the other sub thread. * There are a few formatting issues that I like less in the new docs. These are of course fixable with relative ease. Looking at e.g. http://dlang.org/library/std/algorithm/comparison/among.html: ** the vertical spacing is generally too fluffy (do we really need 25% of the page occupied by Authors and License for _every_ function?) I intended to tackle that multiple times. Definitely agreed that it should be much smaller. On the plus side, it's placed after all of the real meat of the page (except for the comments), so it doesn't really do any harm. ** Do we need the actual "Prototypes" heading or is it implicit? That's definitely debatable. The first special purpose sections could be layed out in a few different ways: Primary heading brief description [prototypes] detailed description Parameters and other sections or maybe Primary heading brief description detailed description [prototypes] [parameters] or maybe Primary heading brief description [prototypes] [parameters] detailed description They all have their pros and cons, and those also depend on what is displayed on the parent page. If, for example, the parent page already displays the prototypes (which I intended to add in some form), moving them further down below the page can make sense. Otherwise it can be good to have them near the top, so that the reader gets an early impression about the function, before reading through the detailed description. ** The booktable style is nice for tables that span the entire page width; for parameters the boxes at http://dlang.org/phobos/std_algorithm_comparison.html#among seem nicer to me. While I don't particularly like those either (but I'll leave the design discussion to other people) - I can of course add that right away. I just used the default table style there and simply didn't notice the style change in /phobos/. ** Do we need the actual heading "Parameters", or it suffices to put "Parameter" instead of "Name" in the parameter box heading? Depending on the general page layout we should definitely be able to drop the heading. ** Examples are nice but again vertical spacing overall and between consecutive examples is too large. Partially agreed, the spacing could indeed be reduced a bit, however, one of the big advantages of the single-page layout is that we can use more vertical space to give the page more visual structure and thus make it easier to scan for a particular section. ** Generally the sentiment of the formatting is "too much foreplay". Get to the point already, don't have people scroll through fluff. Relegate Authors and License to a small footnote below everything - even after disqus. Authors+License sure, but the rest is content, right? Or what else would you consider foreplay? * The hotlinks to parameters (e.g. "value", "values") seem unnecessary unless the synopsis is very long which shouldn't be the case. It's also inconsistent - pred is not hotlinked. You are probably right. Linking to template arguments is currently not supported, so dropping parameter linking in general is definitely the more comfortable change ;-) * How are we going to deal with links (possibly with hashes) pointing to the existing artifacts? Again the simplest way is to keep the old docs but just deemphasize them on dlang.org. Ideally we'd set up redirection rules to forward to the new pages, but page anchors will be problematic. One idea to sol
Re: Fixed date to move to ddox for Phobos documentation
On Saturday, 4 June 2016 at 16:10:14 UTC, Seb wrote: Imho it's quite impressive that he still pushes the project and as Adam correctly said - we need to make a decision and have a clear deadline like 2.072 will be the last documentation build with ddoc, once it's released we will remove the ddoc Phobos build and make ddox (/library) the standard (with redirect, of course). This gives us also two to three months to test it properly again (it has been tested now for 2.5 years!!) and resolve issues if occurring. One wholesome switch is probably too harsh. We might discover new issues only after the switch. Instead it could be done gradually, in this order: 1. /library/ is promoted as the primary Phobos documentation in the site navigation. 2. /phobos/ is removed from search indexing. 3. /phobos/ is removed from site navigation. 4. /phobos/ is removed. We could start with step 1 upon 2.072's release.
Re: Blocking points for further D adoption
On Saturday, 4 June 2016 at 16:36:18 UTC, Artem Tarasov wrote: Embedding is easy indeed, and it is rather fun (also speaking from experience), but it again shifts the focus towards D, while most people would like to call D from R, and that's where things start to go awry, especially once you start to think about all the complicacies of cross-platform/multithreaded/GC-reliant extensions, possibly several loaded at the same time. That requires quite a bit of enthusiasm that I admittedly no longer have. I've also gone in the other direction, basically Rcpp with D. Until there is actually interest though (beyond "show me which button to push so I can use it for my research") it's still not going to happen.
Re: Let's avoid range specializations - make ranges great again!
On Saturday, 4 June 2016 at 17:35:23 UTC, Seb wrote: ldc does the job (nearly) correctly. Scratch the "nearly" – at least you can't infer that from the performance results alone, the differences are well within the run-to-run noise. dmd -release -O -boundscheck=off test_looping.d && ./test_looping You seem to be missing `-inline` here, although it doesn't seem to influence the results (DMD 2.071.0/OS X x86_64). 2) Should it matter whether I use r.save or r[i..$]? Answer: it does for dmd and ldc. […] Open question: how can we make DMD/LDC smarter, so that they inline range primitives […]? Is this really the conclusion to draw from this benchmark? First of all, unless I'm mistaken, the three implementations are not equivalent – f_for seems to ignore the first element by doing an extra `popFront()`. Fixing that, the implementations seem to be essentially equivalent in terms of execution time (LDC master, i7-4980HQ@2.8GHz): ``` 0 17 secs, 840 ms, 770 μs, and 9 hnsecs 1 16 secs, 680 ms, 80 μs, and 6 hnsecs 2 17 secs, 635 ms, 548 μs, and 1 hnsec ``` Even though this is using a larger number of iterations for illustrative purposes, the difference still is to be in the run-to-run noise. Comparing the x86_64 assembly produced for 1 and 2, everything is inlined correctly in both cases. The "reversed" loop structure does lead to a slightly higher number of instructions in the loop, though, which might or might not be measurable: --- __D9test_save17__T9f_foreachTAmZ9f_foreachFNaNbNiNfAmmZAm: xor eax, eax testrsi, rsi je LBB11_4 mov rcx, rdx .align 4, 0x90 LBB11_2: cmp qword ptr [rcx], rdi je LBB11_5 inc rax add rcx, 8 cmp rsi, rax ja LBB11_2 LBB11_4: mov rax, rsi ret LBB11_5: sub rsi, rax mov rax, rsi mov rdx, rcx ret __D9test_save13__T5f_forTAmZ5f_forFNaNbNiNfAmmZAm: testrsi, rsi je LBB13_4 mov rax, rsi dec rax mov rcx, rdx add rcx, 8 .align 4, 0x90 LBB13_2: cmp qword ptr [rcx - 8], rdi je LBB13_4 mov rdx, rcx mov rsi, rax dec rax add rcx, 8 cmp rax, -1 jne LBB13_2 LBB13_4: mov rax, rsi ret --- — David
Re: Fixed date to move to ddox for Phobos documentation
On 6/4/16 2:23 PM, Sönke Ludwig wrote: Am 04.06.2016 um 19:01 schrieb Andrei Alexandrescu: I recall there were a few issues with ddox rendering up until relatively recently, have all been fixed? I think they have. Vladimir has reported a bunch of them over time and all of those have been fixed. Great, thanks. * http://dlang.org/phobos/ looks nicer than http://dlang.org/library/. The intro text is not very important, but the grouping is nice. The table vertical split ratio is also nicer. This is currently just the generic module overview. We could simply replace it by a handcrafted version. Shall I copy the one from /phobos/? ...or I could also just read the corresponding .ddoc file and render that. Any way you prefer. I'd be fine with copying the manually-crafted table from /phobos/, and eliminate or modify the short text preceding it. ** the vertical spacing is generally too fluffy (do we really need 25% of the page occupied by Authors and License for _every_ function?) I intended to tackle that multiple times. Definitely agreed that it should be much smaller. On the plus side, it's placed after all of the real meat of the page (except for the comments), so it doesn't really do any harm. Cool, thanks. (I think the comments will play a nice role in the future.) ** Do we need the actual "Prototypes" heading or is it implicit? That's definitely debatable. The first special purpose sections could be layed out in a few different ways: Oh, I meant the actual word. "Prototypes". It takes a lot of vertical space and doesn't add information, because obviously what follows are the prototypes. ** The booktable style is nice for tables that span the entire page width; for parameters the boxes at http://dlang.org/phobos/std_algorithm_comparison.html#among seem nicer to me. While I don't particularly like those either (but I'll leave the design discussion to other people) - I can of course add that right away. I just used the default table style there and simply didn't notice the style change in /phobos/. Makes sense. Any reasonable design should work fine there. ** Do we need the actual heading "Parameters", or it suffices to put "Parameter" instead of "Name" in the parameter box heading? Depending on the general page layout we should definitely be able to drop the heading. Awesome. ** Examples are nice but again vertical spacing overall and between consecutive examples is too large. Partially agreed, the spacing could indeed be reduced a bit, however, one of the big advantages of the single-page layout is that we can use more vertical space to give the page more visual structure and thus make it easier to scan for a particular section. Reasonable. ** Generally the sentiment of the formatting is "too much foreplay". Get to the point already, don't have people scroll through fluff. Relegate Authors and License to a small footnote below everything - even after disqus. Authors+License sure, but the rest is content, right? Or what else would you consider foreplay? I'm thinking just rendering the information in an obvious and immediate manner that doesn't require scrolling or eyes to jump across the page. Consider e.g. https://doc.rust-lang.org/std/boxed/trait.FnBox.html. You open the page and boom, you know it's a trait, you see the signature (without the need to read "Hey hey hey...! here comes the signature!" etc), you see a small yet conspicuous colored box with important information about instability, you see the synopsys, and the example is right after. No need to scroll! It's all clean, immediate, and matter-of-fact, no pomp and circumstance. Imagine that rendered with all components delineated by own headings "Warning:Unstable (fnbox #28796): Newly introduced" etc. * The hotlinks to parameters (e.g. "value", "values") seem unnecessary unless the synopsis is very long which shouldn't be the case. It's also inconsistent - pred is not hotlinked. You are probably right. Linking to template arguments is currently not supported, so dropping parameter linking in general is definitely the more comfortable change ;-) Noice. * How are we going to deal with links (possibly with hashes) pointing to the existing artifacts? Again the simplest way is to keep the old docs but just deemphasize them on dlang.org. Ideally we'd set up redirection rules to forward to the new pages, but page anchors will be problematic. One idea to solve that would be, instead of defining server-side redirection rules, to put some JavaScript code inside of the original /phobos/ docs that detects the active page anchor and redirects to the proper /library/ sub page. Seems we can easily defer that if we continue to produce both forms. * I'd like a "Show entire module documentation as single page" link for e.g. http://dlang.org/library/std/algorithm/comparison.html. So, conceptually a link to /phobos/? What's the motivation for that? Can we solve that by adding more informat
Re: Fixed date to move to ddox for Phobos documentation
On 6/4/16 2:33 PM, Vladimir Panteleev wrote: On Saturday, 4 June 2016 at 16:10:14 UTC, Seb wrote: Imho it's quite impressive that he still pushes the project and as Adam correctly said - we need to make a decision and have a clear deadline like 2.072 will be the last documentation build with ddoc, once it's released we will remove the ddoc Phobos build and make ddox (/library) the standard (with redirect, of course). This gives us also two to three months to test it properly again (it has been tested now for 2.5 years!!) and resolve issues if occurring. One wholesome switch is probably too harsh. We might discover new issues only after the switch. Instead it could be done gradually, in this order: 1. /library/ is promoted as the primary Phobos documentation in the site navigation. 2. /phobos/ is removed from search indexing. 3. /phobos/ is removed from site navigation. 4. /phobos/ is removed. We could start with step 1 upon 2.072's release. Sounds good to me, thanks. Delegation/lieutenantship/empowering for the win. I think we should also secure Martin's buy-in to make sure. -- Andrei
Re: Let's avoid range specializations - make ranges great again!
On Saturday, 4 June 2016 at 17:35:23 UTC, Seb wrote: In Phobos (especially in std.algorithm) a lot of specialization between a RandomAccessRange and an InputRange are used even though only an InputRange is required. The reason for this is solely performance - imho we should start to tackle this and remove such specializations. Let's create better code! The reason for this is solely performance - we should remove such specializations. Performance will always be a driving factor. But then again let's not forget an array is not a range, and isn't treated as one unless the primitives are present (which by default they are not). Arrays in their basic forms are also very common, easy, fairly cheap, etc. Same as slices. I see no reason to drop array based specializations, unless they fail to produce good code.
Generalized Ranges
In a mathematical space we can define operators that allow us to
move around in it. Take your typical Euclidean N space. We can
define a set of orthogonal motions for each dimension. These can
be represented by vectors(your typical orthogonal matrix),
derivatives(infinitesimal differential increments), etc.
We can generalize this to many other spaces of dimension N by
providing a set of N movement operators M_j.
Each M_j represents an incremental movement in the jth dimension.
In D, we can represent these as lambdas. The user provides how to
move to the next element on the space. Iterators are a simple
concept of this as they blindly step to the next element. In
general though, we can have very complex spaces and move through
those spaces in very complex ways depending on our Lambdas.
For example, suppose we have a Tree structure. In this case we
have two orthogonal operators "sibling" and "ancestor". One moves
us "up and down" levels of the tree and the other moves us along
the "leafs". We can even visualize this by embedding the tree in
to R^2 and correlating sibling movement with vertical movement in
R^2 and ancestor movement with horizontal movement. Traversing
the Tree can then be mapped out in R^2 and visualized.
Most common computer programming data structures have simple
movement operators:
Single dimension spaces:
List: Incremental: (list, k) => { return list[k]; }
Stack: Incremental: (stack) => { return stack.pop(); }
Queue: etc.
The movement operators can be written quite simply because they
are generally internally defined and the language supports it.
The point here is that most of the basis structures are
"linear"... that is, use a single movement operator to define how
to "move" from one position in the space to another. (e.g.,
list[k] move k steps from the origin and returns the "point" at k)
The above cases generally are all the same but only tailor the
movement operator to specific performance or behavioral
characteristics.
Two dimension spaces:
Tree: Incremental, Incremental: (Tree) => { return
Tree.Child; }, (Tree) => { return Tree.Sibling; }
The Single dimension product cases(List^2 = List[List[]],
etc).
etc.
The point I'm trying to make is that when we deal with
structures, the motions specify the structure. Most of the time
we deal with simple motions(linear/incremental).
Can D deal with the general case?
i.e., Can I specify a series of movement operators/lambdas that
describe how I want to move through the space(maybe very complex
where the operators are more like tensors(position dependent))
and then leverage D ranges to access the space?
A simple example could be a 2D List, where instead of
incremental, I have:
(x,y) => { if (x % 2 == 0) return list[0, y]; return list[x, y %
2]; }
Here the movement operator is cannot be written orthogonality.
(e.g., we can't separate it as movement in the "x" direction then
in the "y")
An orthogonal case might be
(x) => { l = list[x, Y]; X = x; return l; } (y) => { list[X, y];
y = Y; return l; }
Then call (x) then (y), or vice versa gives us our element at
(x,y). (the intermediate results are projections which are only
"half-solutions")
I noticed that when I was recursing a tree structure:
var Tree = new Dictionary();
Where each object may be another Dictionary(), it
would be much easier to be able to specify the motions in this
space in which case I could simply "iterate" over it rather than
use recursion directly. While we can abstract away the explicit
recursion for simple cases, it becomes much harder when there are
many "paths" to take at once.
In terms of iterators though, it is simply specifying the
movement operators then the action to take at the point. If a
nice generic set of tools existed(maybe D's ranges are suitable
as a foundation?) then I could do something like
Tree.Traverse(
(x, T) => { foreach(var child in T) yield child; },
(y, T) => { if (y is Dictionary) yield y; else
Eval(y); },
(e, T) => { ... do something with e, a leaf ... })
or equivalently
void recurse()
{
foreach(var child in T)
{
if (child is Dictionary)
recurse();
else
Eval(child);
}
}
If you were paying attention you could essentially see the two
orthogonal movements(the foreach is a motion in one direction and
the recurse is the other). What is nice about the first example
is that it separates the traversing of the space from the "work"
done on the "points" in the space. For complex cases this is a
benefit, although there may be some cross referencing between
evaluation and movement. E.g., move left if point has some
property, else move right.
Any Ideas?
PS. The above code is more like pseudo-code. I've been using .NET
a lot lately and it shows! ;) Also, The idea is half baked... it
may need some work.
Re: Generalized Ranges
On Sunday, 5 June 2016 at 00:28:36 UTC, Pie? wrote:
The point I'm trying to make is that when we deal with
structures, the motions specify the structure. Most of the time
we deal with simple motions(linear/incremental).
Can D deal with the general case?
D can be made to do so, but I don't think ranges can, as they
assume linear traversal.
i.e., Can I specify a series of movement operators/lambdas that
describe how I want to move through the space(maybe very
complex where the operators are more like tensors(position
dependent)) and then leverage D ranges to access the space?
For structures where a consistent linear traversal can be
specified, perhaps you can map it to D ranges, though that may
not always make sense for that structure.
A simple example could be a 2D List, where instead of
incremental, I have:
(x,y) => { if (x % 2 == 0) return list[0, y]; return list[x, y
% 2]; }
Here the movement operator is cannot be written orthogonality.
(e.g., we can't separate it as movement in the "x" direction
then in the "y")
An orthogonal case might be
(x) => { l = list[x, Y]; X = x; return l; } (y) => { list[X,
y]; y = Y; return l; }
Then call (x) then (y), or vice versa gives us our element at
(x,y). (the intermediate results are projections which are only
"half-solutions")
I noticed that when I was recursing a tree structure:
var Tree = new Dictionary();
Where each object may be another Dictionary(), it
would be much easier to be able to specify the motions in this
space in which case I could simply "iterate" over it rather
than use recursion directly. While we can abstract away the
explicit recursion for simple cases, it becomes much harder
when there are many "paths" to take at once.
In terms of iterators though, it is simply specifying the
movement operators then the action to take at the point. If a
nice generic set of tools existed(maybe D's ranges are suitable
as a foundation?) then I could do something like
Tree.Traverse(
(x, T) => { foreach(var child in T) yield child; },
(y, T) => { if (y is Dictionary) yield y; else
Eval(y); },
(e, T) => { ... do something with e, a leaf ... })
or equivalently
void recurse()
{
foreach(var child in T)
{
if (child is Dictionary)
recurse();
else
Eval(child);
}
}
If you were paying attention you could essentially see the two
orthogonal movements(the foreach is a motion in one direction
and the recurse is the other). What is nice about the first
example is that it separates the traversing of the space from
the "work" done on the "points" in the space. For complex cases
this is a benefit, although there may be some cross referencing
between evaluation and movement. E.g., move left if point has
some property, else move right.
Any Ideas?
PS. The above code is more like pseudo-code. I've been using
.NET a lot lately and it shows! ;) Also, The idea is half
baked... it may need some work.
It is an interesting idea, basically generalizing D's linear
range operators to other non-linear structures. It might be
worthwhile to define such basic traversal functions for common
structures like trees or certain kinds of graphs, so they have a
common interface. I don't think using D ranges underneath will
work most of the time though.
