Re: std.containers - WAT
On Thursday, 29 March 2012 at 21:04:13 UTC, Steven Schveighoffer wrote: Just the fact that we need javascript to generate the links at the top should tell you something. It tells the data *is* there, but it needs fancier processing... We could put fancier stuff in the compiler, but that's kinda a pain and I worry that it won't fit all needs. Of course, a table of contents is so obviously useful that it probably should be in there. But, I think a more general approach is to fix the default ddoc output to give as much info as possible, and then do a standard tool (like rdmd) that expands it more, and is easy for the users to extend with more info.
Re: std.containers - WAT
On Thursday, March 29, 2012 17:04:13 Steven Schveighoffer wrote: > On Thu, 29 Mar 2012 16:53:42 -0400, Adam D. Ruppe > > wrote: > > The ddoc macros can do a LOT better than it looks > > in phobos. > > My concern is not with the macros. The raw data availability and cross > linking is severely lacking. Just the fact that we need javascript to > generate the links at the top should tell you something. Yeah, compared with other documentation generators, such as doxygen or javadoc, ddoc is pretty weak. The macros work fairly well overall, but not enough gets generated from them. At minimum, links for all of the types and functions in a module in a proper hierarchical format is a must, and ddoc doesn't do anything of the sort. ddoc is the base of something solid, and its simplicity is a definite plus in many ways, but it could be much more powerful even without adding all of the complication that doc generators such as doxygen have. - Jonathan M Davis
Re: std.containers - WAT
On Thu, 29 Mar 2012 16:53:42 -0400, Adam D. Ruppe wrote: The ddoc macros can do a LOT better than it looks in phobos. My concern is not with the macros. The raw data availability and cross linking is severely lacking. Just the fact that we need javascript to generate the links at the top should tell you something. -Steve
Re: std.containers - WAT
The ddoc macros can do a LOT better than it looks in phobos.
Re: std.containers - WAT
This is precisely the wrong way to fix this. DDoc needs to be fixed in the compiler. It's one of the lowest hanging fruits I think we have (a great concept (using compiler knowledge of code structure to generate documentaiton) that is poorly implemented). -Steve Would be a good change to move from ugly macros that make you cry to something more suitable like pythons sphinx. The output should be roughly compatible and a pragma(doc, sphinxalike) at the top of the module lets the compiler use the new system.
Re: std.containers - WAT
On Wed, 28 Mar 2012 10:37:30 -0400, Andrei Alexandrescu wrote: On 3/27/12 10:58 PM, Alex Rønne Petersen wrote: I think one more thing that needs some changing is the usability of the documentation, right now you get a dense list at the top, in mostly-alphabetical order (I think it puts caps first, then lower case) and then you get a massive list of classes and functions that are difficult to navigate. I think basically everyone dislikes this. I agree, and I think std.algorithm offers a good alternative without requiring changes to ddoc. This is a good place where the community might greatly help - it's work that's neither too difficult nor too boring, and has huge impact. This is precisely the wrong way to fix this. DDoc needs to be fixed in the compiler. It's one of the lowest hanging fruits I think we have (a great concept (using compiler knowledge of code structure to generate documentaiton) that is poorly implemented). -Steve
Re: std.containers - WAT
On Wednesday, March 28, 2012 08:37:30 Andrei Alexandrescu wrote: > On 3/27/12 10:58 PM, Alex Rønne Petersen wrote: > >> I think one more thing that needs some changing is the usability of > >> the documentation, right now you get a dense list at the top, in > >> mostly-alphabetical order (I think it puts caps first, then lower > >> case) and then you get a massive list of classes and functions that > >> are difficult to navigate. > > > > I think basically everyone dislikes this. > > I agree, and I think std.algorithm offers a good alternative without > requiring changes to ddoc. This is a good place where the community > might greatly help - it's work that's neither too difficult nor too > boring, and has huge impact. What std.algorithm works great and should be emulated in other modules, but without the anchor situation being fixed, it won't completely work for any modules which include enums, structs, or classes. - Jonathan M Davis
Re: std.containers - WAT
On 3/27/12 10:58 PM, Alex Rønne Petersen wrote: I think one more thing that needs some changing is the usability of the documentation, right now you get a dense list at the top, in mostly-alphabetical order (I think it puts caps first, then lower case) and then you get a massive list of classes and functions that are difficult to navigate. I think basically everyone dislikes this. I agree, and I think std.algorithm offers a good alternative without requiring changes to ddoc. This is a good place where the community might greatly help - it's work that's neither too difficult nor too boring, and has huge impact. Andrei
Re: std.containers - WAT
On Tue, Mar 27, 2012 at 10:00 PM, H. S. Teoh wrote: > On Wed, Mar 28, 2012 at 02:46:19PM +1300, James Miller wrote: > > In helping someone in D.learn, I ended up looking through the > > documentation and code for std.containers. > > > > There is a lot wrong with whats going on there right now. > [...] > > [snip] > > Anyway, it would be great if more people can help with the docs. Just > create an account on github and fork D-Programming-Language/phobos and > D-Programming-Language/d-programming-language.org, run `git clone` on > both repositories in the same directory, create a new branch, and edit > away. When you're ready to test, run a `make html` in the phobos > directory. This will generate a copy of the docs on your local machine > under ../web/phobos-prerelease/, which you can point your browser at. > When you're happy with your changes, go to github and initiate a pull > request on phobos from your working branch. > > > See also the "Improve this page" button I added that lets you make changes online. Good for typos and other smaller changes. It won't show up on Phobos' documentation pages until new docs are uploaded but it's available on the rest of the site already. Regards, Brad Anderson
Re: std.containers - WAT
On Tue, Mar 27, 2012 at 11:59 PM, Jonathan M Davis wrote: > On Tuesday, March 27, 2012 22:20:56 H. S. Teoh wrote: > > On Wed, Mar 28, 2012 at 06:58:45AM +0200, Alex Rønne Petersen wrote: > > > On 28-03-2012 06:13, James Miller wrote: > > [...] > > > > > >I think one more thing that needs some changing is the usability of > > > >the documentation, right now you get a dense list at the top, in > > > >mostly-alphabetical order (I think it puts caps first, then lower > > > >case) and then you get a massive list of classes and functions that > > > >are difficult to navigate. > > > > > > I think basically everyone dislikes this. > > > > [...] > > > > The docs for classes and functions really need to be properly > > categorized according to utility, and not just presented in the order in > > the code or some arbitrary ordering (like alphabetical). Alphabetical is > > OK if you already know what you're looking for, but is horrible for > > newbies who have no idea where to even start. (And arguably even if you > > know what you want, alphabetical is only useful if you already know the > > exact name... otherwise you'd still want to look it up by category > > instead of name.) > > Alphabetical isn't all that big a deal if the docs would just maintain the > hierarchy (e.g. putting the functions of a class with a class rather than > lumping everything in the list at the top). But unfortunately, ddoc doesn't > really support that right now. Some modules do have additional stuff at > the top > for organizing their stuff beyond lumping it together (e.g. std.alorithm > and > std.container), but ddoc itself needs some improvements with regards to > anchors and the like for the links at the top to really be organized like > they > should be. > > - Jonathan M Davis > Adam has a pull request addressing this issue < https://github.com/D-Programming-Language/dmd/pull/770>. It basically makes anchors the full hierarchy of the symbol but without the containing module as that would be redundant and break almost all existing links. Regards, Brad Anderson
Re: std.containers - WAT
On Tuesday, March 27, 2012 22:20:56 H. S. Teoh wrote: > On Wed, Mar 28, 2012 at 06:58:45AM +0200, Alex Rønne Petersen wrote: > > On 28-03-2012 06:13, James Miller wrote: > [...] > > > >I think one more thing that needs some changing is the usability of > > >the documentation, right now you get a dense list at the top, in > > >mostly-alphabetical order (I think it puts caps first, then lower > > >case) and then you get a massive list of classes and functions that > > >are difficult to navigate. > > > > I think basically everyone dislikes this. > > [...] > > The docs for classes and functions really need to be properly > categorized according to utility, and not just presented in the order in > the code or some arbitrary ordering (like alphabetical). Alphabetical is > OK if you already know what you're looking for, but is horrible for > newbies who have no idea where to even start. (And arguably even if you > know what you want, alphabetical is only useful if you already know the > exact name... otherwise you'd still want to look it up by category > instead of name.) Alphabetical isn't all that big a deal if the docs would just maintain the hierarchy (e.g. putting the functions of a class with a class rather than lumping everything in the list at the top). But unfortunately, ddoc doesn't really support that right now. Some modules do have additional stuff at the top for organizing their stuff beyond lumping it together (e.g. std.alorithm and std.container), but ddoc itself needs some improvements with regards to anchors and the like for the links at the top to really be organized like they should be. - Jonathan M Davis
Re: std.containers - WAT
On Wed, Mar 28, 2012 at 06:58:45AM +0200, Alex Rønne Petersen wrote: > On 28-03-2012 06:13, James Miller wrote: [...] > >I think one more thing that needs some changing is the usability of > >the documentation, right now you get a dense list at the top, in > >mostly-alphabetical order (I think it puts caps first, then lower > >case) and then you get a massive list of classes and functions that > >are difficult to navigate. > > I think basically everyone dislikes this. [...] The docs for classes and functions really need to be properly categorized according to utility, and not just presented in the order in the code or some arbitrary ordering (like alphabetical). Alphabetical is OK if you already know what you're looking for, but is horrible for newbies who have no idea where to even start. (And arguably even if you know what you want, alphabetical is only useful if you already know the exact name... otherwise you'd still want to look it up by category instead of name.) T -- "How are you doing?" "Doing what?"
Re: std.containers - WAT
On 28-03-2012 06:13, James Miller wrote: On 28 March 2012 17:00, H. S. Teoh wrote: Anyway, it would be great if more people can help with the docs. Just create an account on github and fork D-Programming-Language/phobos and D-Programming-Language/d-programming-language.org, run `git clone` on both repositories in the same directory, create a new branch, and edit away. When you're ready to test, run a `make html` in the phobos directory. This will generate a copy of the docs on your local machine under ../web/phobos-prerelease/, which you can point your browser at. When you're happy with your changes, go to github and initiate a pull request on phobos from your working branch. Lol, I am doing my best, I currently have a pull request for a typo (not exactly a massive change, but every bit helps) and I will probably contribute more when I have more experience with what is there. I would say that I could improve the documentation for std.algoritm, since I'm using it a lot right now, but its actually pretty good, so that's kinda out of the question. Knowing more about ranges would be very useful for people, since while people have the understand that a range is a sequence of values between two values, its not that obvious how that affects the programming side of it. While Andrei's article is good, the audience is clearly not people that just want to know "what is a Range, and how do I use it". I think one more thing that needs some changing is the usability of the documentation, right now you get a dense list at the top, in mostly-alphabetical order (I think it puts caps first, then lower case) and then you get a massive list of classes and functions that are difficult to navigate. I think basically everyone dislikes this. -- James Miller -- - Alex
Re: std.containers - WAT
On Wednesday, March 28, 2012 16:40:56 James Miller wrote: > On 28 March 2012 15:28, Andrei Alexandrescu > > wrote: > >> In fact, why are any of the functions accepting and > >> returning /Ranges/ which are internal types specific to the container? > > > > Ranges are not internal to containers, they are the way containers are > > manipulated. > > But there is a Range struct inside each container, it implements the > Range interface, but as far as I am aware, it is its own type, and > therefore anything that explicitly expects a Range is going to > expecting its own internal type. From what I can tell, the internal > Range struct on each container is just the underlying storage > mechanism for that container, and you can even have more than one > defined internal Range for a container. The issue comes about that > several methods want that type. Now there may be some voodoo going on > that I don't understand, but it seems to me that I can't do some thing > like: > > SList!int s = [1,2,3,4,5,6,7]; > s.linearRemove([4]) // Arrays are ranges > > To remove a certain value, because [4] is not a valid type, despite > the fact that isForwardRange!(int[]) returns true. This is because > although int[] is a range, it is not a Range (as in the inner type > inside SList named Range). This is incredibly confusing to people > reading the documentation, since Range and range could mean anything. > > I guess what I want is the why's for everything, why do things return > certain values? Why do certain methods only accept data that was > pulled from the container in the first place? Simply saying that some > generally returns some value doesn't help, I need to know the whys > behind it. Is is because the common use case is that, is it because it > was the easiest way to do it, is it because you just randomly decided > to because you felt that void was a cop-out? > > I just feel that the documentation could do with some work in this > regard, but I wasn't there when this was being designed, I don't know > the whys behind the decisions. If I point out where I think the > documentation is lacking, simply in a "That is completely useless" > kind of way, could somebody either update the documentation, or > provide me with better explanations. > > I have also attached the screenshot you asked for, sorry for the > quality, I don't have the tools on my machine to deal with images > properly right now. It's like iterators in C++. Each container has its own, and that's the type that you use to iterate over and manipulate the elements of a container. It's just that instead of using iterators, std.container is using ranges. As to why some functions require ranges from the container specifically, it's for the same reason that many functions on STL containers require iterators from the container specifically: they need to operate on those exact elements, not the values of those elements or some other iterator or range which points to the same values. They must operate on those exact elements. std.container still needs some work to be sure (parts of it are going to be redesigned to work with the custom allocators which are currently being worked on), and the documentation probably does need some work, but much of how it works is pretty much the same as how the containers work in C++. - Jonathan M Davis
Re: std.containers - WAT
On 28 March 2012 17:00, H. S. Teoh wrote: > Anyway, it would be great if more people can help with the docs. Just > create an account on github and fork D-Programming-Language/phobos and > D-Programming-Language/d-programming-language.org, run `git clone` on > both repositories in the same directory, create a new branch, and edit > away. When you're ready to test, run a `make html` in the phobos > directory. This will generate a copy of the docs on your local machine > under ../web/phobos-prerelease/, which you can point your browser at. > When you're happy with your changes, go to github and initiate a pull > request on phobos from your working branch. Lol, I am doing my best, I currently have a pull request for a typo (not exactly a massive change, but every bit helps) and I will probably contribute more when I have more experience with what is there. I would say that I could improve the documentation for std.algoritm, since I'm using it a lot right now, but its actually pretty good, so that's kinda out of the question. Knowing more about ranges would be very useful for people, since while people have the understand that a range is a sequence of values between two values, its not that obvious how that affects the programming side of it. While Andrei's article is good, the audience is clearly not people that just want to know "what is a Range, and how do I use it". I think one more thing that needs some changing is the usability of the documentation, right now you get a dense list at the top, in mostly-alphabetical order (I think it puts caps first, then lower case) and then you get a massive list of classes and functions that are difficult to navigate. -- James Miller
Re: std.containers - WAT
On Wed, Mar 28, 2012 at 02:46:19PM +1300, James Miller wrote: > In helping someone in D.learn, I ended up looking through the > documentation and code for std.containers. > > There is a lot wrong with whats going on there right now. [...] IIRC, Andrei is planning to redesign (or do some major cleanup of) std.containers at some point. OTOH, I have to agree that a lot of Phobos documentation can do with some improvement. I've been trying to do just this whenever I get some free time. Currently I have a pull request for improving std.range's docs -- the current docs are very unfriendly to a newbie; you can only make sense of it if you already know what ranges are. Anyway, it would be great if more people can help with the docs. Just create an account on github and fork D-Programming-Language/phobos and D-Programming-Language/d-programming-language.org, run `git clone` on both repositories in the same directory, create a new branch, and edit away. When you're ready to test, run a `make html` in the phobos directory. This will generate a copy of the docs on your local machine under ../web/phobos-prerelease/, which you can point your browser at. When you're happy with your changes, go to github and initiate a pull request on phobos from your working branch. T -- Recently, our IT department hired a bug-fix engineer. He used to work for Volkswagen.
Re: std.containers - WAT
On 28 March 2012 15:28, Andrei Alexandrescu wrote: >> In fact, why are any of the functions accepting and >> returning /Ranges/ which are internal types specific to the container? > > > Ranges are not internal to containers, they are the way containers are > manipulated. But there is a Range struct inside each container, it implements the Range interface, but as far as I am aware, it is its own type, and therefore anything that explicitly expects a Range is going to expecting its own internal type. From what I can tell, the internal Range struct on each container is just the underlying storage mechanism for that container, and you can even have more than one defined internal Range for a container. The issue comes about that several methods want that type. Now there may be some voodoo going on that I don't understand, but it seems to me that I can't do some thing like: SList!int s = [1,2,3,4,5,6,7]; s.linearRemove([4]) // Arrays are ranges To remove a certain value, because [4] is not a valid type, despite the fact that isForwardRange!(int[]) returns true. This is because although int[] is a range, it is not a Range (as in the inner type inside SList named Range). This is incredibly confusing to people reading the documentation, since Range and range could mean anything. I guess what I want is the why's for everything, why do things return certain values? Why do certain methods only accept data that was pulled from the container in the first place? Simply saying that some generally returns some value doesn't help, I need to know the whys behind it. Is is because the common use case is that, is it because it was the easiest way to do it, is it because you just randomly decided to because you felt that void was a cop-out? I just feel that the documentation could do with some work in this regard, but I wasn't there when this was being designed, I don't know the whys behind the decisions. If I point out where I think the documentation is lacking, simply in a "That is completely useless" kind of way, could somebody either update the documentation, or provide me with better explanations. I have also attached the screenshot you asked for, sorry for the quality, I don't have the tools on my machine to deal with images properly right now. -- James Miller <>
Re: std.containers - WAT
On 3/27/12 7:46 PM, James Miller wrote:
For a start, the documentation is pretty unclear, The table for the
container primitives is poorly explained, and the complexity columns
wraps on most screen sizes, making understanding them a nightmare.
I plan to redo that table. A screenshot would help - thanks!
The
explanations for what things are, and what they do is pretty unclear,
and since its not completely standardized over the different
containers, you end up with some truly mindboggling documentation. A
good example is this: "c.removeAny() - Removes some element from c and
returns it.", what some element at random?
Actually removeAny is one of the better ones. It just removes an element
from the container that is most convenient to the container.
Or does it take an argument
that isn't show like what is implied by this: "c.stableRemoveAny(v) -
Same as c.removeAny(v), but is guaranteed to not invalidate any
iterators.", but that doesn't seem right because SList has "T
removeAny() - Picks one value from the front of the container, removes
it from the container, and returns it." which doesn't take a value.
This is endless confusing, and I ended up consulting the unittests in
order to understand what the hell was going on.
That's a mistake in the documentation, stableRemoveAny does not take a
value. I just fixed it in
https://github.com/D-Programming-Language/phobos/commit/9d7168be5fc91da74231ab86d989ac6013280830.
Then there are the strange input and output types/value for the
arguments. For example, why does SList.linearRemove return an empty
Range on the Take!Range overload, and an empty Range on the Range
overload?
It's simple - generally removeRange returns the portion after the
deleted part. The structure of SList dictates that removing a range
means removing through the end. Removing a Take!Range naturally will
return some potentially nonempty range.
In fact, why are any of the functions accepting and
returning /Ranges/ which are internal types specific to the container?
Ranges are not internal to containers, they are the way containers are
manipulated.
Why don't they all just take a type that can be converted to a range?
I don't see the advantage of doing so.
As is stands, you end up with ridiculous requirements like:
//Remove a single element from an SList, given its "index"
size_t index = 5;
SList!int s = [0,1,2,3,4,5,6,7,8,9];
auto r = s[0..index-1];
auto r1 = take(r, 1);
s.linearRemove(r1);
assert(equal(s, [0,1,2,3,5,6,7,8,9]);
Which could be replaced with an overload for linearRemove that looks like this:
/** Removes the element at index, return false if the element
could not be removed **/
bool linearRemove(size_t index){...}
We can add overloads that use index-based operations, but generally you
already have a range and would like to do a positioned delete. If you
find yourself doing a lot of indexed operations in a SList, you chose
the wrong containers.
Almost all of the "primitives" seem to be based around adding or
removing from the front or back of the container, but there's no
random-access removal.
Why the quotes? More containers are likely to perform removal at an end
efficiently than randomly positioned.
And since the behaviour is Container specific,
its impossible to figure out what a function does anyway! For example,
SList's linearRemove(Range) just removes the front elements from the
container, not obvious.
I'm not sure I get this. Might be a bug somewhere. The linearRemove
primitive is supposed to do the same for all containers - remove a range
in linear time.
Is there any reason why things are the way they are? Because all I see
are people getting exited about ranges and operator overloading and
not really thinking about what a useful set of features any given
container should have.
Proposals for improving things are always welcome.
P.S. I'm happy to help out improving the documentation, but I have no
idea what these things are supposed to do in the first place...
Just ask.
Andrei
std.containers - WAT
In helping someone in D.learn, I ended up looking through the
documentation and code for std.containers.
There is a lot wrong with whats going on there right now.
For a start, the documentation is pretty unclear, The table for the
container primitives is poorly explained, and the complexity columns
wraps on most screen sizes, making understanding them a nightmare. The
explanations for what things are, and what they do is pretty unclear,
and since its not completely standardized over the different
containers, you end up with some truly mindboggling documentation. A
good example is this: "c.removeAny()- Removes some element from c and
returns it.", what some element at random? Or does it take an argument
that isn't show like what is implied by this: "c.stableRemoveAny(v) -
Same as c.removeAny(v), but is guaranteed to not invalidate any
iterators.", but that doesn't seem right because SList has "T
removeAny() - Picks one value from the front of the container, removes
it from the container, and returns it." which doesn't take a value.
This is endless confusing, and I ended up consulting the unittests in
order to understand what the hell was going on.
Then there are the strange input and output types/value for the
arguments. For example, why does SList.linearRemove return an empty
Range on the Take!Range overload, and an empty Range on the Range
overload? In fact, why are any of the functions accepting and
returning /Ranges/ which are internal types specific to the container?
Why don't they all just take a type that can be converted to a range?
As is stands, you end up with ridiculous requirements like:
//Remove a single element from an SList, given its "index"
size_t index = 5;
SList!int s = [0,1,2,3,4,5,6,7,8,9];
auto r = s[0..index-1];
auto r1 = take(r, 1);
s.linearRemove(r1);
assert(equal(s, [0,1,2,3,5,6,7,8,9]);
Which could be replaced with an overload for linearRemove that looks like this:
/** Removes the element at index, return false if the element
could not be removed **/
bool linearRemove(size_t index){...}
Almost all of the "primitives" seem to be based around adding or
removing from the front or back of the container, but there's no
random-access removal. And since the behaviour is Container specific,
its impossible to figure out what a function does anyway! For example,
SList's linearRemove(Range) just removes the front elements from the
container, not obvious.
Is there any reason why things are the way they are? Because all I see
are people getting exited about ranges and operator overloading and
not really thinking about what a useful set of features any given
container should have.
--
James Miller
P.S. I'm happy to help out improving the documentation, but I have no
idea what these things are supposed to do in the first place...
