Re: Dscanner - It exists

2012-08-01 Thread Jacob Carlborg

On 2012-08-02 08:26, Jonathan M Davis wrote:


It's really not all that hard to special case for strings, especially when
you're operating primarily on code units. And I think that the lexer should be
flexible enough to be usable with ranges other than strings. We're trying to
make most stuff in Phobos range-based, not string-based or array-based.


Ok. I just don't think it's worth giving up some performance or make the 
design overly complicated just to make a range interface. But if ranges 
doesn't cause these problems I'm happy.


--
/Jacob Carlborg


Re: Dscanner - It exists

2012-08-01 Thread Jonathan M Davis
On Thursday, August 02, 2012 08:18:39 Jacob Carlborg wrote:
> On 2012-08-01 22:20, Jonathan M Davis wrote:
> > If you want really good performance out of a range-based solution
> > operating on ranges of dchar, then you need to special case for the
> > built-in string types all over the place, and if you have to wrap them in
> > other range types (generally because of calling another range-based
> > function), then there's a good chance that you will indeed get a
> > performance hit. D's range-based approach is really nice from the
> > perspective of usability, but you have to work at it a bit if you want it
> > to be efficient when operating on strings. It _can_ be done though.
> 
> Is it really worth it though? Most use cases will just be with regular
> strings?

It's really not all that hard to special case for strings, especially when 
you're operating primarily on code units. And I think that the lexer should be 
flexible enough to be usable with ranges other than strings. We're trying to 
make most stuff in Phobos range-based, not string-based or array-based.

- Jonathan M Davis


Re: Dscanner - It exists

2012-08-01 Thread Jacob Carlborg

On 2012-08-02 00:23, David wrote:

I think the best way here is to define a BufferedRange that takes any
other range and supplies a buffer for it (with the appropriate
primitives) in a native array.

Andrei


Don't you think, this range stuff is overdone? Define some fancy Range
stuff, if an array just works perfectly?

Ranges > Iterators, yes, but I think they are overdone.




I think so. Some parts of the community seem to be obsessed about ranges.

--
/Jacob Carlborg


Re: Dscanner - It exists

2012-08-01 Thread Jacob Carlborg

On 2012-08-01 22:20, Jonathan M Davis wrote:


If you want really good performance out of a range-based solution operating on
ranges of dchar, then you need to special case for the built-in string types
all over the place, and if you have to wrap them in other range types
(generally because of calling another range-based function), then there's a
good chance that you will indeed get a performance hit. D's range-based
approach is really nice from the perspective of usability, but you have to
work at it a bit if you want it to be efficient when operating on strings. It
_can_ be done though.


Is it really worth it though? Most use cases will just be with regular 
strings?


--
/Jacob Carlborg


Re: Dscanner - It exists

2012-08-01 Thread Walter Bright

On 8/1/2012 3:44 PM, Bernard Helyer wrote:

I would be concerned with potential performance ramifications,
though.


As well you should be. A poorly constructed range can have terrible performance.

But one thing to take careful note of: you *can* define a range that is nothing 
more than a pointer. Of course, you must be careful using such, because it won't 
be safe, but if performance overrides everything else, that option is available.


And best of all, one could still supply a safe range to the same algorithm code, 
without changing any of it.




Re: Dscanner - It exists

2012-08-01 Thread Walter Bright

On 8/1/2012 10:35 AM, Walter Bright wrote:

I suggest proposing the D lexer as an addition to Phobos. But if that is done,
its interface would need to accept a range as input, and its output should be a
range of tokens.



See the thread over in digitalmars.D about a proposed std.d.lexer.




Re: Dscanner - It exists

2012-08-01 Thread deadalnix

Le 01/08/2012 23:19, Andrei Alexandrescu a écrit :

On 8/1/12 5:09 PM, deadalnix wrote:

Le 01/08/2012 19:58, Brian Schott a écrit :

On Wednesday, 1 August 2012 at 17:36:16 UTC, Walter Bright wrote:


I suggest proposing the D lexer as an addition to Phobos. But if that
is done, its interface would need to accept a range as input, and its
output should be a range of tokens.


It used to be range-based, but the performance was terrible. The
inability to use slicing on a forward-range of characters and the
gigantic block on KCachegrind labeled "std.utf.decode" were the reasons
that I chose this approach. I wish I had saved the measurements on
this


Maybe a RandomAccessRange could do the trick ?


I think the best way here is to define a BufferedRange that takes any
other range and supplies a buffer for it (with the appropriate
primitives) in a native array.

Andrei



This make sense. The stuff can be a noop on arrays, and that solve 
everything.


Re: Dscanner - It exists

2012-08-01 Thread Piotr Szturmaj

Brian Schott wrote:

On Wednesday, 1 August 2012 at 17:36:16 UTC, Walter Bright wrote:


I suggest proposing the D lexer as an addition to Phobos. But if that
is done, its interface would need to accept a range as input, and its
output should be a range of tokens.


It used to be range-based, but the performance was terrible. The
inability to use slicing on a forward-range of characters and the
gigantic block on KCachegrind labeled "std.utf.decode" were the reasons
that I chose this approach. I wish I had saved the measurements on this


Ranges are usually taken as template parameters, so you can use static 
if to provide different code for arrays and regular ranges.


Re: Dscanner - It exists

2012-08-01 Thread Bernard Helyer
On Wednesday, 1 August 2012 at 22:31:39 UTC, Andrei Alexandrescu 
wrote:

On 8/1/12 6:23 PM, David wrote:

Ranges > Iterators, yes, but I think they are overdone.


I don't.


I think the main problem is that you need that abstraction
for Phobos. Whereas if you're writing stuff for yourself,
you don't bother. Even if it's a library for consumption.

I wonder if there's an abstraction that would make defining
a range around some data trivial. Maybe even just a good
article on "why use ranges over X" where X == array of data,
or iterators for the C++ crowd.

I know in SDC's lexer we actually do have things that could
be turned into input and output ranges fairly trivially.

I would be concerned with potential performance ramifications,
though.

-Bernard.




Re: Dscanner - It exists

2012-08-01 Thread Andrei Alexandrescu

On 8/1/12 6:23 PM, David wrote:

I think the best way here is to define a BufferedRange that takes any
other range and supplies a buffer for it (with the appropriate
primitives) in a native array.

Andrei


Don't you think, this range stuff is overdone? Define some fancy Range
stuff, if an array just works perfectly?

Ranges > Iterators, yes, but I think they are overdone.


I don't.

Andrei


Re: Dscanner - It exists

2012-08-01 Thread David

I use them quite frequently in unittest {} blocks, if only to import
std.stdio to get why my unittests don't work :)


version(unittest) { private import std.stdio; }

^ Place this where you have your other imports and you don't have to 
import it in your unittest{} blocks.





Re: Dscanner - It exists

2012-08-01 Thread David

I think the best way here is to define a BufferedRange that takes any
other range and supplies a buffer for it (with the appropriate
primitives) in a native array.

Andrei


Don't you think, this range stuff is overdone? Define some fancy Range 
stuff, if an array just works perfectly?


Ranges > Iterators, yes, but I think they are overdone.




Re: Dscanner - It exists

2012-08-01 Thread Philippe Sigaud
On Wed, Aug 1, 2012 at 11:35 PM, Marco Leise  wrote:
> Am Wed, 1 Aug 2012 22:39:41 +0200
> schrieb Philippe Sigaud :
>
>> I just tested the JSON output and it works nicely. Finally, a way to
>> get imports!
>
> What does it do if you import from _inside_ a function ?
> Not that this would happen often, but it can. :-]

I use them quite frequently in unittest {} blocks, if only to import
std.stdio to get why my unittests don't work :)


Re: Dscanner - It exists

2012-08-01 Thread Brian Schott

On Wednesday, 1 August 2012 at 21:35:08 UTC, Marco Leise wrote:

Am Wed, 1 Aug 2012 22:39:41 +0200
schrieb Philippe Sigaud :

I just tested the JSON output and it works nicely. Finally, a 
way to

get imports!


What does it do if you import from _inside_ a function ?
Not that this would happen often, but it can. :-]


It ignores the insides of functions, mostly because writing a 
full D parser was not a design goal. I'm mostly concerned with 
autocomplete, ctags, and summarizing. Unfortunately it also 
ignores the insides of static if and version statements as well. 
I've thought about having versions be a command line or 
configuration option, but the only way to handle static if is to 
actually be a compiler.


Re: Wiki page for C bindings / wrappers and reimplementations

2012-08-01 Thread Marco Leise
Am Wed, 01 Aug 2012 21:35:32 +0200
schrieb David :

> You could make it output json, and someone else does the fancy html?

Maybe I'll make it a little more fancy, settle for a practical database table 
structure and then output JSON.

-- 
Marco



Re: Dscanner - It exists

2012-08-01 Thread Marco Leise
Am Wed, 1 Aug 2012 22:39:41 +0200
schrieb Philippe Sigaud :

> I just tested the JSON output and it works nicely. Finally, a way to
> get imports!

What does it do if you import from _inside_ a function ?
Not that this would happen often, but it can. :-]

-- 
Marco



Re: Pull freeze

2012-08-01 Thread Brad Anderson
On Wednesday, 1 August 2012 at 17:04:06 UTC, David Nadlinger 
wrote:
On Wednesday, 1 August 2012 at 11:56:48 UTC, Andrei 
Alexandrescu wrote:
Well this doesn't do a lot in the way of substantiating. I do 
want to be illuminated. I want to get DVCS! And my 
understanding is that we need to branch whenever we plan a new 
release, and cherry-pick bugfixes from the mainline, and such.


Disregard what I said about cherry-picking – while it would 
work, I was thinking too much in terms of SVN here. With Git, 
the better choice usually is to make all the commits which 
should end up in the release directly on the release branch, 
and merge that branch periodically back to master. If all the 
commits from the release branch should also make it into 
master, which is usually the case, you don't end up with two 
logically separate commits in the same repository this way.


David


Yeah, this is exactly how it should be done. cherry-picking 
should only be needed in very special cases, not as part of a 
regular workflow.


Basically like this:

1. Walter posts the usual "Time for a beta" message and does 
`checkout -b release-2.061 master` and `git push origin 
release-2.061` to put it on GitHub.


2. During the beta period as people find regressions and bugs 
that must be fixed before release they make pull requests 
targeting release-2.061 instead of master. People doing regular, 
non-release-blocking changes just target master as usual.


3. Once the final release is made and out the door a tag is made, 
release-2.061 is merged into develop and deleted: git checkout 
master; git merge release-2.061; git branch -d release-2.061; git 
push origin :release-2.061 (this is all safe to do, git prevents 
you from deleting branches with unmerged changes by default).


If master does need bugfixes from the release branch (I think 
this would be rare) you can merge the release branch or 
cherry-pick. Either works fine. This solves the freeze problem 
(to be honest, I don't think the freezing problem is really that 
huge of deal).


I still think a more important change is using feature/issue 
branches instead of committing partial work to master. master 
should always be in a position where it's ready for a release, 
even if some project someone is working on isn't ready yet (and 
this is mostly the case now due to the nature of GitHub).


Re: Dscanner - It exists

2012-08-01 Thread Philippe Sigaud
On Wed, Aug 1, 2012 at 11:03 PM, Brian Schott  wrote:

> It's more likely that I'll remember things if they're enhancement
> requests/bugs on Github.

Right, I say the same to people asking for things in my projects :)
OK, done.


Re: Dscanner - It exists

2012-08-01 Thread Andrei Alexandrescu

On 8/1/12 5:09 PM, deadalnix wrote:

Le 01/08/2012 19:58, Brian Schott a écrit :

On Wednesday, 1 August 2012 at 17:36:16 UTC, Walter Bright wrote:


I suggest proposing the D lexer as an addition to Phobos. But if that
is done, its interface would need to accept a range as input, and its
output should be a range of tokens.


It used to be range-based, but the performance was terrible. The
inability to use slicing on a forward-range of characters and the
gigantic block on KCachegrind labeled "std.utf.decode" were the reasons
that I chose this approach. I wish I had saved the measurements on
this


Maybe a RandomAccessRange could do the trick ?


I think the best way here is to define a BufferedRange that takes any 
other range and supplies a buffer for it (with the appropriate 
primitives) in a native array.


Andrei



Re: Dscanner - It exists

2012-08-01 Thread deadalnix

Le 01/08/2012 19:58, Brian Schott a écrit :

On Wednesday, 1 August 2012 at 17:36:16 UTC, Walter Bright wrote:


I suggest proposing the D lexer as an addition to Phobos. But if that
is done, its interface would need to accept a range as input, and its
output should be a range of tokens.


It used to be range-based, but the performance was terrible. The
inability to use slicing on a forward-range of characters and the
gigantic block on KCachegrind labeled "std.utf.decode" were the reasons
that I chose this approach. I wish I had saved the measurements on this


Maybe a RandomAccessRange could do the trick ?


Re: Dscanner - It exists

2012-08-01 Thread Brian Schott
On Wednesday, 1 August 2012 at 20:39:49 UTC, Philippe Sigaud 
wrote:

I have have two remarks (not critics!)

- there seem to be two "structs" objects in the JSON, unless 
I'm mistaken.
- alias declaration are not parsed, seemingly.  (as in "alias 
int MyInt;")


Also, do you think comments could be included in the JSON?

Nice work, keep going!


It's more likely that I'll remember things if they're enhancement 
requests/bugs on Github.


Structs: I'll look into it
Alias: Not implemented yet.
Comments: It's planned. I want to be able to give doc comments in 
the autocomplete information.


Re: Dscanner - It exists

2012-08-01 Thread Jonathan M Davis
On Wednesday, August 01, 2012 22:34:14 Marco Leise wrote:
> I still hope for some
> super-smart solution, that doesn't need a book of documentation and allows
> some kind of ASCII-equivalent range.

If you want pure ASCII, then just cast to ubyte[] (or const(ubyte)[] or 
immutable(ubyte)[], depending on the constness involved). string functions 
won't work, because they require UTF-8 (or UTF-16 or UTF-32 if they're 
templatized on string type), but other range-based and array functions will 
work just fine.

- Jonathan M Davis


Re: Dscanner - It exists

2012-08-01 Thread Philippe Sigaud
On Wed, Aug 1, 2012 at 7:30 PM, Brian Schott  wrote:
> First: This is not a release announcement.
>
> I want to let people know that Dscanner *exists*.
>
> https://github.com/Hackerpilot/Dscanner/

> What it does:
> * Has a D lexer
(...)
> * Can generate a JSON summary of D code.

I just tested the JSON output and it works nicely. Finally, a way to
get imports!

I have have two remarks (not critics!)

- there seem to be two "structs" objects in the JSON, unless I'm mistaken.
- alias declaration are not parsed, seemingly.  (as in "alias int MyInt;")

Also, do you think comments could be included in the JSON?

Nice work, keep going!


Re: Dscanner - It exists

2012-08-01 Thread Marco Leise
Am Wed, 01 Aug 2012 19:58:46 +0200
schrieb "Brian Schott" :

> On Wednesday, 1 August 2012 at 17:36:16 UTC, Walter Bright wrote:
> >
> > I suggest proposing the D lexer as an addition to Phobos. But 
> > if that is done, its interface would need to accept a range as 
> > input, and its output should be a range of tokens.
> 
> It used to be range-based, but the performance was terrible. The 
> inability to use slicing on a forward-range of characters and the 
> gigantic block on KCachegrind labeled "std.utf.decode" were the 
> reasons that I chose this approach. I wish I had saved the 
> measurements on this

I can understand you. I was reading a dictionary file with 
readText().splitLines(); and wondering why a unicode decoding was performed. 
Unfortunately ranges work on Unicode units and all structured text files are 
structured by ASCII characters. While these file formats probably just old or 
done with some false sense of compatibility in mind, it is also clear to their 
inventors, that parsing them is easier and faster with single-byte characters 
to delimit tokens.
But we have talked about UTF-8 vs. ASCII and foreach vs. ranges before. I still 
hope for some super-smart solution, that doesn't need a book of documentation 
and allows some kind of ASCII-equivalent range. I've heard that foreach over 
UTF-8 with a dchar loop variable, does an implicit decoding of the UTF-8 
string. While this is useful it is also not self-explanatory and needs some 
reading into the topic.

-- 
Marco



Re: Dscanner - It exists

2012-08-01 Thread Jonathan M Davis
On Wednesday, August 01, 2012 19:58:46 Brian Schott wrote:
> On Wednesday, 1 August 2012 at 17:36:16 UTC, Walter Bright wrote:
> > I suggest proposing the D lexer as an addition to Phobos. But
> > if that is done, its interface would need to accept a range as
> > input, and its output should be a range of tokens.
> 
> It used to be range-based, but the performance was terrible. The
> inability to use slicing on a forward-range of characters and the
> gigantic block on KCachegrind labeled "std.utf.decode" were the
> reasons that I chose this approach. I wish I had saved the
> measurements on this

If you want really good performance out of a range-based solution operating on 
ranges of dchar, then you need to special case for the built-in string types 
all over the place, and if you have to wrap them in other range types 
(generally because of calling another range-based function), then there's a 
good chance that you will indeed get a performance hit. D's range-based 
approach is really nice from the perspective of usability, but you have to 
work at it a bit if you want it to be efficient when operating on strings. It 
_can_ be done though.

The D lexer that I'm currently writing special-cases strings pretty much 
_everywhere_ (string mixins really help reduce the cost of that in terms of 
code duplication). The result is that if I do it right, its performance for 
strings should be very close to what dmd can do (it probably won't quite reach 
dmd's performance simply because of some extra stuff it does to make it more 
usable for stuff other than compilers - e.g. syntax highlighters). But you'll 
still likely get a performance hit of you did something like

string source = getSource();
auto result = tokenRange(filter!"true"(source));

instead of

string source = getSource();
auto result = tokenRange(source);

It won't be quite as bad a performance hit with 2.060 thanks to some recent 
optimizations to string's popFront, but you're going to lose out on some 
performance regardless, because nothing can special-case for every possible 
range type, and one of the keys to fast string processing is to minimizing how 
much you decode characters, which generally requires special-casing.

- Jonathan M Davis


Re: Wiki page for C bindings / wrappers and reimplementations

2012-08-01 Thread dnewbie

On Wednesday, 1 August 2012 at 17:18:36 UTC, Marco Leise wrote:

Am Mon, 30 Jul 2012 16:50:57 +0200
schrieb Marco Leise :

I have now written a simple web site the lists C bindings by 
category. It is updated every day.

http://mleise.abcz8.com/d/bindings.php
Currently it only lists Deimos repositories and includes them 
even if they only contain C headers. Oh and it is ugly.


libnotify description is truncated.


Re: Wiki page for C bindings / wrappers and reimplementations

2012-08-01 Thread David

Am 01.08.2012 19:18, schrieb Marco Leise:

Am Mon, 30 Jul 2012 16:50:57 +0200
schrieb Marco Leise :

I have now written a simple web site the lists C bindings by category. It is 
updated every day.
http://mleise.abcz8.com/d/bindings.php
Currently it only lists Deimos repositories and includes them even if they only 
contain C headers. Oh and it is ugly.



You could make it output json, and someone else does the fancy html?


Re: Dscanner - It exists

2012-08-01 Thread Brian Schott

On Wednesday, 1 August 2012 at 17:36:16 UTC, Walter Bright wrote:


I suggest proposing the D lexer as an addition to Phobos. But 
if that is done, its interface would need to accept a range as 
input, and its output should be a range of tokens.


It used to be range-based, but the performance was terrible. The 
inability to use slicing on a forward-range of characters and the 
gigantic block on KCachegrind labeled "std.utf.decode" were the 
reasons that I chose this approach. I wish I had saved the 
measurements on this


Re: Dscanner - It exists

2012-08-01 Thread Walter Bright

On 8/1/2012 10:30 AM, Brian Schott wrote:

First: This is not a release announcement.

I want to let people know that Dscanner *exists*.

https://github.com/Hackerpilot/Dscanner/

It's a utility that I designed to be used by text editors such as VIM,
Textadept, etc., for getting information about D source code.

I've held off on anoncing this in the past because I don't think that it's
really ready for a release, but after seeing several of the threads about lexers
in the D newsgroup I decided I should make some sort of announcement.

What it does:
* Has a D lexer
* Can syntax-highlight D source files as HTML
* Can generate CTAGS files from D code
* VERY BASIC autocomplete <- The reason I don't consider it "done"
* Can generate a JSON summary of D code.
* Line of code counter. Basically just a filter on the range of tokens that
looks for things like semicolons.

It's Boost licensed, so feel free to use (or submit improvements for) the
tokenizer.


I suggest proposing the D lexer as an addition to Phobos. But if that is done, 
its interface would need to accept a range as input, and its output should be a 
range of tokens.




Dscanner - It exists

2012-08-01 Thread Brian Schott

First: This is not a release announcement.

I want to let people know that Dscanner *exists*.

https://github.com/Hackerpilot/Dscanner/

It's a utility that I designed to be used by text editors such as 
VIM, Textadept, etc., for getting information about D source code.


I've held off on anoncing this in the past because I don't think 
that it's really ready for a release, but after seeing several of 
the threads about lexers in the D newsgroup I decided I should 
make some sort of announcement.


What it does:
* Has a D lexer
* Can syntax-highlight D source files as HTML
* Can generate CTAGS files from D code
* VERY BASIC autocomplete <- The reason I don't consider it "done"
* Can generate a JSON summary of D code.
* Line of code counter. Basically just a filter on the range of 
tokens that looks for things like semicolons.


It's Boost licensed, so feel free to use (or submit improvements 
for) the tokenizer.


Re: Pull freeze

2012-08-01 Thread Brad Roberts
On 8/1/2012 12:30 AM, Russel Winder wrote:
> On Tue, 2012-07-31 at 01:10 -0700, Walter Bright wrote:
> […]
>>
>> We're already using Git.
> 
> I will be "robust".
> 
> You may be making use of Git commands but you are still using
> Subversion, you are not using Git.

You keep blaming it on using subversion or subversion mentality.  It has 
nothing to do with subversion in the slightest.
 It has everything to do with the evolution of the development process.  We 
were only using SVN for a short period of
time and the use of it was a direct evolution from the not using any scm at 
all, no branching, etc, development process
that existed prior to that point.

Please, ignore that svn exists, it's irrelevant to D at this point.



Re: Wiki page for C bindings / wrappers and reimplementations

2012-08-01 Thread Marco Leise
Am Mon, 30 Jul 2012 16:50:57 +0200
schrieb Marco Leise :

I have now written a simple web site the lists C bindings by category. It is 
updated every day.
http://mleise.abcz8.com/d/bindings.php
Currently it only lists Deimos repositories and includes them even if they only 
contain C headers. Oh and it is ugly.

-- 
Marco



Re: Pull freeze

2012-08-01 Thread David Nadlinger
On Wednesday, 1 August 2012 at 11:56:48 UTC, Andrei Alexandrescu 
wrote:
Well this doesn't do a lot in the way of substantiating. I do 
want to be illuminated. I want to get DVCS! And my 
understanding is that we need to branch whenever we plan a new 
release, and cherry-pick bugfixes from the mainline, and such.


Disregard what I said about cherry-picking – while it would 
work, I was thinking too much in terms of SVN here. With Git, the 
better choice usually is to make all the commits which should end 
up in the release directly on the release branch, and merge that 
branch periodically back to master. If all the commits from the 
release branch should also make it into master, which is usually 
the case, you don't end up with two logically separate commits in 
the same repository this way.


David


Re: Pull freeze

2012-08-01 Thread Jesse Phillips

On Sunday, 29 July 2012 at 12:00:16 UTC, David Nadlinger wrote:

On Sunday, 29 July 2012 at 11:43:41 UTC, Robert Clipsham wrote:

4. You can pull regression/bug fixes into the release branch


Just to clarify: »Pulling« in this context means 
cherry-picking individual bug fixes critical for the release 
from the main branch. Virtually all changes and pull requests 
still go on the main branch first, and are then »backported« 
onto the release branch.


And by the way, you don't even need a DVCS for that. Many 
SVN-based open source projects use a »branch-before-release« 
workflow as well.


David


That couldn't possibly be clarification. Otherwise step 5 
wouldn't be needed.


I don't know much about using cherry picking, but it doesn't 
really seem like what we'd want.


Walter would need to pull into master then cherry pick it into 
release. Those trying to resolve a regression will be thrown into 
any changes being made against master, which could be a 
refactoring of const...


It seems to me cherry picking works better for a living branch. 
Release will be short lived. If we went with a stable branch... 
then I could see the cherry picking come into play.


Re: Pull freeze

2012-08-01 Thread Andrei Alexandrescu

On 8/1/12 3:44 AM, Russel Winder wrote:

On Tue, 2012-07-31 at 11:38 -0400, Andrei Alexandrescu wrote:
[…]

You can't suggest a revolution - only carry it through. But I'm a bit
confused. We already use git, and the idea is to use it better. What's
the thing with subversion etc? Where's the revolution?


As has been noted many time now I'm afraid, Git is currently being used
as though it were Subversion. Subversion mind set is being applied when
using Git commands.  In changing from Subversion to Git all mindsets as
well as processes need to be changed. The revolution started with the
actual repository move, but sadly it was not carried through by amending
the processes.


We will amend the processes to branch for each release.


[…]

Agreed. But that means we'd need to use branching and tagging better,
not to "revolutionize" things.


Well actually there is an element of using branching and tagging at all.
Branching and tagging in Subversion is cheap in the database and very
expensive for clients. Worse merging still remains a problem for
Subversion hence branching is a tool of last resort. Branching, tagging
and merging are cheap for Git, but there needs to be a move from CVCS
thinking to DVCS thinking on the part of those people with write
permission to the mainline.


Well this doesn't do a lot in the way of substantiating. I do want to be 
illuminated. I want to get DVCS! And my understanding is that we need to 
branch whenever we plan a new release, and cherry-pick bugfixes from the 
mainline, and such. Or (when we have multiple contributors) use one 
branch per feature. When I ask you, you neither confirm nor deny that, 
but instead go for the vague "well you need to change your mindset". I 
hope you see how this is less than useful.



[…]

To be honest I think we've reaped a lot of low-hanging fruit so far.
Improving the process will bring some marginal efficiency improvements,
but adding more good committers and contributors would be much more
impactful. As far as I can tell there's not (there used to be) a hoard
of disgruntled contributors unable to push things forward.


This paragraph appears (apologies if I have it wrong) to highlight part
of the problem. The way you speak of committers and contributors
indicates Subversion hangover.  DVCS is about having reviewers of
changesets, and gatekeepers who make the merges into the mainline. The D
process has much of this already but at the core the approach to the
mainline is CVCS not DVCS mindset.


DVCS is a lot about "D" - many people working on the project. We don't 
have all that many, and it might help if I explained to you what I meant 
by "pull freeze this Sunday". It appears to me that you made your 
comments in neglect of the dynamics of this project.


There are only a few contributors and gatekeepers to Druntime, Phobos, 
tools, and dlang.org - the repos I manage. Pardon me if I use the wrong 
vocabulary, since "contributors" seems to be the wrong thing and coming 
from the wrong mindset. Let's say "people who make pull requests". I'd 
used them all with the mini-ceremony of making a thorough pass through 
each and every pull request each Sunday afternoon.


This has had beneficial effects. People know that their pull requests 
will be looked at and either commented on or merged. There is a sense of 
progress and of thoroughness. At the end of the day, even if we used the 
mother of all processes, I'd still need to put the same amount of time 
into actually looking at the code. This (which is the bulk of the time 
I'm spending on these projects) cannot be optimized through process.


Clearly if we'd used branching for release I could have done the merging 
last Sunday too. Since we don't have that yet (we should), last Sunday I 
left I got a babysitter, had a nice dinner with my wife, and watched 
some 80% of "The Watch". My perception is that there was no major 
disruption for contributors aside from missing feedback from me (which, 
again, is a reviewer time issue rather than a process issue). Otherwise 
they could have continued work on their branches, or create new ones.


This was the meaning of "pull freeze". I meant to tell I won't be 
looking at stuff this Sunday.



If there isn't a new process in place immediately 2.060 is released and
master tagged, this I think this would have to be considered a "red
flag". The corollary is, I guess, to delay releasing 2.060 till you have
a new process as well as the release being ready to ship.


Why do you think that would be a good decision? There's a lot of value
added right now and a lot of pent-up demand for the many bug fixes and
improvements.


If the road to 2.061 starts without the new process mostly in place, the
danger is there will a mainline freeze to put out 2.061.


The new process is to branch when we decide to release 2.061. Please let 
me know whether this is true or false.



Of course none of this stops people preparing evolutions of the mainline
now even during a mainli

Re: Pull freeze

2012-08-01 Thread Regan Heath
On Tue, 31 Jul 2012 16:54:17 +0100, Andrei Alexandrescu  
 wrote:



On 7/31/12 11:46 AM, deadalnix wrote:

We actually have to « reverse » the way thing are done. You can't go to
the other side of a gap in 2 steps. We face a change that cannot be
gradually adopted.


Then I need more education. I thought a good thing to do is use  
branching for releases, and that we can start doing that without much  
difficulty. No?


If not, then someone or several someones need to produce a good document  
to describe the process they're suggesting.  It seems that some people  
here agree on a process, but others have slightly differing ideas.  The  
issue is deciding on the best model for D, and making it clear to everyone.


So, definition is step 1, education is step 2 and implementation is steps  
3+.


I think a wiki page which is editable by those with concerns might be a  
good idea, or even a DIP.  It should describe the ideal workflow and give  
concrete examples of common processes i.e. releases, dealing with  
merges/rebases etc for bug fixes etc.


I've never used GIT, nor do I contribute to D at present or I might have  
had a crack at this, someone else would definitely be better suited.


R

--
Using Opera's revolutionary email client: http://www.opera.com/mail/


Re: Pull freeze

2012-08-01 Thread Jacob Carlborg

On 2012-08-01 10:42, Mirko Pilger wrote:


the newsgroup is already available via gmane.comp.lang.d.dmd.beta


It's read only.

--
/Jacob Carlborg


Re: Pull freeze

2012-08-01 Thread Mirko Pilger

Along those lines, I really think dmd-beta should me moved to the
newsgroups.


the newsgroup is already available via gmane.comp.lang.d.dmd.beta



Re: Pull freeze

2012-08-01 Thread Nick Sabalausky
On Wed, 01 Aug 2012 08:24:57 +0200
Jacob Carlborg  wrote:

> On 2012-08-01 00:55, Nick Sabalausky wrote:
> 
> > +1 ALL
> >
> > Along those lines, I really think dmd-beta should me moved to the
> > newsgroups. Granted, I am biased since I hate mailing lists. But
> > moving it to NG means:
> >
> > - Consistency with the rest of the D traffic.
> > - Easier to find/discover/subscribe.
> > - Easier to follow the branches of discussion: Not everyone's email
> >client does threading, but it's standard on NG readers.
> > - We get forum.dlang.org integration and the associated visibility
> > and google/bing-ability basically for free.
> 
> I completely agree. I think Walter once said it was a mailing list to 
> keep it more hidden.
> 

Maybe that was a good move at the time, but at this point, I think we
need to encourage more people to participate in the betas. And that
means more visibility.



Re: Pull freeze

2012-08-01 Thread Russel Winder
On Tue, 2012-07-31 at 19:46 +0200, Tobias Pankrath wrote:
[…]
> Does that mean that you do
> 
> git checkout featurebranch
> git rebase master
> git merge featurebranch // fast forward?

Rule 0: Never use rebase on a published repository.
Rule 1: Never use rebase on a published repository.
Rule 2: Never use rebase on a published repository.
…

but yes many people do use rebase on a private repository to create
changesets for submission to review. There are many who think this is
anathema as it changes history, but unless the repository has been
published no-one can actually observe that. 

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


signature.asc
Description: This is a digitally signed message part


Re: Pull freeze

2012-08-01 Thread Russel Winder
On Tue, 2012-07-31 at 11:38 -0400, Andrei Alexandrescu wrote:
[…]
> You can't suggest a revolution - only carry it through. But I'm a bit 
> confused. We already use git, and the idea is to use it better. What's 
> the thing with subversion etc? Where's the revolution?

As has been noted many time now I'm afraid, Git is currently being used
as though it were Subversion. Subversion mind set is being applied when
using Git commands.  In changing from Subversion to Git all mindsets as
well as processes need to be changed. The revolution started with the
actual repository move, but sadly it was not carried through by amending
the processes.

[…]
> Agreed. But that means we'd need to use branching and tagging better, 
> not to "revolutionize" things.

Well actually there is an element of using branching and tagging at all.
Branching and tagging in Subversion is cheap in the database and very
expensive for clients. Worse merging still remains a problem for
Subversion hence branching is a tool of last resort. Branching, tagging
and merging are cheap for Git, but there needs to be a move from CVCS
thinking to DVCS thinking on the part of those people with write
permission to the mainline.

[…]
> To be honest I think we've reaped a lot of low-hanging fruit so far. 
> Improving the process will bring some marginal efficiency improvements, 
> but adding more good committers and contributors would be much more 
> impactful. As far as I can tell there's not (there used to be) a hoard 
> of disgruntled contributors unable to push things forward.

This paragraph appears (apologies if I have it wrong) to highlight part
of the problem. The way you speak of committers and contributors
indicates Subversion hangover.  DVCS is about having reviewers of
changesets, and gatekeepers who make the merges into the mainline. The D
process has much of this already but at the core the approach to the
mainline is CVCS not DVCS mindset. 

> > If there isn't a new process in place immediately 2.060 is released and
> > master tagged, this I think this would have to be considered a "red
> > flag". The corollary is, I guess, to delay releasing 2.060 till you have
> > a new process as well as the release being ready to ship.
> 
> Why do you think that would be a good decision? There's a lot of value 
> added right now and a lot of pent-up demand for the many bug fixes and 
> improvements.

If the road to 2.061 starts without the new process mostly in place, the
danger is there will a mainline freeze to put out 2.061.

> > Of course none of this stops people preparing evolutions of the mainline
> > now even during a mainline repository freeze, this is DVCS / Git and so
> > Subversion trunk rules just do not apply!
> 
> Sure. I agree that should (and can easily) change. But I fail to get the 
> overarching theme of your post - I just see agitation, agitation, 
> agitation, but no coherence.

Sorry but if you haven't got the points already, then my points are more
than valid.

Anyway, it is clear you are asking me to shut up on this topic so this
will be my last post on it.

Back to Java :-((

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


signature.asc
Description: This is a digitally signed message part


Re: Pull freeze

2012-08-01 Thread Russel Winder
On Tue, 2012-07-31 at 21:45 +0200, David Nadlinger wrote:
[…]
> No. The Go guys also use a separate Mercurial branch for 
> preparing releases, while development continues on the main 
> branch.

Just to note that Mercurial and Git differ crucially in how to work with
branches. The Go process is a Mercurial one that does not transfer well
to Git.

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


signature.asc
Description: This is a digitally signed message part


Re: Pull freeze

2012-08-01 Thread Russel Winder
On Tue, 2012-07-31 at 01:10 -0700, Walter Bright wrote:
[…]
> 
> We're already using Git.

I will be "robust".

You may be making use of Git commands but you are still using
Subversion, you are not using Git.

> > To be honest there is never a reason to freeze a repository, even with
> > Subversion, and definitely not with Git, Mercurial and Bazaar. With
> > these latter DVCSs, branching and cherry-picking, means that you just
> > branch from master to create the branch for the release. Whether this
> > becomes a full-blown maintenance branch or just a temporary release
> > branch that merges back post release is a fundamental question of
> > process on which there are opinions. Go has a "there will only ever be
> > the default branch" model,
> 
> Which is what we currently have with dmd on Git.

I don't see how you can be doing a freeze-free process if you declare a
freeze. There are fundamental and crucial differences between master
HEAD in Git and trunk in Subversion. This whole thread is about not
using Subversion trunk thinking when using Git master HEAD.

To say more would be to repeat what has already been repeated.

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


signature.asc
Description: This is a digitally signed message part