[no subject]

[Marc Strapetz]

[no subject]

[Gavin McDonald]

Hello to all users, contributors and Committers!

The Travel Assistance Committee (TAC) are pleased to announce that
travel assistance applications for Community over Code EU 2024 are now
open!

We will be supporting Community over Code EU, Bratislava, Slovakia,
June 3th - 5th, 2024.

TAC exists to help those that would like to attend Community over Code
events, but are unable to do so for financial reasons. For more info
on this years applications and qualifying criteria, please visit the
TAC website at < https://tac.apache.org/ >. Applications are already
open on https://tac-apply.apache.org/, so don't delay!

The Apache Travel Assistance Committee will only be accepting
applications from those people that are able to attend the full event.

Important: Applications close on Friday, March 1st, 2024.

Applicants have until the the closing date above to submit their
applications (which should contain as much supporting material as
required to efficiently and accurately process their request), this
will enable TAC to announce successful applications shortly
afterwards.

As usual, TAC expects to deal with a range of applications from a
diverse range of backgrounds; therefore, we encourage (as always)
anyone thinking about sending in an application to do so ASAP.

For those that will need a Visa to enter the Country - we advise you apply
now so that you have enough time in case of interview delays. So do not
wait until you know if you have been accepted or not.

We look forward to greeting many of you in Bratislava, Slovakia in June,
2024!

Kind Regards,

Gavin

(On behalf of the Travel Assistance Committee)

Re: [Bug] Export path is subject to peg revision parsing

[Daniel Sahlberg]

Den mån 19 juni 2023 kl 00:29 skrev Nathan Hartman :

> On Sat, Jun 17, 2023 at 7:33 AM Daniel Sahlberg <
> daniel.l.sahlb...@gmail.com> wrote:
>
>> Den fre 16 juni 2023 kl 09:49 skrev Osipov, Michael (SMD IT IN) via users
>> :
>>
>>> Scratch that. My thread from five years ago is still valid:
>>> https://lists.apache.org/thread/lonftwtj2kmnjf5mlp91jyxz9xlsgv3d
>>>
>>> The issue sill persists. The doc improvement from Daniel Shahaf haven't
>>> been implemented yet.
>>>
>>
>> Moving the discussion to dev@:
>>
>> Do we want to implement the suggested doc improvement? I started testing
>> each of the commands and the @ trick is required on almost all except for
>> svn checkout so it will be a lot of changed text. Does it make sense or
>> will it only be a cause of confusion?
>>
>
>
> I think it's a good idea as it will help to make the documentation more
> clear and complete.
>

First try committed as r1910826 (and ..33). I'm not sure if it makes the
documentation more clear or if it just adds word that doesn't make any
sense.

I was considering changing URL[@REV] to URL[@[REV]] since the revision is
optional. Otherwise we might mention this explicitly. Check the svn book in
peg revisions: https://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html


I added the explanation as a separate text to allow for some re-use but the
>> argument has many different names (PATH, WCPATH, URL ...) so it is repeated
>> a few times, maybe it can be reworded to leave out the argument name?
>>
>
>
> Hmmm, it looks like we're not entirely consistent: In my mind, PATH could
> be a working copy path or a repository path, while WCPATH must be a working
> copy path. But, looking at the help strings for, e.g., 'svn add', it calls
> for a PATH; I would expect to see WCPATH... (Is there a way to schedule a
> path already in the repository for addition?)
>

There is some explaination in the code:
[[[
 * In most of the help text "PATH" is used where a working copy path is
 * required, "URL" where a repository URL is required and "TARGET" when
 * either a path or a url can be used.  Hmm, should this be part of the
 * help text?
]]]
But that doesn't explain WCPATH. I agree that it can sometimes be confusing
and it would probably be good to go through the text again.

Kind regards,
Daniel Sahlberg

Re: [Bug] Export path is subject to peg revision parsing

[Osipov, Michael (SMD IT IN) via dev]

On 2023-06-20 13:45, Daniel Sahlberg wrote:
Den tis 20 juni 2023 kl 13:00 skrev Osipov, Michael (SMD IT IN) 
mailto:michael.osi...@siemens.com>>:



  From a user's perspective this does not make sense at all:
 > osipovmi@deblndw011x:~/var/Projekte/Playground
 > $ echo foo > m...@apache.org 
 > osipovmi@deblndw011x:~/var/Projekte/Playground
 > $ LC_ALL=C svn add m...@apache.org 
 > svn: E29: 'm...@apache.org ': a peg
revision is not allowed here

I would really expect that the peg rev would used where it makes sense
and not on local paths at all.


I don't say you are wrong but in the previous thread, three persons who 
have been involved in the project a long time argued that:

- Yes, it is a pita and sometimes it doesn't even make sense
- There was maybe a reason long time ago that it was thought to be more 
consistent to always parse the PEG revision on all paths (however, as I 
found out there is at least one case where the peg revision is not NOT 
parsed). (I didn't dig through the version history and the mailing list 
archives from where this was added, it is left as an exersice for the 
reader).
- Changing things now would have backwards compatibility implications, 
any user of the command line client (for example for the Maven SVM 
Subversion Provider) would need to be aware of when this changed and 
behave accordingly.


We CAN change the documentation, which is what I started looking at.


Agree, doc update is imperative here. As well as the breaking of the 
behavior if this is fixed. I guess this would qualify for 2.0.0, not 
before as it might break users and automated code.


M

Re: [Bug] Export path is subject to peg revision parsing

[Daniel Sahlberg]

Den tis 20 juni 2023 kl 13:00 skrev Osipov, Michael (SMD IT IN) <
michael.osi...@siemens.com>:

>
>  From a user's perspective this does not make sense at all:
> > osipovmi@deblndw011x:~/var/Projekte/Playground
> > $ echo foo > m...@apache.org
> > osipovmi@deblndw011x:~/var/Projekte/Playground
> > $ LC_ALL=C svn add m...@apache.org
> > svn: E29: 'm...@apache.org': a peg revision is not allowed here
>
> I would really expect that the peg rev would used where it makes sense
> and not on local paths at all.
>

I don't say you are wrong but in the previous thread, three persons who
have been involved in the project a long time argued that:
- Yes, it is a pita and sometimes it doesn't even make sense
- There was maybe a reason long time ago that it was thought to be more
consistent to always parse the PEG revision on all paths (however, as I
found out there is at least one case where the peg revision is not NOT
parsed). (I didn't dig through the version history and the mailing list
archives from where this was added, it is left as an exersice for the
reader).
- Changing things now would have backwards compatibility implications, any
user of the command line client (for example for the Maven SVM Subversion
Provider) would need to be aware of when this changed and behave
accordingly.

We CAN change the documentation, which is what I started looking at.

Kind regards,
Daniel Sahlberg

Re: [Bug] Export path is subject to peg revision parsing

[Osipov, Michael (SMD IT IN) via dev]

On 2023-06-17 16:32, Daniel Sahlberg wrote:
Den fre 16 juni 2023 kl 09:49 skrev Osipov, Michael (SMD IT IN) via 
users mailto:us...@subversion.apache.org>>:


Scratch that. My thread from five years ago is still valid:
https://lists.apache.org/thread/lonftwtj2kmnjf5mlp91jyxz9xlsgv3d


The issue sill persists. The doc improvement from Daniel Shahaf haven't
been implemented yet.


Moving the discussion to dev@:

Do we want to implement the suggested doc improvement? I started testing 
each of the commands and the @ trick is required on almost all except 
for svn checkout so it will be a lot of changed text. Does it make sense 
or will it only be a cause of confusion?


I added the explanation as a separate text to allow for some re-use but 
the argument has many different names (PATH, WCPATH, URL ...) so it is 
repeated a few times, maybe it can be reworded to leave out the argument 
name?


From a user's perspective this does not make sense at all:

osipovmi@deblndw011x:~/var/Projekte/Playground
$ echo foo > m...@apache.org
osipovmi@deblndw011x:~/var/Projekte/Playground
$ LC_ALL=C svn add m...@apache.org
svn: E29: 'm...@apache.org': a peg revision is not allowed here


I would really expect that the peg rev would used where it makes sense 
and not on local paths at all.


Here [1] is a list of commands I had to modify for Maven SCM Subversion 
Provider back then.


Michael

[1] 
https://github.com/apache/maven-scm/commit/66a260a9107b6cec0c4ebfb9a42e9cc808512e90?w=1

Re: [Bug] Export path is subject to peg revision parsing

[Daniel Sahlberg]

Den mån 19 juni 2023 kl 00:29 skrev Nathan Hartman :

> On Sat, Jun 17, 2023 at 7:33 AM Daniel Sahlberg <
> daniel.l.sahlb...@gmail.com> wrote:
>
>> Den fre 16 juni 2023 kl 09:49 skrev Osipov, Michael (SMD IT IN) via users
>> :
>>
>>> Scratch that. My thread from five years ago is still valid:
>>> https://lists.apache.org/thread/lonftwtj2kmnjf5mlp91jyxz9xlsgv3d
>>>
>>> The issue sill persists. The doc improvement from Daniel Shahaf haven't
>>> been implemented yet.
>>>
>>
>> Moving the discussion to dev@:
>>
>> Do we want to implement the suggested doc improvement? I started testing
>> each of the commands and the @ trick is required on almost all except for
>> svn checkout so it will be a lot of changed text. Does it make sense or
>> will it only be a cause of confusion?
>>
>
>
> I think it's a good idea as it will help to make the documentation more
> clear and complete.
>

As long as the addition actually make it more clear in the general case.
(Documenting edge cases is always difficult). Does the description make
sense, also for a non-experienced user? Do we also need to document that
PEGREVSION can be empty (for URLs containing an @ character)? Maybe
URL[@[PEGREVISION]]?


> I added the explanation as a separate text to allow for some re-use but
>> the argument has many different names (PATH, WCPATH, URL ...) so it is
>> repeated a few times, maybe it can be reworded to leave out the argument
>> name?
>>
>
>
> Hmmm, it looks like we're not entirely consistent: In my mind, PATH could
> be a working copy path or a repository path, while WCPATH must be a working
> copy path. But, looking at the help strings for, e.g., 'svn add', it calls
> for a PATH; I would expect to see WCPATH... (Is there a way to schedule a
> path already in the repository for addition?)
>

Agree it seems a bit random at times, but I didn't look at it overall.

Adding a reminder that we should update the SVN book as well, it contains
what looks like a copy of the help command, eg:
https://svnbook.red-bean.com/nightly/en/svn.ref.svn.c.add.html

Kind regards,
Daniel Sahlberg

Re: [Bug] Export path is subject to peg revision parsing

[Nathan Hartman]

On Sat, Jun 17, 2023 at 7:33 AM Daniel Sahlberg 
wrote:

> Den fre 16 juni 2023 kl 09:49 skrev Osipov, Michael (SMD IT IN) via users <
> us...@subversion.apache.org>:
>
>> Scratch that. My thread from five years ago is still valid:
>> https://lists.apache.org/thread/lonftwtj2kmnjf5mlp91jyxz9xlsgv3d
>>
>> The issue sill persists. The doc improvement from Daniel Shahaf haven't
>> been implemented yet.
>>
>
> Moving the discussion to dev@:
>
> Do we want to implement the suggested doc improvement? I started testing
> each of the commands and the @ trick is required on almost all except for
> svn checkout so it will be a lot of changed text. Does it make sense or
> will it only be a cause of confusion?
>


I think it's a good idea as it will help to make the documentation more
clear and complete.


I added the explanation as a separate text to allow for some re-use but the
> argument has many different names (PATH, WCPATH, URL ...) so it is repeated
> a few times, maybe it can be reworded to leave out the argument name?
>


Hmmm, it looks like we're not entirely consistent: In my mind, PATH could
be a working copy path or a repository path, while WCPATH must be a working
copy path. But, looking at the help strings for, e.g., 'svn add', it calls
for a PATH; I would expect to see WCPATH... (Is there a way to schedule a
path already in the repository for addition?)

Cheers
Nathan

Re: [Bug] Export path is subject to peg revision parsing

[Daniel Sahlberg]

Den fre 16 juni 2023 kl 09:49 skrev Osipov, Michael (SMD IT IN) via users <
us...@subversion.apache.org>:

> Scratch that. My thread from five years ago is still valid:
> https://lists.apache.org/thread/lonftwtj2kmnjf5mlp91jyxz9xlsgv3d
>
> The issue sill persists. The doc improvement from Daniel Shahaf haven't
> been implemented yet.
>

Moving the discussion to dev@:

Do we want to implement the suggested doc improvement? I started testing
each of the commands and the @ trick is required on almost all except for
svn checkout so it will be a lot of changed text. Does it make sense or
will it only be a cause of confusion?

I added the explanation as a separate text to allow for some re-use but the
argument has many different names (PATH, WCPATH, URL ...) so it is repeated
a few times, maybe it can be reworded to leave out the argument name?

[[[
Index: ../subversion/subversion/svn/svn.c
===
--- ../subversion/subversion/svn/svn.c  (revision 1910468)
+++ ../subversion/subversion/svn/svn.c  (working copy)
@@ -424,7 +424,7 @@
 {
   { "add", svn_cl__add, {0}, {N_(
  "Put new files and directories under version control.\n"
- "usage: add PATH...\n"
+ "usage: add PATH[@]...\n"
  "\n"), N_(
  "  Schedule unversioned PATHs for addition, so they will become
versioned and\n"
  "  be added to the repository in the next commit. Recurse into
directories by\n"
@@ -444,7 +444,9 @@
  "\n"), N_(
  "  The selection of items to add may be influenced by the 'ignores'
feature.\n"
  "  Properties may be attached to the items as configured by the
'auto-props'\n"
- "  feature.\n"
+ "  feature.\n", N_(
+ "  If PATH contains an @ character, an additional @ must be specified
at the\n"
+ "  end of PATH to avoid interpreting the first @ as a peg revision
indicator.\n"
 )},
 {opt_targets, 'N', opt_depth, 'q', opt_force, opt_no_ignore,
opt_autoprops,
  opt_no_autoprops, opt_parents },
@@ -516,8 +518,8 @@
   { "changelist", svn_cl__changelist, {"cl"}, {N_(
  "Associate (or dissociate) changelist CLNAME with the named\n"
  "files.\n"
- "usage: 1. changelist CLNAME PATH...\n"
- "   2. changelist --remove PATH...\n"
+ "usage: 1. changelist CLNAME PATH[@]...\n"
+ "   2. changelist --remove PATH[@]...\n"
 )},
 { 'q', 'R', opt_depth, opt_remove, opt_targets, opt_changelist} },

@@ -539,6 +541,8 @@
  "  out into a sub-directory of PATH, with the name of the
sub-directory\n"
  "  being the basename of the URL.\n"
  "\n"), N_(
+ "  If PATH contains an @ character, an additional @ must be specified
at the\n"
+ "  end of PATH to avoid interpreting the first @ as a peg revision
indicator.\n", N_(
  "  If --force is used, unversioned obstructing paths in the working\n"
  "  copy destination do not automatically cause the check out to
fail.\n"
  "  If the obstructing path is the same type (file or directory) as
the\n"
@@ -560,10 +564,10 @@
   { "cleanup", svn_cl__cleanup, {0}, {N_(
  "Either recover from an interrupted operation that left the working\n"
  "copy locked, or remove unwanted files.\n"
- "usage: 1. cleanup [WCPATH...]\n"
- "   2. cleanup --remove-unversioned [WCPATH...]\n"
- "  cleanup --remove-ignored [WCPATH...]\n"
- "   3. cleanup --vacuum-pristines [WCPATH...]\n"
+ "usage: 1. cleanup [WCPATH[@]...]\n"
+ "   2. cleanup --remove-unversioned [WCPATH[@]...]\n"
+ "  cleanup --remove-ignored [WCPATH[@]...]\n"
+ "   3. cleanup --vacuum-pristines [WCPATH[@]...]\n"
  "\n"), N_(
  "  1. When none of the options --remove-unversioned,
--remove-ignored, and\n"
  "--vacuum-pristines is specified, remove all write locks (shown
as 'L' by\n"
@@ -583,7 +587,9 @@
  "\n"), N_(
  "  3. If the --vacuum-pristines option is given, remove pristine
copies of\n"
  "files which are stored inside the .svn directory and which are
no longer\n"
- "referenced by any file in the working copy.\n"
+ "referenced by any file in the working copy.\n", N_(
+ "  If WCPATH contains an @ character, an additional @ must be
specified at the\n"
+ "  end of WCPATH to avoid interpreting the first @ as a peg revision
indicator.\n"
 )},
 { opt_remove_unversioned, opt_remove_ignored, opt_vacuum_pristines,
   opt_include_externals, 'q', opt_merge_cmd },
]]]

Kind regards,
Daniel Sahlberg

Re: mailer.py can produce subject header violates RFC 5321/5322 if truncate_subject is not set

[Daniel Shahaf]

Yasuhito FUTATSUKI wrote on Thu, 09 Jan 2020 01:06 +00:00:
> (I'm sorry I cannot spare much time in this month later, so
> it is next month if I work for this issue...)

No worries; the bug has been there since time immemorial.  There's no
urgency to fix it.

Cheers,

Daniel

Re: mailer.py can produce subject header violates RFC 5321/5322 if truncate_subject is not set

[Yasuhito FUTATSUKI]

On 2020/01/08 2:03, Daniel Shahaf wrote:
> Yasuhito FUTATSUKI wrote on Tue, Jan 07, 2020 at 06:52:20 +0900:
>> I found tools/hook-scripts/mailer/mailer.py can produce very long
>> subject header line without folding. It can be easily over 1000
>> characters [1] if some large source tree is imported in a repository
>> and truncate_subject config value is not specified appropriately. 
>> The mailer.py script send it without regard if server can accept over
>> 1000 octets line [2], and don't have way of recovery when received
>> response like "500 line too long" (of course, as this response code
>> doesn't show the reason, it is no wonder).
>>
>> I also found similar suggestion for commit-email.pl in users@ list
>> archive [3], but on mailer.py we can avoid it by setting apropriate 
>> truncate_value, such as 200 (, which is shown as comment in
>> mailer.conf.example). Is it succifient?
>>
> 
> Admins shouldn't need to edit the config file in order to have the tool comply
> with relevant RFC's.  Therefore, we should not generate lines longer than
> 998 octets unless we're specifically aware that the remote SMTP server can
> accept such lines.
> 
> Thus, I wouldn't consider editing the comments in mailer.conf.example to 
> suffice:
> that wouldn't prevent mailer.py from generating overlong lines by default.
> 
>> (1) It is suffient because this is a code example and setting example,
>>and not to use as is.
>> (2) We should change the default value not to violate them.
>> (3) We should change the default value and ignore if larger values is
>> set.
>> (4) We should implement line folding
> 
> I think there are two separate questions here: the physical lines [the raw
> rfc822 form] and the logical lines [after unfolding and MIME decoding].
> 
> The physical lines shouldn't be longer than the 1000 octet limit stipulated by
> the relevant RFC's.  Longer lines should be folded (or else truncated with a
> warning).  I'm surprised that that isn't already the case; we should let the
> 'email' module handle this for us.  (We we already do it this way in
> tools/dist/security/mailer.py (sic).)

I agree that we should entrust with 'email' module to handle this issue,
at least with Python 3. It's better to replace the code using modern
API, not with legacy API. (I think the author of current code didn't
trust 'email' package or didn't like its policy to encode...)

> The logical lines shouldn't be longer than the limit given by TRUNCATE_VALUE 
> in
> the config file.  It's fair game to ask whether the default value of
> TRUNCATE_VALUE should be changed, but this is just a usability issue, whereas
> the length of physical lines is an interoperability issue.

Agreed.

> Is TRUNCATE_VALUE specified in characters or in bytes?  The docstring doesn't
> make this clear.  If it applies to the logical line it should be specified in
> characters, but it sounds like the code interprets it in bytes?

I think current code don't care multi-bytes characters, as I worte
other reply, so it is not clear now.

Length limits in characters is used in context of display areas for
fixed font width in most case, however, it is often ignored existence
of CJK ideographs which width are twice of latin alphabets. So, it
exists the third indicator of the length limit, 
'per character width unit' :)
(The latter half of this paragrah is a just idle complaint, so please
ignore)

On the other hand, limits in bytes represents just size of data,
or quantity of information (although it ignores entropy).

I think it is mainly for display purpose to introduce the limits
for subject.

(I'm sorry I cannot spare much time in this month later, so
it is next month if I work for this issue...)

Cheers,
-- 
Yasuhito FUTATSUKI 

Re: mailer.py cannot handle utf-8 path in Subject correctly (Re: mailer.py can produce subject header violates RFC 5321/5322 if truncate_subject is not set)

[Daniel Shahaf]

Yasuhito FUTATSUKI wrote on Wed, Jan 08, 2020 at 00:26:39 +0900:
> On 2020/01/07 9:41, Yasuhito FUTATSUKI wrote:
> > On 2020/01/07 6:52, Yasuhito FUTATSUKI wrote:
> >> By the way, it seems another issue about truncate_subject that current
> >> implementation of truncate_subject may break utf-8 multi-bytes character
> >> sequence, but I didn't reproduce it(because I always use ascii
> >> characters only for file names...).
> 
> I could reproduce this problem.
⋮
> UnicodeDecodeError: 'utf8' codec can't decode bytes in position 3-4: invalid 
> continuation byte
>  
> > Probably it needs something like this (but it doesn't support conbining
> > characters, and I didn't any test...):
> > [[[
> > Index: tools/hook-scripts/mailer/mailer.py
> > ===
> > --- tools/hook-scripts/mailer/mailer.py (revision 1872398)
> > +++ tools/hook-scripts/mailer/mailer.py (working copy)
> > @@ -159,7 +159,13 @@
> >    truncate_subject = 0
> >  
> >  if truncate_subject and len(subject) > truncate_subject:
> > -  subject = subject[:(truncate_subject - 3)] + "..."
> > +  # To avoid breaking utf-8 multi-bytes character sequence, we should
> > +  # search the top of the sequence if the byte of the truncate point is
> > +  # secound or later part of multi-bytes character sequence. 
> > +  idx = truncate_subject - 3
> > +  while  0x80 <= ord(subject[idx]) <= 0xbf:
> > +idx -= 1
> > +  subject = subject[:idx] + "..."
> >  return subject
> >  
> >def start(self, group, params):
> > ]]]
> 
> After this patch applied, the script above runs without error. 
> 
> However, this produces Subject line below.
> 
> [[[
> Subject: r1 - =?utf-8?b?44CH44CH44CH5LiA?= =?utf-8?b?44CH44CH44CH5LiJ?= 
> =?utf-8?b?44CH44CH44CH5LqM?= =?utf-8?b?44CH44CH44CH5LqU?= 
> =?utf-8?b?44CH44CH44CH5YWt?= =?utf-8?b?44CHLi4u?=^M
> ]]]
> 
> and decoded Results is
> 
> "Subject: r1 - 〇〇〇一〇〇〇三〇〇〇二〇〇〇五〇〇〇六〇..."
> 
> because white space(s) between encoded words are ignored.
> I think this is not what we want.

We shouldn't be handling a UTF-8 string bytewise.  If it needs truncating, then
we should truncate it characterwise or wordwise, not bytewise.

We shouldn't be doing the MIME-encoding ourselves.  We should just provide the
subject line to the 'email' module and let it worry about MIME encoding, line
folding, and everything else.  (This is a preëxisting problem.)

Makes sense?

Cheers,

Daniel

Re: mailer.py can produce subject header violates RFC 5321/5322 if truncate_subject is not set

[Daniel Shahaf]

Yasuhito FUTATSUKI wrote on Tue, Jan 07, 2020 at 06:52:20 +0900:
> I found tools/hook-scripts/mailer/mailer.py can produce very long
> subject header line without folding. It can be easily over 1000
> characters [1] if some large source tree is imported in a repository
> and truncate_subject config value is not specified appropriately. 
> The mailer.py script send it without regard if server can accept over
> 1000 octets line [2], and don't have way of recovery when received
> response like "500 line too long" (of course, as this response code
> doesn't show the reason, it is no wonder).
> 
> I also found similar suggestion for commit-email.pl in users@ list
> archive [3], but on mailer.py we can avoid it by setting apropriate 
> truncate_value, such as 200 (, which is shown as comment in
> mailer.conf.example). Is it succifient?
> 

Admins shouldn't need to edit the config file in order to have the tool comply
with relevant RFC's.  Therefore, we should not generate lines longer than
998 octets unless we're specifically aware that the remote SMTP server can
accept such lines.

Thus, I wouldn't consider editing the comments in mailer.conf.example to 
suffice:
that wouldn't prevent mailer.py from generating overlong lines by default.

> (1) It is suffient because this is a code example and setting example,
>and not to use as is.
> (2) We should change the default value not to violate them.
> (3) We should change the default value and ignore if larger values is
> set.
> (4) We should implement line folding

I think there are two separate questions here: the physical lines [the raw
rfc822 form] and the logical lines [after unfolding and MIME decoding].

The physical lines shouldn't be longer than the 1000 octet limit stipulated by
the relevant RFC's.  Longer lines should be folded (or else truncated with a
warning).  I'm surprised that that isn't already the case; we should let the
'email' module handle this for us.  (We we already do it this way in
tools/dist/security/mailer.py (sic).)

The logical lines shouldn't be longer than the limit given by TRUNCATE_VALUE in
the config file.  It's fair game to ask whether the default value of
TRUNCATE_VALUE should be changed, but this is just a usability issue, whereas
the length of physical lines is an interoperability issue.

Is TRUNCATE_VALUE specified in characters or in bytes?  The docstring doesn't
make this clear.  If it applies to the logical line it should be specified in
characters, but it sounds like the code interprets it in bytes?

Cheers,

Daniel

mailer.py cannot handle utf-8 path in Subject correctly (Re: mailer.py can produce subject header violates RFC 5321/5322 if truncate_subject is not set)

[Yasuhito FUTATSUKI]

On 2020/01/07 9:41, Yasuhito FUTATSUKI wrote:
> On 2020/01/07 6:52, Yasuhito FUTATSUKI wrote:
>> By the way, it seems another issue about truncate_subject that current
>> implementation of truncate_subject may break utf-8 multi-bytes character
>> sequence, but I didn't reproduce it(because I always use ascii
>> characters only for file names...).

I could reproduce this problem.

with shell script:
[[[
#!/bin/sh

# LC_CTYPE should be valid utf-8 locale. Please change if below is not
# appropriate
export LC_CTYPE=en_US.UTF-8

# assuming 'svnadmin', 'sed', 'chmod', 'cp', 'mkdir', 'python2', and 'cat'
# in command search path, Python bindings installed correctly,
# and subversion_wc pointing appropriate checkout path
subversion_wc='/path/to/subversion/trunk/working/copy'

# set up new repo for mailer.py testing
svnadmin create newrepo
cp ${subversion_wc}/tools/hook-scripts/mailer/mailer.py newrepo/hooks
sed -e 's/^#truncate_subject = 200/truncate_subject = 78/' \
  -e 's/^#mail_command.*/mail_command = cat -/' \
  ${subversion_wc}/tools/hook-scripts/mailer/mailer.conf.example \
  > newrepo/hooks/mailer.conf
sed -e 's/^\(mailer\.py.*\)\/path\/to\/\(mailer\.conf\)/env python2 
"\$REPOS"\/hooks\/\1 "\$REPOS"\/hooks\/\2/' \
  newrepo/hooks/post-commit.tmpl > newrepo/hooks/post-commit
chmod +x newrepo/hooks/post-commit

svn checkout file:///`pwd`/newrepo wd && cd wd
svn mkdir '〇〇〇一' '〇〇〇二' '〇〇〇三' '〇〇〇四' '〇〇〇五' '〇〇〇六'
svn commit -m 'test for mailer.py'
]]]

the result is...
[[[
Checked out revision 0.
A 〇〇〇一
A 〇〇〇二
A 〇〇〇三
A 〇〇〇四
A 〇〇〇五
A 〇〇〇六
Adding 〇〇〇一
Adding 〇〇〇三
Adding 〇〇〇二
Adding 〇〇〇五
Adding 〇〇〇六
Adding 〇〇〇四
Committing transaction...
Committed revision 1.

Warning: post-commit hook failed (exit code 1) with output:
Traceback (most recent call last):
  File "/home/futatuki/tmp/svn-test/mailer_test/newrepo/hooks/mailer.py", line 
1534, in 
sys.argv[3:3+expected_args])
  File "/usr/local/lib/python2.7/site-packages/svn/core.py", line 310, in 
run_app
return func(application_pool, *args, **kw)
  File "/home/futatuki/tmp/svn-test/mailer_test/newrepo/hooks/mailer.py", line 
126, in main
return messenger.generate()
  File "/home/futatuki/tmp/svn-test/mailer_test/newrepo/hooks/mailer.py", line 
489, in generate
self.output.start(group, params)
  File "/home/futatuki/tmp/svn-test/mailer_test/newrepo/hooks/mailer.py", line 
394, in start
self.write(self.mail_headers(group, params))
  File "/home/futatuki/tmp/svn-test/mailer_test/newrepo/hooks/mailer.py", line 
251, in mail_headers
subject  = self._rfc2047_encode(self.make_subject(group, params))
  File "/home/futatuki/tmp/svn-test/mailer_test/newrepo/hooks/mailer.py", line 
246, in _rfc2047_encode
return ' '.join(map(_maybe_encode_header, hdr.split()))
  File "/home/futatuki/tmp/svn-test/mailer_test/newrepo/hooks/mailer.py", line 
244, in _maybe_encode_header
return Header(hdr_token, 'utf-8').encode()
  File "/usr/local/lib/python2.7/email/header.py", line 183, in __init__
self.append(s, charset, errors)
  File "/usr/local/lib/python2.7/email/header.py", line 267, in append
ustr = unicode(s, incodec, errors)
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 3-4: invalid 
continuation byte
cat: -f: No such file or directory
cat: inva...@example.com: No such file or directory
cat: inva...@example.com: No such file or directory

]]]
(Lines starts with 'cat:' is expected out put of
 "cat - -f inva...@example.com inva...@example.com")
 
> Probably it needs something like this (but it doesn't support conbining
> characters, and I didn't any test...):
> [[[
> Index: tools/hook-scripts/mailer/mailer.py
> ===
> --- tools/hook-scripts/mailer/mailer.py (revision 1872398)
> +++ tools/hook-scripts/mailer/mailer.py (working copy)
> @@ -159,7 +159,13 @@
>truncate_subject = 0
>  
>  if truncate_subject and len(subject) > truncate_subject:
> -  subject = subject[:(truncate_subject - 3)] + "..."
> +  # To avoid breaking utf-8 multi-bytes character sequence, we should
> +  # search the top of the sequence if the byte of the truncate point is
> +  # secound or later part of multi-bytes character sequence. 
> +  idx = truncate_subject - 3
> +  while  0x80 <= ord(subject[idx]) <= 0xbf:
> +idx -= 1
> +  subject = subject[:idx] + "..."
>  return subject
>  
>def start(self, group, params):
> ]]]

After this patch applied, the script above runs without error. 

However, this produces Subject line below.

[[[
Subject: r1 - =?utf-8?b?44CH44CH44CH5LiA?= =?utf-8?

Re: mailer.py can produce subject header violates RFC 5321/5322 if truncate_subject is not set

[Yasuhito FUTATSUKI]

On 2020/01/07 6:52, Yasuhito FUTATSUKI wrote:
> By the way, it seems another issue about truncate_subject that current
> implementation of truncate_subject may break utf-8 multi-bytes character
> sequence, but I didn't reproduce it(because I always use ascii
> characters only for file names...).

Probably it needs something like this (but it doesn't support conbining
characters, and I didn't any test...):
[[[
Index: tools/hook-scripts/mailer/mailer.py
===
--- tools/hook-scripts/mailer/mailer.py (revision 1872398)
+++ tools/hook-scripts/mailer/mailer.py (working copy)
@@ -159,7 +159,13 @@
   truncate_subject = 0
 
 if truncate_subject and len(subject) > truncate_subject:
-  subject = subject[:(truncate_subject - 3)] + "..."
+  # To avoid breaking utf-8 multi-bytes character sequence, we should
+  # search the top of the sequence if the byte of the truncate point is
+  # secound or later part of multi-bytes character sequence. 
+  idx = truncate_subject - 3
+  while  0x80 <= ord(subject[idx]) <= 0xbf:
+    idx -= 1
+  subject = subject[:idx] + "..."
 return subject
 
   def start(self, group, params):
]]]


Cheers,
-- 
Yasuhito FUTATSUKI  / 

mailer.py can produce subject header violates RFC 5321/5322 if truncate_subject is not set

[Yasuhito FUTATSUKI]

Hi,

I found tools/hook-scripts/mailer/mailer.py can produce very long
subject header line without folding. It can be easily over 1000
characters [1] if some large source tree is imported in a repository
and truncate_subject config value is not specified appropriately. 
The mailer.py script send it without regard if server can accept over
1000 octets line [2], and don't have way of recovery when received
response like "500 line too long" (of course, as this response code
doesn't show the reason, it is no wonder).

I also found similar suggestion for commit-email.pl in users@ list
archive [3], but on mailer.py we can avoid it by setting apropriate 
truncate_value, such as 200 (, which is shown as comment in
mailer.conf.example). Is it succifient?

(1) It is suffient because this is a code example and setting example,
   and not to use as is.
(2) We should change the default value not to violate them.
(3) We should change the default value and ignore if larger values is
set.
(4) We should implement line folding
...
 

By the way, it seems another issue about truncate_subject that current
implementation of truncate_subject may break utf-8 multi-bytes character
sequence, but I didn't reproduce it(because I always use ascii
characters only for file names...).

[1] RFC 5322 Internet Message Format 
2.1.1 Line Length Limits
https://tools.ietf.org/html/rfc5322#section-2.1.1
[2] RFC 5321 Simple Maile Transfer Protocol
4.5.3.1 SizeLimits and Minimums - 4.5.3.1.6 Text Line
https://tools.ietf.org/html/rfc5321#section-4.5.3.1.6
[3] usrs@ thread "commit-email.pl violates rfc2822"
https://svn.haxx.se/users/archive-2011-04/0203.shtml

Cheers,
-- 
Yasuhito FUTATSUKI 

[no subject]

[Leeto.Chen]

Re: Release announcement subject lines

[Daniel Shahaf]

Branko Čibej wrote on Tue, Dec 22, 2015 at 14:46:56 +0100:
> On 21.12.2015 15:45, Daniel Shahaf wrote:
> > Andreas Stieger wrote on Sun, Dec 20, 2015 at 17:27:12 +0100:
> >> Hi,
> >>
> >> Daniel Shahaf wrote:
> >>> Perhaps the subject line of release announcements should indicate
> >>> whether that release includes a security fix?
> >> Desirable as from distribution point of view (if we didn't already know 
> >> through private pre-disclosures) 
> >>
> > Where is the template generating that subject line?  I don't see
> > it in tools/.
> 
> I don't think anything generates the subject line. release.py will
> generate the text of the message, but not the subject. This would best
> be documented in the community guide.

r1721507.

I notice our news items (e.g., [1]) have no mention whatsoever of
security issues; it'd be nice to fix this, too...  Anyone volunteers?

Cheers,

Daniel

[1] https://subversion.apache.org/news#news-20151215-1

Re: Release announcement subject lines

[Branko Čibej]

On 21.12.2015 15:45, Daniel Shahaf wrote:
> Andreas Stieger wrote on Sun, Dec 20, 2015 at 17:27:12 +0100:
>> Hi,
>>
>> Daniel Shahaf wrote:
>>> Perhaps the subject line of release announcements should indicate
>>> whether that release includes a security fix?
>> Desirable as from distribution point of view (if we didn't already know 
>> through private pre-disclosures) 
>>
> Where is the template generating that subject line?  I don't see
> it in tools/.

I don't think anything generates the subject line. release.py will
generate the text of the message, but not the subject. This would best
be documented in the community guide.

-- Brane

Re: Release announcement subject lines

[Daniel Shahaf]

Andreas Stieger wrote on Sun, Dec 20, 2015 at 17:27:12 +0100:
> Hi,
> 
> Daniel Shahaf wrote:
> > Perhaps the subject line of release announcements should indicate
> > whether that release includes a security fix?
> 
> Desirable as from distribution point of view (if we didn't already know 
> through private pre-disclosures) 
> 

Where is the template generating that subject line?  I don't see
it in tools/.

> > For example:
> > Subject: Apache Subversion 1.8.11 released [SECURITY]
> > Subject: Apache Subversion 1.8.12 released
> 
> Good, alternatively, post the security announcements separately.

*nod* That's an option.

Aw: Release announcement subject lines

[Andreas Stieger]

Hi,

Daniel Shahaf wrote:
> Perhaps the subject line of release announcements should indicate
> whether that release includes a security fix?

Desirable as from distribution point of view (if we didn't already know through 
private pre-disclosures) 

> For example:
> Subject: Apache Subversion 1.8.11 released [SECURITY]
> Subject: Apache Subversion 1.8.12 released

Good, alternatively, post the security announcements separately.

Andreas

Release announcement subject lines

[Daniel Shahaf]

Perhaps the subject line of release announcements should indicate
whether that release includes a security fix?

For example:

Subject: Apache Subversion 1.8.6 released [SECURITY]
Subject: Apache Subversion 1.8.7 released
Subject: Apache Subversion 1.8.8 released
Subject: Apache Subversion 1.8.9 released
Subject: Apache Subversion 1.8.10 released [SECURITY]
Subject: Apache Subversion 1.8.11 released [SECURITY]
Subject: Apache Subversion 1.8.12 released

[no subject]

[Hiroaki Nakamura]

ゆうちょのATMは9時からだったので、それまで待ってからお金おろして帰りますね

--

[no subject]

[Jacky wong]

[no subject]

[Jacky wong]

Support

[no subject]

[Markus Schaber]

Hi,

The attached patch fixes the README file for the cmdline tests to
reflect the files which are actually present in the svntest subdirectory.

[[[
Fix for the README file for the cmdline tests

* subversion/tests/cmdline/README
  Fix the section about the contents of the svntest subdirectory
  to actually reflect the current contents of that directory.
]]]


Best regards

Markus Schaber

(This email was sent from a mobile device...)

CODESYS® a trademark of 3S-Smart Software Solutions GmbH

Inspiring Automation Solutions

3S-Smart Software Solutions GmbH
Dipl.-Inf. Markus Schaber | Product Development Core Technology
Memminger Str. 151 | 87439 Kempten | Germany
Tel. +49-831-54031-979 | Fax +49-831-54031-50

E-Mail: m.scha...@codesys.com | Web: codesys.com
CODESYS internet forum: forum.codesys.com

Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner | Trade 
register: Kempten HRB 6186 | Tax ID No.: DE 167014915
Index: subversion/tests/cmdline/README
===
--- subversion/tests/cmdline/README (revision 1492149)
+++ subversion/tests/cmdline/README (working copy)
@@ -245,10 +245,26 @@ Directory Contents
 
   /verify.py:   Verifies output from Subversion.
 
-  /entry.py:Parse an `entries' file (### not used yet)
+  /testcase.py: Control of test case execution - contains
+decorators for expected failures and conditionally
+executed tests.
 
+  /sandbox.py:  Tools for manipulating a test's working area 
+(a sandbox), those are handy for most simple
+actions a test might want to perform on a wc.
 
+  /objects.py:  Objects that keep track of state during a test.
+(not directly used by the test scripts.)
 
+  /mergetrees.py:   Routines that create merge scenarios.
+  
+  /factory.py:  Automatically generate a (near-)complete new 
+cmdline test from a series of shell commands.
+  
+  /error.py:Error codes as constants, for convenience.
+(auto-generated by tools/dev/gen-py-error.py)
+
+
 What the Python Tests are Doing
 ===
 

RE: [Janos Gyerik: Re: [PATCH] minor change to mailer.py for subject formatting]

[Gavin Baumanis]

Ping.
This thread has received no new comments.


 -Original Message-
 From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]
 Sent: Sunday, 27 January 2013 03:07
 To: dev@subversion.apache.org
 Cc: Janos Gyerik
 Subject: Fwd: [Janos Gyerik: Re: [PATCH] minor change to mailer.py for
 subject formatting]
 
 Forwarding back to the list
 (@Janos I'll have a look later when I check my svn-dev mailbox, rather
than my
 personal one)
 
 - Forwarded message from Janos Gyerik janos.gye...@gmail.com -
 
  From: Janos Gyerik janos.gye...@gmail.com
  Subject: Re: [PATCH] minor change to mailer.py for subject formatting
  To: Daniel Shahaf d...@daniel.shahaf.name
  Date: Sat, 26 Jan 2013 21:37:39 +0100
  Message-ID:
 
 caahxzx+epbb6+4fbhxvb7pk8nteox4fc+59hprtkgexmcad...@mail.gm
 ail.com
 
  Hi Daniel,
 
  Thanks a lot for your comments!
 
  Please check the new patch I attached, I hope it's good now.
 
  Thanks,
  Janos
 
  On Sat, Jan 26, 2013 at 9:03 PM, Daniel Shahaf d...@daniel.shahaf.name
 wrote:
   Please attach patches as MIME text/plain (usually naming them *.txt
   does that).  More below.
  
   +if prefix and re.search(r'REPONAME', prefix):
  
   Needlessly complicated, ('REPONAME' in prefix) would do.
  
   Also I think you should use the %() syntax like mailer.conf.example
   does.  Is that possible?
  
   +reponame =
   + os.path.basename(os.path.dirname(os.path.dirname(__file__)))
  
   That's outright wrong.  You can't assume that post-commit is a
   symlink to mailer.py.  (Just read post-commit.tmpl for a
   counterexample.)
  
   +prefix = re.sub(r'REPONAME', reponame, prefix)
  
   Daniel
   (very brief review, and the setup I help maintain uses svnmailer not
   mailer.py, so if the above doesn't make sense sorry and please
   correct me)
 
 
 
  --
  Janos Gyerik
  http://www.janosgyerik.com/
  https://twitter.com/janosgyerik/
 
  Index: tools/hook-scripts/mailer/mailer.py
 
 
 ===
  --- tools/hook-scripts/mailer/mailer.py (revision 1438886)
  +++ tools/hook-scripts/mailer/mailer.py (working copy)
  @@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
 if cmd == 'commit':
   revision = int(cmd_args[0])
   repos = Repository(repos_dir, revision, pool)
  -cfg = Config(config_fname, repos, { 'author' : repos.author })
  +cfg = Config(config_fname, repos, { 'author' : repos.author,
  + 'repodir' : os.path.basename(repos.repos_dir) })
   messenger = Commit(pool, cfg, repos)
 elif cmd == 'propchange' or cmd == 'propchange2':
   revision = int(cmd_args[0])
 
 
 - End forwarded message -

Re: [Janos Gyerik: Re: [PATCH] minor change to mailer.py for subject formatting]

[Janos Gyerik]

The patch has been merged a long time ago. Unfortunately there was a
bug in it, but to my knowledge that has also been fixed and committed
in the trunk.

On Thu, Mar 7, 2013 at 3:58 PM, Gavin Baumanis gav...@thespidernet.com wrote:
 Ping.
 This thread has received no new comments.


 -Original Message-
 From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]
 Sent: Sunday, 27 January 2013 03:07
 To: dev@subversion.apache.org
 Cc: Janos Gyerik
 Subject: Fwd: [Janos Gyerik: Re: [PATCH] minor change to mailer.py for
 subject formatting]

 Forwarding back to the list
 (@Janos I'll have a look later when I check my svn-dev mailbox, rather
 than my
 personal one)

 - Forwarded message from Janos Gyerik janos.gye...@gmail.com -

  From: Janos Gyerik janos.gye...@gmail.com
  Subject: Re: [PATCH] minor change to mailer.py for subject formatting
  To: Daniel Shahaf d...@daniel.shahaf.name
  Date: Sat, 26 Jan 2013 21:37:39 +0100
  Message-ID:
 
 caahxzx+epbb6+4fbhxvb7pk8nteox4fc+59hprtkgexmcad...@mail.gm
 ail.com
 
  Hi Daniel,
 
  Thanks a lot for your comments!
 
  Please check the new patch I attached, I hope it's good now.
 
  Thanks,
  Janos
 
  On Sat, Jan 26, 2013 at 9:03 PM, Daniel Shahaf d...@daniel.shahaf.name
 wrote:
   Please attach patches as MIME text/plain (usually naming them *.txt
   does that).  More below.
  
   +if prefix and re.search(r'REPONAME', prefix):
  
   Needlessly complicated, ('REPONAME' in prefix) would do.
  
   Also I think you should use the %() syntax like mailer.conf.example
   does.  Is that possible?
  
   +reponame =
   + os.path.basename(os.path.dirname(os.path.dirname(__file__)))
  
   That's outright wrong.  You can't assume that post-commit is a
   symlink to mailer.py.  (Just read post-commit.tmpl for a
   counterexample.)
  
   +prefix = re.sub(r'REPONAME', reponame, prefix)
  
   Daniel
   (very brief review, and the setup I help maintain uses svnmailer not
   mailer.py, so if the above doesn't make sense sorry and please
   correct me)
 
 
 
  --
  Janos Gyerik
  http://www.janosgyerik.com/
  https://twitter.com/janosgyerik/

  Index: tools/hook-scripts/mailer/mailer.py
 
 
 ===
  --- tools/hook-scripts/mailer/mailer.py (revision 1438886)
  +++ tools/hook-scripts/mailer/mailer.py (working copy)
  @@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
 if cmd == 'commit':
   revision = int(cmd_args[0])
   repos = Repository(repos_dir, revision, pool)
  -cfg = Config(config_fname, repos, { 'author' : repos.author })
  +cfg = Config(config_fname, repos, { 'author' : repos.author,
  + 'repodir' : os.path.basename(repos.repos_dir) })
   messenger = Commit(pool, cfg, repos)
 elif cmd == 'propchange' or cmd == 'propchange2':
   revision = int(cmd_args[0])


 - End forwarded message -




--
Janos Gyerik
http://www.janosgyerik.com/
https://twitter.com/janosgyerik/

RE: [Janos Gyerik: Re: [PATCH] minor change to mailer.py for subject formatting]

[Gavin Baumanis]

Hi Janos,
Thanks for the reply - 
I have obviously missed the commit message for the patch.

I'll remove it from my watch List.

Gavin.

 -Original Message-
 From: Janos Gyerik [mailto:janos.gye...@gmail.com]
 Sent: Thursday, 7 March 2013 10:56
 To: Gavin Baumanis
 Cc: Daniel Shahaf; dev@subversion.apache.org
 Subject: Re: [Janos Gyerik: Re: [PATCH] minor change to mailer.py for subject
 formatting]
 
 The patch has been merged a long time ago. Unfortunately there was a bug in
 it, but to my knowledge that has also been fixed and committed in the trunk.
 
 On Thu, Mar 7, 2013 at 3:58 PM, Gavin Baumanis
 gav...@thespidernet.com wrote:
  Ping.
  This thread has received no new comments.
 
 
  -Original Message-
  From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]
  Sent: Sunday, 27 January 2013 03:07
  To: dev@subversion.apache.org
  Cc: Janos Gyerik
  Subject: Fwd: [Janos Gyerik: Re: [PATCH] minor change to mailer.py
  for subject formatting]
 
  Forwarding back to the list
  (@Janos I'll have a look later when I check my svn-dev mailbox,
  rather
  than my
  personal one)
 
  - Forwarded message from Janos Gyerik janos.gye...@gmail.com
  -
 
   From: Janos Gyerik janos.gye...@gmail.com
   Subject: Re: [PATCH] minor change to mailer.py for subject
   formatting
   To: Daniel Shahaf d...@daniel.shahaf.name
   Date: Sat, 26 Jan 2013 21:37:39 +0100
   Message-ID:
  
 
 caahxzx+epbb6+4fbhxvb7pk8nteox4fc+59hprtkgexmcad...@mail.gm
  ail.com
  
   Hi Daniel,
  
   Thanks a lot for your comments!
  
   Please check the new patch I attached, I hope it's good now.
  
   Thanks,
   Janos
  
   On Sat, Jan 26, 2013 at 9:03 PM, Daniel Shahaf
   d...@daniel.shahaf.name
  wrote:
Please attach patches as MIME text/plain (usually naming them
*.txt does that).  More below.
   
+if prefix and re.search(r'REPONAME', prefix):
   
Needlessly complicated, ('REPONAME' in prefix) would do.
   
Also I think you should use the %() syntax like
mailer.conf.example does.  Is that possible?
   
+reponame =
+ os.path.basename(os.path.dirname(os.path.dirname(__file__)))
   
That's outright wrong.  You can't assume that post-commit is a
symlink to mailer.py.  (Just read post-commit.tmpl for a
counterexample.)
   
+prefix = re.sub(r'REPONAME', reponame, prefix)
   
Daniel
(very brief review, and the setup I help maintain uses svnmailer
not mailer.py, so if the above doesn't make sense sorry and
please correct me)
  
  
  
   --
   Janos Gyerik
   http://www.janosgyerik.com/
   https://twitter.com/janosgyerik/
 
   Index: tools/hook-scripts/mailer/mailer.py
  
 
 
  ===
   --- tools/hook-scripts/mailer/mailer.py (revision 1438886)
   +++ tools/hook-scripts/mailer/mailer.py (working copy)
   @@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir,
 cmd_a
  if cmd == 'commit':
revision = int(cmd_args[0])
repos = Repository(repos_dir, revision, pool)
   -cfg = Config(config_fname, repos, { 'author' : repos.author })
   +cfg = Config(config_fname, repos, { 'author' : repos.author,
   + 'repodir' : os.path.basename(repos.repos_dir) })
messenger = Commit(pool, cfg, repos)
  elif cmd == 'propchange' or cmd == 'propchange2':
revision = int(cmd_args[0])
 
 
  - End forwarded message -
 
 
 
 
 --
 Janos Gyerik
 http://www.janosgyerik.com/
 https://twitter.com/janosgyerik/

Re: [PATCH] minor change to mailer.py for subject formatting

[C. Michael Pilato]

On 01/27/2013 10:02 AM, Janos Gyerik wrote:
 If it helps, here's a last try:
 - wrapped the 3 long lines to 80 columns
 - PEP8 fixes (only on these 3 lines), such as removed space after {,
 before :, before }
 
 The only PEP8 violation that remain on these lines is continuation
 line under-indented for visual indent
 
 I don't know how to format best, please do as you see fit.

I can't speak for the PEPs, but prefer to see function parameters and
dictionary keys left-aligned:

cfg = Config(config_fname, repos,
 {'author': repos.author,
  'repodir': os.path.basename(repos.repos_dir),
 })

But that's just me.  I think the formatting you used was fine.

My only remaining minor nit is the variable name itself.  First, other
variable names in this script use underscores to separate words:  for_repos,
label_from, etc.  Secondly, it's really only the basename of the directory
that's being substituted in, and I think the variable name should carry that
information.  (I had to read your docs to make sure I wouldn't wind up with
[svn-/path/to/my/repository commit].)  So, I'd suggest using
repos_basename rather than repodir.  Again, it's a minor thing -- and
one that the patch applier can probably make on your behalf -- but if the
name caused me some initial confusion, it'll probably do the same to someone
else, too.

-- 
C. Michael Pilato cmpil...@collab.net
CollabNet  www.collab.net  Enterprise Cloud Development



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] minor change to mailer.py for subject formatting

[Ben Reser]

On Mon, Jan 28, 2013 at 5:59 AM, C. Michael Pilato cmpil...@collab.net wrote:
 My only remaining minor nit is the variable name itself.  First, other
 variable names in this script use underscores to separate words:  for_repos,
 label_from, etc.  Secondly, it's really only the basename of the directory
 that's being substituted in, and I think the variable name should carry that
 information.  (I had to read your docs to make sure I wouldn't wind up with
 [svn-/path/to/my/repository commit].)  So, I'd suggest using
 repos_basename rather than repodir.  Again, it's a minor thing -- and
 one that the patch applier can probably make on your behalf -- but if the
 name caused me some initial confusion, it'll probably do the same to someone
 else, too.

I almost said something about the name too when I reviewed it.  I let
it go since you could easily say repodir means the name of the
directory that the repo is in.  I'd have gone with repos_name
personally.

But at this point I feel like we're painting the bikeshed.

Re: [PATCH] minor change to mailer.py for subject formatting

[Daniel Shahaf]

Ben Reser wrote on Mon, Jan 28, 2013 at 09:25:30 -0800:
 On Mon, Jan 28, 2013 at 5:59 AM, C. Michael Pilato cmpil...@collab.net 
 wrote:
  My only remaining minor nit is the variable name itself.  First, other
  variable names in this script use underscores to separate words:  for_repos,
  label_from, etc.  Secondly, it's really only the basename of the directory
  that's being substituted in, and I think the variable name should carry that
  information.  (I had to read your docs to make sure I wouldn't wind up with
  [svn-/path/to/my/repository commit].)  So, I'd suggest using
  repos_basename rather than repodir.  Again, it's a minor thing -- and
  one that the patch applier can probably make on your behalf -- but if the
  name caused me some initial confusion, it'll probably do the same to someone
  else, too.
 
 I almost said something about the name too when I reviewed it.  I let
 it go since you could easily say repodir means the name of the
 directory that the repo is in.  I'd have gone with repos_name
 personally.
 
 But at this point I feel like we're painting the bikeshed.

I'll commit with cmpilato's tweaks (indentation and varname) as I agree
with them both.

Re: [PATCH] minor change to mailer.py for subject formatting

[Greg Stein]

On Jan 28, 2013 3:59 AM, C. Michael Pilato cmpil...@collab.net wrote:

 On 01/27/2013 10:02 AM, Janos Gyerik wrote:
  If it helps, here's a last try:
  - wrapped the 3 long lines to 80 columns
  - PEP8 fixes (only on these 3 lines), such as removed space after {,
  before :, before }
 
  The only PEP8 violation that remain on these lines is continuation
  line under-indented for visual indent
 
  I don't know how to format best, please do as you see fit.

 I can't speak for the PEPs, but prefer to see function parameters and
 dictionary keys left-aligned:

 cfg = Config(config_fname, repos,
  {'author': repos.author,
   'repodir': os.path.basename(repos.repos_dir),
  })

 But that's just me.  I think the formatting you used was fine.

 My only remaining minor nit is the variable name itself.  First, other
 variable names in this script use underscores to separate words:
 for_repos,
 label_from, etc.  Secondly, it's really only the basename of the directory
 that's being substituted in, and I think the variable name should carry
that
 information.  (I had to read your docs to make sure I wouldn't wind up
with
 [svn-/path/to/my/repository commit].)  So, I'd suggest using
 repos_basename rather than repodir.  Again, it's a minor thing -- and

Seriously? 14 chars instead of 7. To satisfy some naming thought which
wouldn't be abundantly clear from the doc?

repodir seems totally fine.

Re: [PATCH] minor change to mailer.py for subject formatting

[C. Michael Pilato]

On 01/28/2013 12:29 PM, Daniel Shahaf wrote:
 Ben Reser wrote on Mon, Jan 28, 2013 at 09:25:30 -0800:
 On Mon, Jan 28, 2013 at 5:59 AM, C. Michael Pilato cmpil...@collab.net 
 wrote:
 My only remaining minor nit is the variable name itself.  First, other
 variable names in this script use underscores to separate words:  for_repos,
 label_from, etc.  Secondly, it's really only the basename of the directory
 that's being substituted in, and I think the variable name should carry that
 information.  (I had to read your docs to make sure I wouldn't wind up with
 [svn-/path/to/my/repository commit].)  So, I'd suggest using
 repos_basename rather than repodir.  Again, it's a minor thing -- and
 one that the patch applier can probably make on your behalf -- but if the
 name caused me some initial confusion, it'll probably do the same to someone
 else, too.

 I almost said something about the name too when I reviewed it.  I let
 it go since you could easily say repodir means the name of the
 directory that the repo is in.  I'd have gone with repos_name
 personally.

 But at this point I feel like we're painting the bikeshed.
 
 I'll commit with cmpilato's tweaks (indentation and varname) as I agree
 with them both.
 

Cool.  Thanks, Daniel.

And thanks, Janos, for the contribution and patient survival of our
iterative patch review process. :-)

-- 
C. Michael Pilato cmpil...@collab.net
CollabNet  www.collab.net  Enterprise Cloud Development



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] minor change to mailer.py for subject formatting

[Daniel Shahaf]

C. Michael Pilato wrote on Mon, Jan 28, 2013 at 13:21:55 -0500:
 On 01/28/2013 12:29 PM, Daniel Shahaf wrote:
  I'll commit with cmpilato's tweaks (indentation and varname) as I agree
  with them both.
  

r1439592.

@Greg - ACK, but 'repos' _is_ consistent with the naming scheme of just
about everywhere else in Subversion.

 
 Cool.  Thanks, Daniel.
 
 And thanks, Janos, for the contribution and patient survival of our
 iterative patch review process. :-)

+1, thanks Janos.

Re: [PATCH] minor change to mailer.py for subject formatting

[Janos Gyerik]

 And thanks, Janos, for the contribution and patient survival of our
 iterative patch review process. :-)

It would have been a shorter process if I had got it right in the
first place ;-)

And you're totally welcome!

Janos

On Mon, Jan 28, 2013 at 8:23 PM, Daniel Shahaf d...@daniel.shahaf.name wrote:
 C. Michael Pilato wrote on Mon, Jan 28, 2013 at 13:21:55 -0500:
 On 01/28/2013 12:29 PM, Daniel Shahaf wrote:
  I'll commit with cmpilato's tweaks (indentation and varname) as I agree
  with them both.
 

 r1439592.

 @Greg - ACK, but 'repos' _is_ consistent with the naming scheme of just
 about everywhere else in Subversion.


 Cool.  Thanks, Daniel.

 And thanks, Janos, for the contribution and patient survival of our
 iterative patch review process. :-)

 +1, thanks Janos.



-- 
Janos Gyerik
http://www.janosgyerik.com/
https://twitter.com/janosgyerik/

Fwd: [Janos Gyerik: Re: [PATCH] minor change to mailer.py for subject formatting]

[Daniel Shahaf]

Forwarding back to the list
(@Janos I'll have a look later when I check my svn-dev mailbox, rather
than my personal one)

- Forwarded message from Janos Gyerik janos.gye...@gmail.com -

 From: Janos Gyerik janos.gye...@gmail.com
 Subject: Re: [PATCH] minor change to mailer.py for subject formatting
 To: Daniel Shahaf d...@daniel.shahaf.name
 Date: Sat, 26 Jan 2013 21:37:39 +0100
 Message-ID: 
 caahxzx+epbb6+4fbhxvb7pk8nteox4fc+59hprtkgexmcad...@mail.gmail.com
 
 Hi Daniel,
 
 Thanks a lot for your comments!
 
 Please check the new patch I attached, I hope it's good now.
 
 Thanks,
 Janos
 
 On Sat, Jan 26, 2013 at 9:03 PM, Daniel Shahaf d...@daniel.shahaf.name 
 wrote:
  Please attach patches as MIME text/plain (usually naming them *.txt does
  that).  More below.
 
  +if prefix and re.search(r'REPONAME', prefix):
 
  Needlessly complicated, ('REPONAME' in prefix) would do.
 
  Also I think you should use the %() syntax like mailer.conf.example
  does.  Is that possible?
 
  +reponame = 
  os.path.basename(os.path.dirname(os.path.dirname(__file__)))
 
  That's outright wrong.  You can't assume that post-commit is a symlink
  to mailer.py.  (Just read post-commit.tmpl for a counterexample.)
 
  +prefix = re.sub(r'REPONAME', reponame, prefix)
 
  Daniel
  (very brief review, and the setup I help maintain uses svnmailer not
  mailer.py, so if the above doesn't make sense sorry and please correct me)
 
 
 
 -- 
 Janos Gyerik
 http://www.janosgyerik.com/
 https://twitter.com/janosgyerik/

 Index: tools/hook-scripts/mailer/mailer.py
 ===
 --- tools/hook-scripts/mailer/mailer.py   (revision 1438886)
 +++ tools/hook-scripts/mailer/mailer.py   (working copy)
 @@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
if cmd == 'commit':
  revision = int(cmd_args[0])
  repos = Repository(repos_dir, revision, pool)
 -cfg = Config(config_fname, repos, { 'author' : repos.author })
 +cfg = Config(config_fname, repos, { 'author' : repos.author, 'repodir' : 
 os.path.basename(repos.repos_dir) })
  messenger = Commit(pool, cfg, repos)
elif cmd == 'propchange' or cmd == 'propchange2':
  revision = int(cmd_args[0])


- End forwarded message -

Re: [Janos Gyerik: Re: [PATCH] minor change to mailer.py for subject formatting]

[Janos Gyerik]

I already sent a newer patch to the list, with more fixes based on
comments by Michael and Ben. Attaching the new patch here again just
in case.

On Sun, Jan 27, 2013 at 9:06 AM, Daniel Shahaf d...@daniel.shahaf.name wrote:
 Forwarding back to the list
 (@Janos I'll have a look later when I check my svn-dev mailbox, rather
 than my personal one)

 - Forwarded message from Janos Gyerik janos.gye...@gmail.com -

 From: Janos Gyerik janos.gye...@gmail.com
 Subject: Re: [PATCH] minor change to mailer.py for subject formatting
 To: Daniel Shahaf d...@daniel.shahaf.name
 Date: Sat, 26 Jan 2013 21:37:39 +0100
 Message-ID: 
 caahxzx+epbb6+4fbhxvb7pk8nteox4fc+59hprtkgexmcad...@mail.gmail.com

 Hi Daniel,

 Thanks a lot for your comments!

 Please check the new patch I attached, I hope it's good now.

 Thanks,
 Janos

 On Sat, Jan 26, 2013 at 9:03 PM, Daniel Shahaf d...@daniel.shahaf.name 
 wrote:
  Please attach patches as MIME text/plain (usually naming them *.txt does
  that).  More below.
 
  +if prefix and re.search(r'REPONAME', prefix):
 
  Needlessly complicated, ('REPONAME' in prefix) would do.
 
  Also I think you should use the %() syntax like mailer.conf.example
  does.  Is that possible?
 
  +reponame = 
  os.path.basename(os.path.dirname(os.path.dirname(__file__)))
 
  That's outright wrong.  You can't assume that post-commit is a symlink
  to mailer.py.  (Just read post-commit.tmpl for a counterexample.)
 
  +prefix = re.sub(r'REPONAME', reponame, prefix)
 
  Daniel
  (very brief review, and the setup I help maintain uses svnmailer not
  mailer.py, so if the above doesn't make sense sorry and please correct me)



 --
 Janos Gyerik
 http://www.janosgyerik.com/
 https://twitter.com/janosgyerik/

 Index: tools/hook-scripts/mailer/mailer.py
 ===
 --- tools/hook-scripts/mailer/mailer.py   (revision 1438886)
 +++ tools/hook-scripts/mailer/mailer.py   (working copy)
 @@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
if cmd == 'commit':
  revision = int(cmd_args[0])
  repos = Repository(repos_dir, revision, pool)
 -cfg = Config(config_fname, repos, { 'author' : repos.author })
 +cfg = Config(config_fname, repos, { 'author' : repos.author, 'repodir' 
 : os.path.basename(repos.repos_dir) })
  messenger = Commit(pool, cfg, repos)
elif cmd == 'propchange' or cmd == 'propchange2':
  revision = int(cmd_args[0])


 - End forwarded message -



-- 
Janos Gyerik
http://www.janosgyerik.com/
https://twitter.com/janosgyerik/
Index: tools/hook-scripts/mailer/mailer.py
===
--- tools/hook-scripts/mailer/mailer.py (revision 1438886)
+++ tools/hook-scripts/mailer/mailer.py (working copy)
@@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
   if cmd == 'commit':
 revision = int(cmd_args[0])
 repos = Repository(repos_dir, revision, pool)
-cfg = Config(config_fname, repos, { 'author' : repos.author })
+cfg = Config(config_fname, repos, { 'author' : repos.author, 'repodir' : 
os.path.basename(repos.repos_dir) })
 messenger = Commit(pool, cfg, repos)
   elif cmd == 'propchange' or cmd == 'propchange2':
 revision = int(cmd_args[0])
@@ -108,14 +108,14 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
 repos = Repository(repos_dir, revision, pool)
 # Override the repos revision author with the author of the propchange
 repos.author = author
-cfg = Config(config_fname, repos, { 'author' : author })
+cfg = Config(config_fname, repos, { 'author' : author, 'repodir' : 
os.path.basename(repos.repos_dir) })
 messenger = PropChange(pool, cfg, repos, author, propname, action)
   elif cmd == 'lock' or cmd == 'unlock':
 author = cmd_args[0]
 repos = Repository(repos_dir, 0, pool) ### any old revision will do
 # Override the repos revision author with the author of the lock/unlock
 repos.author = author
-cfg = Config(config_fname, repos, { 'author' : author })
+cfg = Config(config_fname, repos, { 'author' : author, 'repodir' : 
os.path.basename(repos.repos_dir) })
 messenger = Lock(pool, cfg, repos, author, cmd == 'lock')
   else:
 raise UnknownSubcommand(cmd)
Index: tools/hook-scripts/mailer/mailer.conf.example
===
--- tools/hook-scripts/mailer/mailer.conf.example   (revision 1438886)
+++ tools/hook-scripts/mailer/mailer.conf.example   (working copy)
@@ -146,7 +146,16 @@
 #
 #   from_addr = %(author)s...@example.com
 #
+# The substitution variable repodir is provided, and is set to
+# the directory name of the repository. This can be useful to set
+# a custom subject that can be re-used in multiple repositories:
 #
+#   commit_subject_prefix = [svn-%(repodir)s]
+#
+# For example if the repository is at /path/to/repo/project-x then
+# the subject

Re: [PATCH] minor change to mailer.py for subject formatting

[Daniel Shahaf]

 Index: tools/hook-scripts/mailer/mailer.py
 ===
 --- tools/hook-scripts/mailer/mailer.py   (revision 1438886)
 +++ tools/hook-scripts/mailer/mailer.py   (working copy)
 @@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
if cmd == 'commit':
  revision = int(cmd_args[0])
  repos = Repository(repos_dir, revision, pool)
 -cfg = Config(config_fname, repos, { 'author' : repos.author })
 +cfg = Config(config_fname, repos, { 'author' : repos.author, 'repodir' : 
 os.path.basename(repos.repos_dir) })

My only comment is that these lines should be wrapped to 80 columns.

I'll hold off on committing, though, to let others have a look too.

Thanks for the patch,

Daniel

Re: [PATCH] minor change to mailer.py for subject formatting

[Janos Gyerik]

If it helps, here's a last try:
- wrapped the 3 long lines to 80 columns
- PEP8 fixes (only on these 3 lines), such as removed space after {,
before :, before }

The only PEP8 violation that remain on these lines is continuation
line under-indented for visual indent

I don't know how to format best, please do as you see fit.

Cheers,
Janos

On Sun, Jan 27, 2013 at 3:43 PM, Daniel Shahaf d...@daniel.shahaf.name wrote:
 Index: tools/hook-scripts/mailer/mailer.py
 ===
 --- tools/hook-scripts/mailer/mailer.py   (revision 1438886)
 +++ tools/hook-scripts/mailer/mailer.py   (working copy)
 @@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
if cmd == 'commit':
  revision = int(cmd_args[0])
  repos = Repository(repos_dir, revision, pool)
 -cfg = Config(config_fname, repos, { 'author' : repos.author })
 +cfg = Config(config_fname, repos, { 'author' : repos.author, 'repodir' 
 : os.path.basename(repos.repos_dir) })

 My only comment is that these lines should be wrapped to 80 columns.

 I'll hold off on committing, though, to let others have a look too.

 Thanks for the patch,

 Daniel



-- 
Janos Gyerik
http://www.janosgyerik.com/
https://twitter.com/janosgyerik/
Index: tools/hook-scripts/mailer/mailer.py
===
--- tools/hook-scripts/mailer/mailer.py (revision 1438886)
+++ tools/hook-scripts/mailer/mailer.py (working copy)
@@ -98,7 +98,8 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
   if cmd == 'commit':
 revision = int(cmd_args[0])
 repos = Repository(repos_dir, revision, pool)
-cfg = Config(config_fname, repos, { 'author' : repos.author })
+cfg = Config(config_fname, repos, {'author': repos.author,
+'repodir': os.path.basename(repos.repos_dir)})
 messenger = Commit(pool, cfg, repos)
   elif cmd == 'propchange' or cmd == 'propchange2':
 revision = int(cmd_args[0])
@@ -108,14 +109,16 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
 repos = Repository(repos_dir, revision, pool)
 # Override the repos revision author with the author of the propchange
 repos.author = author
-cfg = Config(config_fname, repos, { 'author' : author })
+cfg = Config(config_fname, repos, {'author': author,
+'repodir': os.path.basename(repos.repos_dir)})
 messenger = PropChange(pool, cfg, repos, author, propname, action)
   elif cmd == 'lock' or cmd == 'unlock':
 author = cmd_args[0]
 repos = Repository(repos_dir, 0, pool) ### any old revision will do
 # Override the repos revision author with the author of the lock/unlock
 repos.author = author
-cfg = Config(config_fname, repos, { 'author' : author })
+cfg = Config(config_fname, repos, {'author': author,
+'repodir': os.path.basename(repos.repos_dir)})
 messenger = Lock(pool, cfg, repos, author, cmd == 'lock')
   else:
 raise UnknownSubcommand(cmd)
Index: tools/hook-scripts/mailer/mailer.conf.example
===
--- tools/hook-scripts/mailer/mailer.conf.example   (revision 1438886)
+++ tools/hook-scripts/mailer/mailer.conf.example   (working copy)
@@ -146,7 +146,16 @@
 #
 #   from_addr = %(author)s...@example.com
 #
+# The substitution variable repodir is provided, and is set to
+# the directory name of the repository. This can be useful to set
+# a custom subject that can be re-used in multiple repositories:
 #
+#   commit_subject_prefix = [svn-%(repodir)s]
+#
+# For example if the repository is at /path/to/repo/project-x then
+# the subject of commit emails will be prefixed with [svn-project-x]
+#
+#
 # SUMMARY
 #
 # While mailer.py will work to minimize the number of mail messages

[PATCH] minor change to mailer.py for subject formatting

[Janos Gyerik]

Hi,

This is a minor change to the mailer.py script often used with the
post-commit hook.

The purpose of the change is to make it possible to dynamically insert
the directory name of the repository in the subject text. Simply it
replaces the term REPONAME in the commit_subject_prefix setting with
the base directory name of the repository. For example if your
repository is in `/path/to/test1` and `commit_subject_prefix =
[svn-REPONAME]`, then the subject of commit emails will be prefixed
with `[svn-test1]`

This simplifies the configuration because multiple repositories can
reuse the same mailer.conf file.

[[[
   When formatting commit email subjects, replace REPONAME in the
commit_subject_prefix setting with the basename of the repository
directory
   ]]]

Regards,
Janos

--
Janos Gyerik
http://www.janosgyerik.com/
https://twitter.com/janosgyerik/


mailer-improvement.patch
Description: Binary data

Re: [PATCH] minor change to mailer.py for subject formatting

[Daniel Shahaf]

Please attach patches as MIME text/plain (usually naming them *.txt does
that).  More below.

 +if prefix and re.search(r'REPONAME', prefix):

Needlessly complicated, ('REPONAME' in prefix) would do.

Also I think you should use the %() syntax like mailer.conf.example
does.  Is that possible?

 +reponame = 
 os.path.basename(os.path.dirname(os.path.dirname(__file__)))

That's outright wrong.  You can't assume that post-commit is a symlink
to mailer.py.  (Just read post-commit.tmpl for a counterexample.)

 +prefix = re.sub(r'REPONAME', reponame, prefix)

Daniel
(very brief review, and the setup I help maintain uses svnmailer not
mailer.py, so if the above doesn't make sense sorry and please correct me)

Re: [PATCH] minor change to mailer.py for subject formatting

[C. Michael Pilato]

On 01/26/2013 03:03 PM, Daniel Shahaf wrote:
 Please attach patches as MIME text/plain (usually naming them *.txt does
 that).  More below.
 
 +if prefix and re.search(r'REPONAME', prefix):
 
 Needlessly complicated, ('REPONAME' in prefix) would do.
 
 Also I think you should use the %() syntax like mailer.conf.example
 does.  Is that possible?

Agreed, please use the %()s style of format string substitution if
possible (for consistency with other configurable bits of this script).

-- 
C. Michael Pilato cmpil...@collab.net
CollabNet  www.collab.net  Enterprise Cloud Development



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] minor change to mailer.py for subject formatting

[Janos Gyerik]

Hey guys,

Thanks a lot for your comments!

Please check the new patch I attached, I hope it's good now.

It will allow this kind of setting in mailer.conf:
commit_subject_prefix = [svn-%(repodir)s]

Thanks,
Janos

On Sun, Jan 27, 2013 at 5:36 AM, C. Michael Pilato cmpil...@collab.net wrote:
 On 01/26/2013 03:03 PM, Daniel Shahaf wrote:
 Please attach patches as MIME text/plain (usually naming them *.txt does
 that).  More below.

 +if prefix and re.search(r'REPONAME', prefix):

 Needlessly complicated, ('REPONAME' in prefix) would do.

 Also I think you should use the %() syntax like mailer.conf.example
 does.  Is that possible?

 Agreed, please use the %()s style of format string substitution if
 possible (for consistency with other configurable bits of this script).

 --
 C. Michael Pilato cmpil...@collab.net
 CollabNet  www.collab.net  Enterprise Cloud Development




-- 
Janos Gyerik
http://www.janosgyerik.com/
https://twitter.com/janosgyerik/
Index: tools/hook-scripts/mailer/mailer.py
===
--- tools/hook-scripts/mailer/mailer.py (revision 1438886)
+++ tools/hook-scripts/mailer/mailer.py (working copy)
@@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
   if cmd == 'commit':
 revision = int(cmd_args[0])
 repos = Repository(repos_dir, revision, pool)
-cfg = Config(config_fname, repos, { 'author' : repos.author })
+cfg = Config(config_fname, repos, { 'author' : repos.author, 'repodir' : 
os.path.basename(repos.repos_dir) })
 messenger = Commit(pool, cfg, repos)
   elif cmd == 'propchange' or cmd == 'propchange2':
 revision = int(cmd_args[0])

Re: [PATCH] minor change to mailer.py for subject formatting

[Ben Reser]

On Sat, Jan 26, 2013 at 9:19 PM, Janos Gyerik janos.gye...@gmail.com wrote:
 Hey guys,

 Thanks a lot for your comments!

 Please check the new patch I attached, I hope it's good now.

 It will allow this kind of setting in mailer.conf:
 commit_subject_prefix = [svn-%(repodir)s]

1) Your patch only works for commits and not propchanges or
locks/unlocks.  Should be available to all the methods.

2) You should patch the documentation in mailer.conf.example.

Re: [PATCH] minor change to mailer.py for subject formatting

[Janos Gyerik]

Thanks again!

I addressed those issues, see the new patch.

Cheers,
Janos

On Sun, Jan 27, 2013 at 6:42 AM, Ben Reser b...@reser.org wrote:
 On Sat, Jan 26, 2013 at 9:19 PM, Janos Gyerik janos.gye...@gmail.com wrote:
 Hey guys,

 Thanks a lot for your comments!

 Please check the new patch I attached, I hope it's good now.

 It will allow this kind of setting in mailer.conf:
 commit_subject_prefix = [svn-%(repodir)s]

 1) Your patch only works for commits and not propchanges or
 locks/unlocks.  Should be available to all the methods.

 2) You should patch the documentation in mailer.conf.example.



-- 
Janos Gyerik
http://www.janosgyerik.com/
https://twitter.com/janosgyerik/
Index: tools/hook-scripts/mailer/mailer.py
===
--- tools/hook-scripts/mailer/mailer.py (revision 1438886)
+++ tools/hook-scripts/mailer/mailer.py (working copy)
@@ -98,7 +98,7 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
   if cmd == 'commit':
 revision = int(cmd_args[0])
 repos = Repository(repos_dir, revision, pool)
-cfg = Config(config_fname, repos, { 'author' : repos.author })
+cfg = Config(config_fname, repos, { 'author' : repos.author, 'repodir' : 
os.path.basename(repos.repos_dir) })
 messenger = Commit(pool, cfg, repos)
   elif cmd == 'propchange' or cmd == 'propchange2':
 revision = int(cmd_args[0])
@@ -108,14 +108,14 @@ def main(pool, cmd, config_fname, repos_dir, cmd_a
 repos = Repository(repos_dir, revision, pool)
 # Override the repos revision author with the author of the propchange
 repos.author = author
-cfg = Config(config_fname, repos, { 'author' : author })
+cfg = Config(config_fname, repos, { 'author' : author, 'repodir' : 
os.path.basename(repos.repos_dir) })
 messenger = PropChange(pool, cfg, repos, author, propname, action)
   elif cmd == 'lock' or cmd == 'unlock':
 author = cmd_args[0]
 repos = Repository(repos_dir, 0, pool) ### any old revision will do
 # Override the repos revision author with the author of the lock/unlock
 repos.author = author
-cfg = Config(config_fname, repos, { 'author' : author })
+cfg = Config(config_fname, repos, { 'author' : author, 'repodir' : 
os.path.basename(repos.repos_dir) })
 messenger = Lock(pool, cfg, repos, author, cmd == 'lock')
   else:
 raise UnknownSubcommand(cmd)
Index: tools/hook-scripts/mailer/mailer.conf.example
===
--- tools/hook-scripts/mailer/mailer.conf.example   (revision 1438886)
+++ tools/hook-scripts/mailer/mailer.conf.example   (working copy)
@@ -146,7 +146,16 @@
 #
 #   from_addr = %(author)s...@example.com
 #
+# The substitution variable repodir is provided, and is set to
+# the directory name of the repository. This can be useful to set
+# a custom subject that can be re-used in multiple repositories:
 #
+#   commit_subject_prefix = [svn-%(repodir)s]
+#
+# For example if the repository is at /path/to/repo/project-x then
+# the subject of commit emails will be prefixed with [svn-project-x]
+#
+#
 # SUMMARY
 #
 # While mailer.py will work to minimize the number of mail messages

Subject: [PATCH] WC rep cache optimization for some file systems like NTFS on Windows

[Vasily Tunegov]

[[[
   WC rep cache optimization for some file systems like NTFS on Windows
]]]

Hello,

Please review my patch (see attached file patch.txt). It improve svn client
performance on file systems like NTFS on Windows. Maybe on some other file
systems, e.g. NFS, but I didn't test it. This patch switch rep cache db
journal mode from DELETE (by default) to PERSISTENT. For more info about
journal modes in SQLite see
http://www.sqlite.org/pragma.html#pragma_journal_mode.

I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7 x64
Enterprise) for two disk drives: standard and ssd. For tests I used
Subversion Benchmark Tool, see
https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool.
For tested application I used svn 1.8.0 from trunk, revision 1387070.

Subversion Benchmark Tool tests, total results:

Standard disk:
svn.r1387070 - 3:05.895
svn.patch - 2:06.548

SSD disk:
svn.r1387070 - 2:11.504
svn.patch - 1:38.397

For detailed results see attached files: std.180.txt, std.patch.txt,
ssd.180.txt, ssd.patch.txt
Index: subversion/libsvn_subr/sqlite.c
===
--- subversion/libsvn_subr/sqlite.c (revision 1387107)
+++ subversion/libsvn_subr/sqlite.c (working copy)
@@ -858,6 +858,12 @@ svn_sqlite__open(svn_sqlite__db_t **db, const char
   SVN_ERR(exec_sql(*db, PRAGMA foreign_keys=ON;));
 #endif
 
+  /* The PERSIST journaling mode is useful as an optimization on platforms
+ where deleting or truncating a file is much more expensive than 
+ overwriting the first block of a file with zeros. For example
+ NTFS on Windows.*/
+  SVN_ERR(exec_sql(*db, PRAGMA journal_mode = PERSIST));
+
   /* Store temporary tables in RAM instead of in temporary files, but don't
  fail on this if this option is disabled in the sqlite compilation by
  setting SQLITE_TEMP_STORE to 0 (always to disk) */
$ svn --version -q
$ svn mkdir -q --parents -m Create folder for this test run 
file:///d:/benchmark/repos/tests/ef842e78-5107-426a-932d-1a1bb8dec1db
$ svn cp -q -m Create trunk folder for tests file:///d:/benchmark/repos/trunk 
file:///d:/benchmark/repos/tests/ef842e78-5107-426a-932d-1a1bb8dec1db/trunk
$ svn co -q -r4 file:///d:/benchmark/repos/trunk .
$ svn up -q
$ svn sw -q 
file:///d:/benchmark/repos/tests/ef842e78-5107-426a-932d-1a1bb8dec1db/trunk
$ svn pl -q -R .
$ svn st -q
$ svn ci -q -m Commit a few files
$ svnversion
$ svn rm -q -m Removing test folder 
file:///d:/benchmark/repos/tests/ef842e78-5107-426a-932d-1a1bb8dec1db
$ svn --version -q
$ svn mkdir -q --parents -m Create folder for this test run 
file:///d:/benchmark/repos/tests/ef66434a-0e8f-476e-9cfc-19fe1abd6464
$ svn cp -q -m Create branch folder for tests -r12 
file:///d:/benchmark/repos/trunk 
file:///d:/benchmark/repos/tests/ef66434a-0e8f-476e-9cfc-19fe1abd6464/branch1
$ svn co -q 
file:///d:/benchmark/repos/tests/ef66434a-0e8f-476e-9cfc-19fe1abd6464/branch1 .
$ svn merge -q file:///d:/benchmark/repos/trunk
$ svn revert -q -R .
$ svn ci -q -m Commit changes to branch
$ svn up -q
$ svn merge -q --accept=theirs-full file:///d:/benchmark/repos/trunk
$ svn ci -q -m Synch up branch with trunk
svn: E155015: Commit failed (details follow):
svn: E155015: Aborting commit: 
'D:\benchmark\ef66434a-0e8f-476e-9cfc-19fe1abd6464\notes\object-model.txt' 
remains in conflict

$ svn sw -q file:///d:/benchmark/repos/trunk
$ svn merge -q --accept=theirs-full --reintegrate 
file:///d:/benchmark/repos/tests/ef66434a-0e8f-476e-9cfc-19fe1abd6464/branch1
$ svn rm -q -m Removing test folder 
file:///d:/benchmark/repos/tests/ef66434a-0e8f-476e-9cfc-19fe1abd6464
$ svn --version -q
$ svn mkdir -q --parents -m Create folder for this test run 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274
$ svn mkdir -q -m Create trunk folder to hold content 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274/main
$ svn cp -q -m Copy trunk folder file:///d:/benchmark/repos/trunk 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274/main/trunk
$ svn cp -q -m Copy tags folder file:///d:/benchmark/repos/tags 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274/main/tags
$ svn cp -q -m Copy tags folder again file:///d:/benchmark/repos/tags 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274/main/more-tags
$ svn cp -q -m Copy branches folder file:///d:/benchmark/repos/branches 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274/main/branches
$ svn cp -q -m Create branch for tests 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274/main 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274/branch1
$ svn co -q 
file:///d:/benchmark/repos/tests/ef329efe-c8b9-48fa-9231-a86f1bb72274/main .
$ svn st -q
$ svn ci -q -m Commit files from WC with lots of folders
$ svn up -q
$ svnversion
$ svn sw -q 

Re: Subject: [PATCH] WC rep cache optimization for some file systems like NTFS on Windows

[Philip Martin]

Vasily Tunegov vasily.tune...@gmail.com writes:

 Please review my patch (see attached file patch.txt). It improve svn client
 performance on file systems like NTFS on Windows. Maybe on some other file
 systems, e.g. NFS, but I didn't test it. This patch switch rep cache db
 journal mode from DELETE (by default) to PERSISTENT. For more info about
 journal modes in SQLite see
 http://www.sqlite.org/pragma.html#pragma_journal_mode.

 I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7 x64
 Enterprise) for two disk drives: standard and ssd. For tests I used
 Subversion Benchmark Tool, see
 https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool.
 For tested application I used svn 1.8.0 from trunk, revision 1387070.

 Subversion Benchmark Tool tests, total results:

 Standard disk:
 svn.r1387070 - 3:05.895
 svn.patch - 2:06.548

 SSD disk:
 svn.r1387070 - 2:11.504
 svn.patch - 1:38.397

Which version of SQLite?

On my Linux systems this gives a small improvement in checkout
performance for working copies on local disks, unfortunately it also
reduces checkout performance for working copies on NFS disks.

I'm using SQLite 3.7.12.1 and 3.7.13.

-- 
Certified  Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download

Re: Subject: [PATCH] WC rep cache optimization for some file systems like NTFS on Windows

[Vasily Tunegov]

Hi, Philip

I'm using SQLite 3.7.14.
I tried to test checkout for our project code base, hosted on svn 1.7.6
server (93 984 Files, 3 204 Folders, 1.97GB). There are results:

svn co snv://server/project/branches/branche src -q

Standard disk:
svn.r1387070 - 256.8 sec
svn.patch - 152.2 sec

My research shows that PERSIST journal mode greatly reducing the number of
I/O operations, such as creating, opening and closing SQLite journal files
on the disk. The best option here is WAL journal mode. But it works only if
SQLite database is used from one computer (it use memory mapped files API).
I don't know is this restriction is important to svn client.

Best regards,
Vasily Tunegov.

On Wed, Sep 19, 2012 at 2:30 PM, Philip Martin
philip.mar...@wandisco.comwrote:

 Vasily Tunegov vasily.tune...@gmail.com writes:

  Please review my patch (see attached file patch.txt). It improve svn
 client
  performance on file systems like NTFS on Windows. Maybe on some other
 file
  systems, e.g. NFS, but I didn't test it. This patch switch rep cache db
  journal mode from DELETE (by default) to PERSISTENT. For more info about
  journal modes in SQLite see
  http://www.sqlite.org/pragma.html#pragma_journal_mode.
 
  I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7 x64
  Enterprise) for two disk drives: standard and ssd. For tests I used
  Subversion Benchmark Tool, see
 
 https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool
 .
  For tested application I used svn 1.8.0 from trunk, revision 1387070.
 
  Subversion Benchmark Tool tests, total results:
 
  Standard disk:
  svn.r1387070 - 3:05.895
  svn.patch - 2:06.548
 
  SSD disk:
  svn.r1387070 - 2:11.504
  svn.patch - 1:38.397

 Which version of SQLite?

 On my Linux systems this gives a small improvement in checkout
 performance for working copies on local disks, unfortunately it also
 reduces checkout performance for working copies on NFS disks.

 I'm using SQLite 3.7.12.1 and 3.7.13.

 --
 Certified  Supported Apache Subversion Downloads:
 http://www.wandisco.com/subversion/download

RE: Subject: [PATCH] WC rep cache optimization for some file

[Bert Huijben]

 systems like NTFS on Windows
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary=20cf3074b2a0fd54e504ca0ed837

--20cf3074b2a0fd54e504ca0ed837
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

For your own install you can enable the WAL on the file and other users
of the db will use it. I don't think we can enable it as default (like
you said)

Bert Huijben (Cell phone)
From: Vasily Tunegov
Sent: 19-9-2012 7:30
To: Philip Martin
Cc: dev@subversion.apache.org
Subject: Re: Subject: [PATCH] WC rep cache optimization for some file
systems like NTFS on Windows
Hi, Philip

I'm using SQLite 3.7.14.
I tried to test checkout for our project code base, hosted on svn 1.7.6
server (93 984 Files, 3 204 Folders, 1.97GB). There are results:

svn co snv://server/project/branches/branche src -q

Standard disk:
svn.r1387070 - 256.8 sec
svn.patch - 152.2 sec

My research shows that PERSIST journal mode greatly reducing the number of
I/O operations, such as creating, opening and closing SQLite journal files
on the disk. The best option here is WAL journal mode. But it works only if
SQLite database is used from one computer (it use memory mapped files API).
I don't know is this restriction is important to svn client.

Best regards,
Vasily Tunegov.

On Wed, Sep 19, 2012 at 2:30 PM, Philip Martin
philip.mar...@wandisco.comwrote:

 Vasily Tunegov vasily.tune...@gmail.com writes:

  Please review my patch (see attached file patch.txt). It improve svn
 client
  performance on file systems like NTFS on Windows. Maybe on some other
 file
  systems, e.g. NFS, but I didn't test it. This patch switch rep cache db
  journal mode from DELETE (by default) to PERSISTENT. For more info about
  journal modes in SQLite see
  http://www.sqlite.org/pragma.html#pragma_journal_mode.
 
  I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7 x64
  Enterprise) for two disk drives: standard and ssd. For tests I used
  Subversion Benchmark Tool, see
 
 https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool
 .
  For tested application I used svn 1.8.0 from trunk, revision 1387070.
 
  Subversion Benchmark Tool tests, total results:
 
  Standard disk:
  svn.r1387070 - 3:05.895
  svn.patch - 2:06.548
 
  SSD disk:
  svn.r1387070 - 2:11.504
  svn.patch - 1:38.397

 Which version of SQLite?

 On my Linux systems this gives a small improvement in checkout
 performance for working copies on local disks, unfortunately it also
 reduces checkout performance for working copies on NFS disks.

 I'm using SQLite 3.7.12.1 and 3.7.13.

 --
 Certified  Supported Apache Subversion Downloads:
 http://www.wandisco.com/subversion/download


--20cf3074b2a0fd54e504ca0ed837
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

htmlheadmeta content=3Dtext/html; charset=3Dutf-8 http-equiv=3DCont=
ent-Type/headbodydivdiv style=3Dfont-family: Calibri,sans-serif; =
font-size: 11pt;For your own install you can enable the WAL on the file a=
nd other users of the db will use it. I don't think we can enable it as def=
ault (like you said)brbrBert Huijben (Cell phone)br/div/divhrs=
pan style=3Dfont-family: Tahoma,sans-serif; font-size: 10pt; font-weight: =
bold;From: /spanspan style=3Dfont-family: Tahoma,sans-serif; font-siz=
e: 10pt;Vasily Tunegov/spanbrspan style=3Dfont-family: Tahoma,sans-=
serif; font-size: 10pt; font-weight: bold;Sent: /spanspan style=3Dfon=
t-family: Tahoma,sans-serif; font-size: 10pt;19-9-2012 7:30/spanbrsp=
an style=3Dfont-family: Tahoma,sans-serif; font-size: 10pt; font-weight: b=
old;To: /spanspan style=3Dfont-family: Tahoma,sans-serif; font-size: =
10pt;Philip Martin/spanbrspan style=3Dfont-family: Tahoma,sans-seri=
f; font-size: 10pt; font-weight: bold;Cc: /spanspan style=3Dfont-fami=
ly: Tahoma,sans-serif; font-size: 10pt;dev@subversion.apache.org/spanb=
rspan style=3Dfont-family: Tahoma,sans-serif; font-size: 10pt; font-weig=
ht: bold;Subject: /spanspan style=3Dfont-family: Tahoma,sans-serif; f=
ont-size: 10pt;Re: Subject: [PATCH] WC rep cache optimization for some fi=
le systems like NTFS on Windows/spanbrbr/body/htmldiv style=3Dc=
olor:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-c=
olor:rgb(255,255,255)Hi,=C2=A0Philip/divdiv style=3Dcolor:rgb(34,34,3=
4);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255=
,255)
br/divspan style=3Dcolor:rgb(34,34,34);font-family:arial,sans-serif;f=
ont-size:13px;background-color:rgb(255,255,255)I#39;m using SQLite 3.7.1=
4.=C2=A0/spandiv style=3Dcolor:rgb(34,34,34);font-family:arial,sans-ser=
if;font-size:13px;background-color:rgb(255,255,255)
I tried to test checkout for our project code base, hosted on svn 1.7.6 ser=
ver (93 984 Files, 3 204 Folders, 1.97GB). There are results:/divdiv sty=
le=3Dcolor:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;backg=
round-color:rgb(255,255,255)
br/divdiv style=3Dcolor:rgb(34,34,34);font-family:arial,sans

Re: Subject: [PATCH] WC rep cache optimization for some file

[Vasily Tunegov]

Hi, Bert

I understand what WAL mode is not suitable for svn. But my patch uses
PERSIST mode which give very good results, e.g. for NTFS on Windows as I
shown in tests.

One problem, as says Philip Martin, is reduced performance on NFS disks.
But I don't understand why. The number of file operations in PERSIST mode
greatly reduced. Unfortunaly I don't have NFS disk, if it means Network
File System developed by Sun Microsystems. Windows is using SMB protocol to
access files via network. I will test SBM and put result here later.

Best regards,
Vasily Tunegov.

On Wed, Sep 19, 2012 at 7:30 PM, Bert Huijben b...@vmoo.com wrote:

  systems like NTFS on Windows
 MIME-Version: 1.0
 Content-Type: multipart/alternative; boundary=20cf3074b2a0fd54e504ca0ed837

 --20cf3074b2a0fd54e504ca0ed837
 Content-Type: text/plain; charset=utf-8
 Content-Transfer-Encoding: 7bit

 For your own install you can enable the WAL on the file and other users
 of the db will use it. I don't think we can enable it as default (like
 you said)

 Bert Huijben (Cell phone)
 From: Vasily Tunegov
 Sent: 19-9-2012 7:30
 To: Philip Martin
 Cc: dev@subversion.apache.org
 Subject: Re: Subject: [PATCH] WC rep cache optimization for some file
 systems like NTFS on Windows
 Hi, Philip

 I'm using SQLite 3.7.14.
 I tried to test checkout for our project code base, hosted on svn 1.7.6
 server (93 984 Files, 3 204 Folders, 1.97GB). There are results:

 svn co snv://server/project/branches/branche src -q

 Standard disk:
 svn.r1387070 - 256.8 sec
 svn.patch - 152.2 sec

 My research shows that PERSIST journal mode greatly reducing the number of
 I/O operations, such as creating, opening and closing SQLite journal files
 on the disk. The best option here is WAL journal mode. But it works only if
 SQLite database is used from one computer (it use memory mapped files API).
 I don't know is this restriction is important to svn client.

 Best regards,
 Vasily Tunegov.

 On Wed, Sep 19, 2012 at 2:30 PM, Philip Martin
 philip.mar...@wandisco.comwrote:

  Vasily Tunegov vasily.tune...@gmail.com writes:
 
   Please review my patch (see attached file patch.txt). It improve svn
  client
   performance on file systems like NTFS on Windows. Maybe on some other
  file
   systems, e.g. NFS, but I didn't test it. This patch switch rep cache db
   journal mode from DELETE (by default) to PERSISTENT. For more info
 about
   journal modes in SQLite see
   http://www.sqlite.org/pragma.html#pragma_journal_mode.
  
   I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7
 x64
   Enterprise) for two disk drives: standard and ssd. For tests I used
   Subversion Benchmark Tool, see
  
 
 https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool
  .
   For tested application I used svn 1.8.0 from trunk, revision 1387070.
  
   Subversion Benchmark Tool tests, total results:
  
   Standard disk:
   svn.r1387070 - 3:05.895
   svn.patch - 2:06.548
  
   SSD disk:
   svn.r1387070 - 2:11.504
   svn.patch - 1:38.397
 
  Which version of SQLite?
 
  On my Linux systems this gives a small improvement in checkout
  performance for working copies on local disks, unfortunately it also
  reduces checkout performance for working copies on NFS disks.
 
  I'm using SQLite 3.7.12.1 and 3.7.13.
 
  --
  Certified  Supported Apache Subversion Downloads:
  http://www.wandisco.com/subversion/download
 

 --20cf3074b2a0fd54e504ca0ed837
 Content-Type: text/html; charset=utf-8
 Content-Transfer-Encoding: quoted-printable

 htmlheadmeta content=3Dtext/html; charset=3Dutf-8
 http-equiv=3DCont=
 ent-Type/headbodydivdiv style=3Dfont-family: Calibri,sans-serif;
 =
 font-size: 11pt;For your own install you can enable the WAL on the file
 a=
 nd other users of the db will use it. I don't think we can enable it as
 def=
 ault (like you said)brbrBert Huijben (Cell
 phone)br/div/divhrs=
 pan style=3Dfont-family: Tahoma,sans-serif; font-size: 10pt; font-weight:
 =
 bold;From: /spanspan style=3Dfont-family: Tahoma,sans-serif;
 font-siz=
 e: 10pt;Vasily Tunegov/spanbrspan style=3Dfont-family:
 Tahoma,sans-=
 serif; font-size: 10pt; font-weight: bold;Sent: /spanspan
 style=3Dfon=
 t-family: Tahoma,sans-serif; font-size: 10pt;19-9-2012
 7:30/spanbrsp=
 an style=3Dfont-family: Tahoma,sans-serif; font-size: 10pt; font-weight:
 b=
 old;To: /spanspan style=3Dfont-family: Tahoma,sans-serif; font-size:
 =
 10pt;Philip Martin/spanbrspan style=3Dfont-family:
 Tahoma,sans-seri=
 f; font-size: 10pt; font-weight: bold;Cc: /spanspan
 style=3Dfont-fami=
 ly: Tahoma,sans-serif; font-size: 10pt;dev@subversion.apache.org
 /spanb=
 rspan style=3Dfont-family: Tahoma,sans-serif; font-size: 10pt;
 font-weig=
 ht: bold;Subject: /spanspan style=3Dfont-family: Tahoma,sans-serif;
 f=
 ont-size: 10pt;Re: Subject: [PATCH] WC rep cache optimization for some
 fi=
 le systems like NTFS on Windows/spanbrbr/body/htmldiv
 style=3Dc=

 olor:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px

Re: Subject: [PATCH] WC rep cache optimization for some file

[Mark Phippard]

On Wed, Sep 19, 2012 at 2:18 PM, Vasily Tunegov vasily.tune...@gmail.comwrote:


 I understand what WAL mode is not suitable for svn. But my patch uses
 PERSIST mode which give very good results, e.g. for NTFS on Windows as I
 shown in tests.

 One problem, as says Philip Martin, is reduced performance on NFS disks.
 But I don't understand why. The number of file operations in PERSIST mode
 greatly reduced. Unfortunaly I don't have NFS disk, if it means Network
 File System developed by Sun Microsystems. Windows is using SMB protocol to
 access files via network. I will test SBM and put result here later.


Off-topic but related.  What has happened to Philip's patch to enable
exclusive locking for SQLite which solves the performance issue with NFS?
 I thought we were going to try to enable that via some runtime option or
perhaps when we detect the working copy is on network?

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: Subject: [PATCH] WC rep cache optimization for some file

[Philip Martin]

Mark Phippard markp...@gmail.com writes:

 Off-topic but related.  What has happened to Philip's patch to enable
 exclusive locking for SQLite which solves the performance issue with NFS?
  I thought we were going to try to enable that via some runtime option or
 perhaps when we detect the working copy is on network?

I've just updated the issue.  I started on a runtime patch and ran into
a problem, the config hash in the working copy library isn't connected
to the config has used in the client.

-- 
Certified  Supported Apache Subversion Downloads:
http://www.wandisco.com/subversion/download

Re: Subject: [PATCH] WC rep cache optimization for some file

[Philip Martin]

Philip Martin philip.mar...@wandisco.com writes:

 Mark Phippard markp...@gmail.com writes:

 Off-topic but related.  What has happened to Philip's patch to enable
 exclusive locking for SQLite which solves the performance issue with NFS?
  I thought we were going to try to enable that via some runtime option or
 perhaps when we detect the working copy is on network?

 I've just updated the issue.  I started on a runtime patch and ran into
 a problem, the config hash in the working copy library isn't connected
 to the config has used in the client.

If you look at the svn client it svn_client_create_context which pass
NULL to svn_wc_context_create since no config hash is available at that
stage.  Later the svn client modifies the client context to set the
client config hash.  It's not clear how one is supposed to set the
working copy config hash.

-- 
Philip

Re: Subject: [PATCH] WC rep cache optimization for some file

[Vasily Tunegov]

There are new results for local and network disk (checkout large code base):

Local disk:
svn.r1387070 - 256.8 sec
svn.patch - 152.2 sec

Network disk:
svn.r1387070 - 5717 sec
svn.patch - 2790 sec

All systems use Windows 7.

On Wed, Sep 19, 2012 at 10:18 PM, Vasily Tunegov
vasily.tune...@gmail.comwrote:

 Hi, Bert

 I understand what WAL mode is not suitable for svn. But my patch uses
 PERSIST mode which give very good results, e.g. for NTFS on Windows as I
 shown in tests.

 One problem, as says Philip Martin, is reduced performance on NFS disks.
 But I don't understand why. The number of file operations in PERSIST mode
 greatly reduced. Unfortunaly I don't have NFS disk, if it means Network
 File System developed by Sun Microsystems. Windows is using SMB protocol to
 access files via network. I will test SBM and put result here later.

 Best regards,
 Vasily Tunegov.

 On Wed, Sep 19, 2012 at 7:30 PM, Bert Huijben b...@vmoo.com wrote:

  systems like NTFS on Windows
 MIME-Version: 1.0
 Content-Type: multipart/alternative; boundary=20cf3074b2a0fd54e504ca0ed837

 --20cf3074b2a0fd54e504ca0ed837
 Content-Type: text/plain; charset=utf-8
 Content-Transfer-Encoding: 7bit

 For your own install you can enable the WAL on the file and other users
 of the db will use it. I don't think we can enable it as default (like
 you said)

 Bert Huijben (Cell phone)
 From: Vasily Tunegov
 Sent: 19-9-2012 7:30
 To: Philip Martin
 Cc: dev@subversion.apache.org
 Subject: Re: Subject: [PATCH] WC rep cache optimization for some file
 systems like NTFS on Windows
 Hi, Philip

 I'm using SQLite 3.7.14.
 I tried to test checkout for our project code base, hosted on svn 1.7.6
 server (93 984 Files, 3 204 Folders, 1.97GB). There are results:

 svn co snv://server/project/branches/branche src -q

 Standard disk:
 svn.r1387070 - 256.8 sec
 svn.patch - 152.2 sec

 My research shows that PERSIST journal mode greatly reducing the number of
 I/O operations, such as creating, opening and closing SQLite journal files
 on the disk. The best option here is WAL journal mode. But it works only
 if
 SQLite database is used from one computer (it use memory mapped files
 API).
 I don't know is this restriction is important to svn client.

 Best regards,
 Vasily Tunegov.

 On Wed, Sep 19, 2012 at 2:30 PM, Philip Martin
 philip.mar...@wandisco.comwrote:

  Vasily Tunegov vasily.tune...@gmail.com writes:
 
   Please review my patch (see attached file patch.txt). It improve svn
  client
   performance on file systems like NTFS on Windows. Maybe on some other
  file
   systems, e.g. NFS, but I didn't test it. This patch switch rep cache
 db
   journal mode from DELETE (by default) to PERSISTENT. For more info
 about
   journal modes in SQLite see
   http://www.sqlite.org/pragma.html#pragma_journal_mode.
  
   I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7
 x64
   Enterprise) for two disk drives: standard and ssd. For tests I used
   Subversion Benchmark Tool, see
  
 
 https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool
  .
   For tested application I used svn 1.8.0 from trunk, revision 1387070.
  
   Subversion Benchmark Tool tests, total results:
  
   Standard disk:
   svn.r1387070 - 3:05.895
   svn.patch - 2:06.548
  
   SSD disk:
   svn.r1387070 - 2:11.504
   svn.patch - 1:38.397
 
  Which version of SQLite?
 
  On my Linux systems this gives a small improvement in checkout
  performance for working copies on local disks, unfortunately it also
  reduces checkout performance for working copies on NFS disks.
 
  I'm using SQLite 3.7.12.1 and 3.7.13.
 
  --
  Certified  Supported Apache Subversion Downloads:
  http://www.wandisco.com/subversion/download
 

 --20cf3074b2a0fd54e504ca0ed837
 Content-Type: text/html; charset=utf-8
 Content-Transfer-Encoding: quoted-printable

 htmlheadmeta content=3Dtext/html; charset=3Dutf-8
 http-equiv=3DCont=
 ent-Type/headbodydivdiv style=3Dfont-family:
 Calibri,sans-serif; =
 font-size: 11pt;For your own install you can enable the WAL on the file
 a=
 nd other users of the db will use it. I don't think we can enable it as
 def=
 ault (like you said)brbrBert Huijben (Cell
 phone)br/div/divhrs=
 pan style=3Dfont-family: Tahoma,sans-serif; font-size: 10pt;
 font-weight: =
 bold;From: /spanspan style=3Dfont-family: Tahoma,sans-serif;
 font-siz=
 e: 10pt;Vasily Tunegov/spanbrspan style=3Dfont-family:
 Tahoma,sans-=
 serif; font-size: 10pt; font-weight: bold;Sent: /spanspan
 style=3Dfon=
 t-family: Tahoma,sans-serif; font-size: 10pt;19-9-2012
 7:30/spanbrsp=
 an style=3Dfont-family: Tahoma,sans-serif; font-size: 10pt; font-weight:
 b=
 old;To: /spanspan style=3Dfont-family: Tahoma,sans-serif;
 font-size: =
 10pt;Philip Martin/spanbrspan style=3Dfont-family:
 Tahoma,sans-seri=
 f; font-size: 10pt; font-weight: bold;Cc: /spanspan
 style=3Dfont-fami=
 ly: Tahoma,sans-serif; font-size: 10pt;dev@subversion.apache.org
 /spanb=
 rspan style=3Dfont-family: Tahoma

[no subject]

[Owain Lewis]

-- 
Regards,

Owain Lewis
Lead Architect and Principal Engineer
WANdisco, Inc.

Office  : +44 (0)114 3039985
Web: http://www.wandisco.com

uberSVN: Subversion Made Easy
http://www.uberSVN.com http://www.ubersvn.com/

Everything you need to deploy Subversion in the Enterprise
http://www.wandisco.com/subversionhttp://www.wandisco.com/subversion/multisite

Subversion community
http://www.svnforum.org

Read our blogs
http://blogs.wandisco.com/

Follow us on Twitter
http://www.twitter.com/wandisco


THIS MESSAGE AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY, AND MAY BE
PRIVILEGED.  If this message was misdirected, WANdisco, Inc. and its
subsidiaries, (WANdisco) does not waive any confidentiality or privilege.
 If you are not the intended recipient, please notify us immediately and
destroy the message without disclosing its contents to anyone.  Any
distribution, use or copying of this e-mail or the information it contains
by other than an intended recipient is unauthorized.  The views and
opinions expressed in this e-mail message are the author's own and may not
reflect the views and opinions of WANdisco, unless the author is authorized
by WANdisco to express such views or opinions on its behalf.  All email
sent to or from this address is subject to electronic storage and review by
WANdisco.  Although WANdisco operates anti-virus programs, it does not
accept responsibility for any damage whatsoever caused by viruses being
passed.

[no subject]

[schrottili]

Hi,

I get the following message when trying to update my directory
(after having done this approx. 500times without a problem!):

=

---
Subversion Exception!
---
Subversion encountered a serious problem.
Please take the time to report this on the Subversion mailing list
with as much information as possible about what
you were trying to do.
But please first search the mailing list archives for the error message
to avoid reporting the same problem repeatedly.
You can find the mailing list archives at
http://subversion.apache.org/mailing-lists.html

Subversion reported the following
(you can copy the content of this dialog
to the clipboard using Ctrl-C):

In file
 
'D:\Development\SVN\Releases\TortoiseSVN-1.7.1\ext\subversion\subversion\libsvn_wc\wc_db_pristine.c'
 line 228: assertion failed (sha1_checksum != NULL)
---
OK   
---


=

I have no idea why this suddenly occurs .

Cheers
schrottili