Re: Incomplete xml output when using --xml to non-existent server

[Naga Saketh Jaligama]

Hi All,




In my company, developers may use tortoise svn or eclipse svn or any svn
client, so no matter what they use i want to enforce them to give some
commit messages and jira id in it. so, i used the below pre-commit shell
script in the svn server in hooks folder and tested with tortoise, it
doesn't work. Can someone help me?

REPOS="$1"
TXN="$2"
SVNLOOK=/usr/bin/svnlook
CURL=/usr/bin/curl
JIRAURL=http://our.jira.url:8080/rest/api/latest/issue
# Make sure that the log message contains some text.
LOGMSG=$($SVNLOOK log -t "$TXN" "$REPOS")
echo ${LOGMSG} | grep "[a-zA-Z0-9]" > /dev/null || exit 1
check that log message starts with a JIRA ticket
should have format 'FOO-123: my commit message' or 'FOO-123 my commit
message'
JIRAID=$(expr "${LOGMSG}" : '^\([A-Z]*-[0-9]*\)[: ].*')
 if [[ "$JIRAID" == "" ]]
then
echo "No JIRA id found in log message \"${LOGMSG}\"" >&2
echo "Please use log message of the form \"JIRA-ID: My message\"" >&2
exit 1
fi
JIRAISSUE=$(${CURL} ${JIRAURL}/${JIRAID})
if [[ "${JIRAISSUE}" =~ "Issue Does Not Exist" ]]
then
echo "The JIRA id ${JIRAID} was not found" >&2
echo "Please use log message of the form \"JIRA-ID: My message\"" >&2
exit 1
fi












On Thu, Mar 3, 2016 at 8:42 AM, Johan Corveleyn  wrote:

> On Thu, Mar 3, 2016 at 3:11 PM, Vincent Lefevre 
> wrote:
> > On 2016-03-03 10:31:52 +0100, Johan Corveleyn wrote:
> >> No, of course not :-). I just gave an example where the output was
> >> broken (host not found), as opposed to another error condition (server
> >> reponds "URL 'X' non-existent in revision Y") where the xml response
> >> is still valid. Ignoring implementation (which I always do when I'm
> >> arguing about behavior), this seems quite weird to me.
> >
> > No, this is different. In the former case, this is a server or
> > communication problem: it is not possible to output the info
> > because it is not possible to know what it is. In the latter case,
> > the communication is successful, so that it is possible to output
> > the info.
>
> Agreed, it's different. But it's still weird / inconsistent IMO.
>
> As I said, I don't know anything about the implementation of --xml,
> but it seems to me that svn could easily complete the output by
> closing the root element with "". Apparently *something* goes
> wrong executing the info request ... let's just cleanup nicely.
>
> Another example, no server communication needed:
>
> [[[
> C:\>svn info --xml .
> 
> 
> svn: E155007: 'C:\' is not a working copy
>
> ]]]
>
>
> But, go inside a working copy, and use a non-existing path:
>
> [[[
> C:\WorkingCopy>svn info --xml blah
> 
> 
> svn: warning: W155010: The node 'C:\WorkingCopy\blah' was not found.
>
> 
> svn: E29: Could not display info for all targets because some
> targets don't exist
>
> ]]]
>
>
> Why  in one case and not in the other? As a user, I see no
> reason for that.
>
> --
> Johan
>

Re: Incomplete xml output when using --xml to non-existent server

[Johan Corveleyn]

On Thu, Mar 3, 2016 at 3:11 PM, Vincent Lefevre  wrote:
> On 2016-03-03 10:31:52 +0100, Johan Corveleyn wrote:
>> No, of course not :-). I just gave an example where the output was
>> broken (host not found), as opposed to another error condition (server
>> reponds "URL 'X' non-existent in revision Y") where the xml response
>> is still valid. Ignoring implementation (which I always do when I'm
>> arguing about behavior), this seems quite weird to me.
>
> No, this is different. In the former case, this is a server or
> communication problem: it is not possible to output the info
> because it is not possible to know what it is. In the latter case,
> the communication is successful, so that it is possible to output
> the info.

Agreed, it's different. But it's still weird / inconsistent IMO.

As I said, I don't know anything about the implementation of --xml,
but it seems to me that svn could easily complete the output by
closing the root element with "". Apparently *something* goes
wrong executing the info request ... let's just cleanup nicely.

Another example, no server communication needed:

[[[
C:\>svn info --xml .


svn: E155007: 'C:\' is not a working copy

]]]


But, go inside a working copy, and use a non-existing path:

[[[
C:\WorkingCopy>svn info --xml blah


svn: warning: W155010: The node 'C:\WorkingCopy\blah' was not found.


svn: E29: Could not display info for all targets because some
targets don't exist

]]]


Why  in one case and not in the other? As a user, I see no
reason for that.

-- 
Johan

Re: Incomplete xml output when using --xml to non-existent server

[Vincent Lefevre]

On 2016-03-03 10:31:52 +0100, Johan Corveleyn wrote:
> No, of course not :-). I just gave an example where the output was
> broken (host not found), as opposed to another error condition (server
> reponds "URL 'X' non-existent in revision Y") where the xml response
> is still valid. Ignoring implementation (which I always do when I'm
> arguing about behavior), this seems quite weird to me.

No, this is different. In the former case, this is a server or
communication problem: it is not possible to output the info
because it is not possible to know what it is. In the latter case,
the communication is successful, so that it is possible to output
the info.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Unversioned files with invalid UTF-8 sequence in name confuse svn

[Vincent Lefevre]

On 2016-03-01 17:12:05 +0100, Branko Čibej wrote:
> On 01.03.2016 16:58, Markus Schaber wrote:
> > Hi, Bert,
> >
> > From: Bert Huijben [mailto:b...@qqmail.nl]
> >> From: Markus Schaber [mailto:m.scha...@codesys.com]
> >>> Hi, Brane and Vincent,
> >>>
> >>> From: Branko Čibej [mailto:br...@apache.org]
> > However Subversion doesn't handle that (BTW it would be much
> > better to
> > remember the expected locale by storing it in the .svn directory
> > 
> >
> > rather than giving obscure error messages: if it did, Subversion
> > would know that the user was using an incorrect locale without any
> > ambiguity).
>  And if the user changes the locale for valid reasons, the Subversion
>  working copy would break in a different way.
> >>> I guess we would need some "change locale" operation, which would at
> >>> least update the saved locale in the .svn directory.
> >> There is no saved locale in the .svn directory...
> > Currently, yes, but it was suggested in the discussion, see the line above.
> 
> The major problem with the "saved locale" idea is that it creates yet
> another potential discrepancy. I really can't imagine how we'd be doing
> the majority of our users a service by adding another knob that can
> seriously break things when it's misconfigured, but doesn't do anything
> useful most of the time.

I don't understand what you mean here. As you said, Subversion
expects the user to always use the same locale with a given working
copy.

Currently:

  * If the user always uses the same locale, then everything is fine
most of the time. But if some tool writes filenames that cannot
be interpreted in this locale (this was the beginning of this
thread), the working copy gets unusable until the user removes
these files manually (which can be tedious).

  * If the user changes his locale, then the working copy is in an
inconsistent state; it is either unusable, or usable but with
incorrect information, which could lead to incorrect commits.

If the locale were recorded in .svn:

  * If the user always uses the same locale, then everything would be
fine all the time. If some tool writes filenames that cannot be
interpreted in this locale, then svn would know that it is not
a problem due to the change of the locale, i.e. it would know that
the file is necessarily unversioned. So, "svn st" would no longer
have any reason to fail, and svn-clean could work as expected
without this risk of being wrong.

  * If the user changes his locale, then svn would be able to emit
a clear error message about the locale mismatch. Then the user
could easily know what was wrong and change back to the previous
locale (the one recorded in .svn).

So, this would be a big improvement.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Incomplete xml output when using --xml to non-existent server

[Johan Corveleyn]

On Thu, Mar 3, 2016 at 9:53 AM, Bert Huijben  wrote:
>> -Original Message-
>> From: Johan Corveleyn [mailto:jcor...@gmail.com]
...
>> Note that a failure to access a file / directory does *not* give this
>> problem. This incomplete xml only happens when there is a problem
>> accessing the *server* ("nonexistent" in my example).
>
> Did you really try all error conditions of 'svn status --xml' without '-u' ? 
> :)

No, of course not :-). I just gave an example where the output was
broken (host not found), as opposed to another error condition (server
reponds "URL 'X' non-existent in revision Y") where the xml response
is still valid. Ignoring implementation (which I always do when I'm
arguing about behavior), this seems quite weird to me.

I think it's reasonable to expect some consistency here, and
preferably "consistently *valid* output" :-).

It's especially weird in this case, because:

- Broken case: svn client knows *immediately* that something is wrong.
We're asking it to consult a URL, but it can't even contact the
server. Why does it even start to open an xml  element?

- Correct case: agreed, this is much more difficult, because svn is
already talking to the server (and possibly streaming output to the
user), and only finds out partway through that some targets don't
exist on the server (I think that's what happens anyway). But still,
svn gives valid xml output, yay!

-- 
Johan

RE: Incomplete xml output when using --xml to non-existent server

[Bert Huijben]

> -Original Message-
> From: Johan Corveleyn [mailto:jcor...@gmail.com]
> Sent: donderdag 3 maart 2016 09:19
> To: Bert Huijben 
> Cc: Branko Čibej ; Subversion Development
> 
> Subject: Re: Incomplete xml output when using --xml to non-existent server
> 
> On Wed, Mar 2, 2016 at 8:14 PM, Bert Huijben  wrote:
> >
> >
> >> -Original Message-
> >> From: Branko Čibej [mailto:br...@apache.org]
> >> Sent: woensdag 2 maart 2016 19:08
> >> To: dev@subversion.apache.org
> >> Subject: Re: Incomplete xml output when using --xml to non-existent
> server
> >>
> >> On 02.03.2016 15:05, Johan Corveleyn wrote:
> >> > Hi all,
> >> >
> >> > A colleague of mine ran into this, and I'm wondering if it's expected
> >> > behavior or a bug:
> >> >
> >> > For certain commands with the --xml option, the xml output is
> >> > incomplete when sending the request to a non-existent server. This can
> >> > cause an issue for tools that try to parse the output into some data
> >> > structure, and expect correct XML (even though the command has
> exited
> >> > with an error).
> >> >
> >> > For instance, 'svn info' and 'svn ls' behave thusly (maybe other
> >> > commands as well):
> >> >
> >> > [[[
> >> > C:\>svn info --xml https://nonexistent/svn
> >> > 
> >> > 
> >> > svn: E170013: Unable to connect to a repository at URL
> >> 'https://nonexistent/svn'
> >> > svn: E731001: No such host is known.
> >> >
> >> > C:\>svn ls --xml https://nonexistent/svn
> >> > 
> >> > 
> >> >  >> >path="https://nonexistent/svn";>
> >> > svn: E170013: Unable to connect to a repository at URL
> >> 'https://nonexistent/svn'
> >> > svn: E731001: No such host is known.
> >> > ]]]
> >> >
> >> > (The 'svn: Exxx' warnings are printed to stderr, while the incomplete
> >> > xml output is printed to stdout)
> >>
> >> This is clearly a bug; stdout should be either empty, or valid XML.
> >>
> >> In this case I suspect that we're printing the generic XML header too soon.
> >
> > We produce the output streamingly... would you suggest that we now
> buffer all output before producing the first result? (Potentially many
> hundreds of MB on recursive operations, so that would have to be a disk
> backed buffer when the amount of data gets huge)
> >
> > I think I asked the same question somewhere before 1.5, and the answer
> then was +- as I answered now... This is not really fixable in a backwards
> compatible way as there is no way to express errors in the schema. (How do
> we express an error to access a file somewhere deep inside a tree? That
> would make the entire tree invalid as expressed in todays schema)
> >
> 
> Bert,
> 
> Note that a failure to access a file / directory does *not* give this
> problem. This incomplete xml only happens when there is a problem
> accessing the *server* ("nonexistent" in my example).

Did you really try all error conditions of 'svn status --xml' without '-u' ? :)

There are hundreds of cases (if not far more) where common svn operations can 
fail during an operation. Each will break the xml output.

I don't think it is valuable to fix just a few specific cases if we can’t 
promise that it works in general.


'svn ls' is a 100% server side operation, so usually errors reported by this 
will be related to auth and/or network problems, but svn info, svn status, svn 
diff, etc. that also have --xml all have the same problem.

Bert

Re: Incomplete xml output when using --xml to non-existent server

[Johan Corveleyn]

On Wed, Mar 2, 2016 at 8:14 PM, Bert Huijben  wrote:
>
>
>> -Original Message-
>> From: Branko Čibej [mailto:br...@apache.org]
>> Sent: woensdag 2 maart 2016 19:08
>> To: dev@subversion.apache.org
>> Subject: Re: Incomplete xml output when using --xml to non-existent server
>>
>> On 02.03.2016 15:05, Johan Corveleyn wrote:
>> > Hi all,
>> >
>> > A colleague of mine ran into this, and I'm wondering if it's expected
>> > behavior or a bug:
>> >
>> > For certain commands with the --xml option, the xml output is
>> > incomplete when sending the request to a non-existent server. This can
>> > cause an issue for tools that try to parse the output into some data
>> > structure, and expect correct XML (even though the command has exited
>> > with an error).
>> >
>> > For instance, 'svn info' and 'svn ls' behave thusly (maybe other
>> > commands as well):
>> >
>> > [[[
>> > C:\>svn info --xml https://nonexistent/svn
>> > 
>> > 
>> > svn: E170013: Unable to connect to a repository at URL
>> 'https://nonexistent/svn'
>> > svn: E731001: No such host is known.
>> >
>> > C:\>svn ls --xml https://nonexistent/svn
>> > 
>> > 
>> > > >path="https://nonexistent/svn";>
>> > svn: E170013: Unable to connect to a repository at URL
>> 'https://nonexistent/svn'
>> > svn: E731001: No such host is known.
>> > ]]]
>> >
>> > (The 'svn: Exxx' warnings are printed to stderr, while the incomplete
>> > xml output is printed to stdout)
>>
>> This is clearly a bug; stdout should be either empty, or valid XML.
>>
>> In this case I suspect that we're printing the generic XML header too soon.
>
> We produce the output streamingly... would you suggest that we now buffer all 
> output before producing the first result? (Potentially many hundreds of MB on 
> recursive operations, so that would have to be a disk backed buffer when the 
> amount of data gets huge)
>
> I think I asked the same question somewhere before 1.5, and the answer then 
> was +- as I answered now... This is not really fixable in a backwards 
> compatible way as there is no way to express errors in the schema. (How do we 
> express an error to access a file somewhere deep inside a tree? That would 
> make the entire tree invalid as expressed in todays schema)
>

Bert,

Note that a failure to access a file / directory does *not* give this
problem. This incomplete xml only happens when there is a problem
accessing the *server* ("nonexistent" in my example).

As I said in the second part of my mail:

> Contrast this with another failing request, when accessing a
> *non-existent directory* on an existing server:
>
> [[[
> C:\>svn info --xml https://realserver/svn/asdf
> 
> 
> svn: warning: W17: URL 'https://realserver/svn/asdf' non-existent
> in revision 316895
>
> 
> svn: E29: Could not display info for all targets because some
> targets don't exist
>
> C:\>svn ls --xml https://realserver/svn/asdf
> 
> 
> path="https://realserver/svn/asdf";>
> svn: warning: W160013: URL 'https://realserver/svn/asdf' non-existent
> in revision 316895
> 
> 
> svn: E29: Could not list all targets because some targets don't exist
> ]]]

Accessing a non-existent directory on an existing server does yield correct xml.

-- 
Johan