Re: Efficiency of rep-sharing (deduplication) in 1.8 and later (chunking?)
> > Representation cache is based on the sha of the rep. So it does not > matter what the filename is or where it is stored. If it has the same > sha as an existing rep, then it will be be shared. > > The small improvement in 1.8 was simply to do this for files being added > within the same revision, but the other scenario was already supported. > > I think it is worth pointing out that a rep is not necessarily a "file". > It is the specific delta that SVN would be storing in the repository DB. > One improvement that I'd like to suggest is that files over 1MiB (4? 8?) be "chunked" prior to calculating rep-sharing. http://blog.clearpathsg.com/blog/bid/254076/Understanding-Variable-Length-Deduplication My thinking is that there might be storage gains to be made if rep-sharing is done at a lower level then the file level in cases of files over a particular size. For instance, if you commit a few hundred files of mid-size (5-15MB or larger), there is probably a lot of identical data between them (if the files are not already compressed). Those identical chunks could be possibly found via a variable length deduplication algorithm and deduped across the repository. IIRC when I moved our repos from 1.6 to 1.8 format, space usage went down by 10-15% from rep-sharing. I wouldn't mind having another 5-10% space savings.
Re: Problems with selecting log revision by date: -r {yyyy-mm-dd}
On Wed, Dec 3, 2014 at 2:45 PM, Philip Martin
wrote:
> Stefan Sperling writes:
>
>> On Wed, Dec 03, 2014 at 12:22:02PM +, sebb wrote:
>>>
>>> svn co --depth files https://svn.apache.org/repos/asf/subversion/trunk
>>> subversion
>>>
>>> $ svn log -l 10 subversion
>>> -- this works OK
>>>
>>> $ svn log -r {2014-11-30} subversion
>>>
>>> $ svn log -r {2014-11-30T00:00:00} subversion
>>>
>>
>> Date search doesn't work on the ASF repository because svn:date properties
>> of revisions aren't monotonically increasing.
>>
>> See the note at the very bottom of
>> http://svnbook.red-bean.com/en/1.7/svn.tour.revs.specifiers.html#svn.tour.revs.dates
>>
>
> That's correct, but it is not the complete story. The date search does
> return a revision even if that revision is not "correct" is some sense.
> However the date search works on the whole repository
>
>https://svn.apache.org/repos/asf
>
> so the revision produced will often be one which didn't change
> /subversion/trunk. That means that an empty log is shown just like
> it is for this
>
>svn log -r1643098 https://svn.apache.org/repos/asf/subversion/trunk
>
> Even when a repository does have strictly mototonic dates using log with
> a single date will often show a blank log on non-root paths within the
> repository.
>
So perhaps the user should have specified a range, instead of just one date?
Like:
$ svn log -r {2014-11-30}:HEAD subversion
or:
$ svn log -r {2014-11-30}:{2014-11-31} subversion
--
Johan
Re: Tunnel Agent
Hi Bert, Thanks for the response. On Wed, Dec 3, 2014 at 5:35 PM, Bert Huijben wrote: > Hi, > > > > Subversion 1.9 is not released yet, so the api may change before we > release 1.9. > > > > But if you want to test the early code there are some testcases you might > want to look at in > > > http://svn.apache.org/repos/asf/subversion/trunk/subversion/tests/libsvn_ra/ra-test.c > > (for the C api) > > > > Or: > > > http://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings/javahl/tests/org/apache/subversion/javahl/BasicTests.java > > (for the JavaHL api) > > > > Outside the Subversion source SharpSvn’s trunk contains some code that > uses the api to implement svn+ssh:// using libssh2. > > > > Bert > > > > *From:* Rama A [mailto:altodevelopers.r...@gmail.com] > *Sent:* woensdag 3 december 2014 05:09 > *To:* users@subversion.apache.org > *Subject:* Tunnel Agent > > > > Hi, > > I'm in a plan to use Subversion 1.9. While glance through the Subversion > 1.9 API, I have noticed that it contains TunnelAgent to create the tunnel > and run the subversion commands over there. Also it contains the method > openTunnel in TunnelAgent. > > I'm new to this concept. So Can you please clarify me how to set the > tunnel agent in my application, open and run the command in it. > > Thanks, > > Rama >
Re: Problems with selecting log revision by date: -r {yyyy-mm-dd}
Stefan Sperling writes:
> On Wed, Dec 03, 2014 at 12:22:02PM +, sebb wrote:
>>
>> svn co --depth files https://svn.apache.org/repos/asf/subversion/trunk
>> subversion
>>
>> $ svn log -l 10 subversion
>> -- this works OK
>>
>> $ svn log -r {2014-11-30} subversion
>>
>> $ svn log -r {2014-11-30T00:00:00} subversion
>>
>
> Date search doesn't work on the ASF repository because svn:date properties
> of revisions aren't monotonically increasing.
>
> See the note at the very bottom of
> http://svnbook.red-bean.com/en/1.7/svn.tour.revs.specifiers.html#svn.tour.revs.dates
>
That's correct, but it is not the complete story. The date search does
return a revision even if that revision is not "correct" is some sense.
However the date search works on the whole repository
https://svn.apache.org/repos/asf
so the revision produced will often be one which didn't change
/subversion/trunk. That means that an empty log is shown just like
it is for this
svn log -r1643098 https://svn.apache.org/repos/asf/subversion/trunk
Even when a repository does have strictly mototonic dates using log with
a single date will often show a blank log on non-root paths within the
repository.
--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
Re: Problems with selecting log revision by date: -r {yyyy-mm-dd}
On 3 December 2014 at 13:00, Stefan Sperling wrote:
> On Wed, Dec 03, 2014 at 12:22:02PM +, sebb wrote:
>> I cannot get the examples in the manual to work.
>> They don't produce any output apart from the separator.
>>
>> Is this a known bug?
>>
>> How to reproduce:
>>
>> svn co --depth files https://svn.apache.org/repos/asf/subversion/trunk
>> subversion
>>
>> $ svn log -l 10 subversion
>> -- this works OK
>>
>> $ svn log -r {2014-11-30} subversion
>>
>> $ svn log -r {2014-11-30T00:00:00} subversion
>>
>>
>> This was tried on minotaur, using
>>
>> svn, version 1.7.9 (r1462340)
>>compiled Jun 3 2013, 11:33:48
>>
>> I have the same problems with the following version
>>
>> svn, version 1.8.10 (r1615264)
>>compiled Aug 11 2014, 11:33:45 on x86_64-apple-darwin13.0.0
>>
>> =
>>
>> There's also a minor documentation error, the manual says:
>>
>> If you want to include the 27th in your search, you can either specify
>> the 27th with the time ({"2006-11-27 23:59"}), or just specify the
>> next day ({2006-11-28}).
>>
>> The phrase "the next day" should be "the previous day" or "the day before"
>
> Date search doesn't work on the ASF repository because svn:date properties
> of revisions aren't monotonically increasing.
>
> See the note at the very bottom of
> http://svnbook.red-bean.com/en/1.7/svn.tour.revs.specifiers.html#svn.tour.revs.dates
This says:
"... If this ordering isn't maintained, you will likely find that
trying to use dates to specify revision ranges in your repository
doesn't always return the data you might have expected."
Although this may be strictly true, it's not immediately obvious that
nothing might be returned.
It would be helpful if the docs made it a lot clearer that date
selection may not work at all.
This should be stated at the start of the section, rather than a note
at the bottom.
Is the ASF repo unusual in not having monotonic dates? Or is this a
common occurrence (as I suspect)?
Further, it would help if SVN returned some indication that dates
cannot be used on the particular repository.
Though ideally of course it would be nice if the feature worked across
all SVN installations.
Re: Problems with selecting log revision by date: -r {yyyy-mm-dd}
On Wed, Dec 03, 2014 at 12:22:02PM +, sebb wrote:
> I cannot get the examples in the manual to work.
> They don't produce any output apart from the separator.
>
> Is this a known bug?
>
> How to reproduce:
>
> svn co --depth files https://svn.apache.org/repos/asf/subversion/trunk
> subversion
>
> $ svn log -l 10 subversion
> -- this works OK
>
> $ svn log -r {2014-11-30} subversion
>
> $ svn log -r {2014-11-30T00:00:00} subversion
>
>
> This was tried on minotaur, using
>
> svn, version 1.7.9 (r1462340)
>compiled Jun 3 2013, 11:33:48
>
> I have the same problems with the following version
>
> svn, version 1.8.10 (r1615264)
>compiled Aug 11 2014, 11:33:45 on x86_64-apple-darwin13.0.0
>
> =
>
> There's also a minor documentation error, the manual says:
>
> If you want to include the 27th in your search, you can either specify
> the 27th with the time ({"2006-11-27 23:59"}), or just specify the
> next day ({2006-11-28}).
>
> The phrase "the next day" should be "the previous day" or "the day before"
Date search doesn't work on the ASF repository because svn:date properties
of revisions aren't monotonically increasing.
See the note at the very bottom of
http://svnbook.red-bean.com/en/1.7/svn.tour.revs.specifiers.html#svn.tour.revs.dates
Problems with selecting log revision by date: -r {yyyy-mm-dd}
I cannot get the examples in the manual to work.
They don't produce any output apart from the separator.
Is this a known bug?
How to reproduce:
svn co --depth files https://svn.apache.org/repos/asf/subversion/trunk
subversion
$ svn log -l 10 subversion
-- this works OK
$ svn log -r {2014-11-30} subversion
$ svn log -r {2014-11-30T00:00:00} subversion
This was tried on minotaur, using
svn, version 1.7.9 (r1462340)
compiled Jun 3 2013, 11:33:48
I have the same problems with the following version
svn, version 1.8.10 (r1615264)
compiled Aug 11 2014, 11:33:45 on x86_64-apple-darwin13.0.0
=
There's also a minor documentation error, the manual says:
If you want to include the 27th in your search, you can either specify
the 27th with the time ({"2006-11-27 23:59"}), or just specify the
next day ({2006-11-28}).
The phrase "the next day" should be "the previous day" or "the day before"
RE: Tunnel Agent
Hi, Subversion 1.9 is not released yet, so the api may change before we release 1.9. But if you want to test the early code there are some testcases you might want to look at in http://svn.apache.org/repos/asf/subversion/trunk/subversion/tests/libsvn_ra/ra-test.c (for the C api) Or: http://svn.apache.org/repos/asf/subversion/trunk/subversion/bindings/javahl/tests/org/apache/subversion/javahl/BasicTests.java (for the JavaHL api) Outside the Subversion source SharpSvn’s trunk contains some code that uses the api to implement svn+ssh:// using libssh2. Bert From: Rama A [mailto:altodevelopers.r...@gmail.com] Sent: woensdag 3 december 2014 05:09 To: users@subversion.apache.org Subject: Tunnel Agent Hi, I'm in a plan to use Subversion 1.9. While glance through the Subversion 1.9 API, I have noticed that it contains TunnelAgent to create the tunnel and run the subversion commands over there. Also it contains the method openTunnel in TunnelAgent. I'm new to this concept. So Can you please clarify me how to set the tunnel agent in my application, open and run the command in it. Thanks, Rama
