Re: [fossil-users] Wildcards not working on windows

2011-09-29 Thread Jan Danielsson
On 09/26/11 15:02, Richard Hipp wrote: >> Richard seem to have released 1.19 binary for Windows compiled with Visual >> Studio, while 1.18 >> has been compiled with MinGW. >> >> > Correct. I didn't realized there was a difference. > > Should I make a point of always building future Fossil releas

Re: [fossil-users] possible bug + performance hit in timeline_cmd()

2011-09-29 Thread Richard Hipp
On Thu, Sep 29, 2011 at 2:52 PM, Stephan Beal wrote: > Hiya, core dev(s), > > In timeline_cmd(), is there a reason that: > > db_find_and_open_repository(0, 0); > showfilesFlag = find_option("showfiles","f", 0)!=0; > db_find_and_open_repository(0, 0); > > db_find_and_open_repository() is cal

Re: [fossil-users] Protection against timing attacks

2011-09-29 Thread Stephan Beal
2011/9/29 Lluís Batlle i Rossell > Well, if someone has studied those attacks, and the code by dmitry looks > fine > and works based on some heavier studies than our first thoughts on that, > let's > use it. No? > i can't object, i just think it's paranoid :). -- - stephan beal http://wand

Re: [fossil-users] Protection against timing attacks

2011-09-29 Thread Stephan Beal
On Thu, Sep 29, 2011 at 8:57 PM, Dmitry Chestnykh wrote: > I posted a link about this concern: > http://rdist.root.org/2010/01/07/timing-independent-array-comparison/ So why not simply add the following logic to server mode: A) fetch config option "add-random-sleep" (integer, default=0) B) if (

Re: [fossil-users] Protection against timing attacks

2011-09-29 Thread Dmitry Chestnykh
> So why not simply add the following logic to server mode: > > A) fetch config option "add-random-sleep" (integer, default=0) > B) if ((A)>0) AND user is nobody, sleep for random 1..(A) ms. (This attack > would seem to be useless for anyone but the nobody user. If you're logged in, > you've got

Re: [fossil-users] Protection against timing attacks

2011-09-29 Thread Lluís Batlle i Rossell
On Thu, Sep 29, 2011 at 09:07:37PM +0200, Stephan Beal wrote: > On Thu, Sep 29, 2011 at 8:57 PM, Dmitry Chestnykh > wrote: > > > I posted a link about this concern: > > http://rdist.root.org/2010/01/07/timing-independent-array-comparison/ > > > So why not simply add the following logic to server

Re: [fossil-users] Protection against timing attacks

2011-09-29 Thread Dmitry Chestnykh
On Sep 29, 2011, at 20:15 , Stephan Beal wrote: > Given the relatively high overhead fossil has when it opens a db or runs a > query, and network latency, i cannot imagine that someone could accurately > time the difference of a memcmp() operation on 8 or 10 bytes. The number of > factors invol

Re: [fossil-users] Wanted: adventurous JSON fans to take a test spin

2011-09-29 Thread Stephan Beal
On Thu, Sep 29, 2011 at 8:45 PM, David Bovill wrote: > Thanks for this - I'm definitely going to need it, and will try to make > time to do some tests next week :) > If you're looking for a staring point/demo, see ajax/index.html and friends. Any and all feedback is welcomed. Very few details (

[fossil-users] possible bug + performance hit in timeline_cmd()

2011-09-29 Thread Stephan Beal
Hiya, core dev(s), In timeline_cmd(), is there a reason that: db_find_and_open_repository(0, 0); showfilesFlag = find_option("showfiles","f", 0)!=0; db_find_and_open_repository(0, 0); db_find_and_open_repository() is called twice, or is that a mistake? That can cause db_verify_schema() to

Re: [fossil-users] Wanted: adventurous JSON fans to take a test spin

2011-09-29 Thread David Bovill
Thanks for this - I'm definitely going to need it, and will try to make time to do some tests next week :) On 27 September 2011 06:12, Stephan Beal wrote: > Hi, all! > > With almost 90kb of new functionality and 21 pages of "draft spec" docs[1], > the JSON branch has reached a point where some o

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Jeff Slutter
On 9/29/2011 2:12 PM, Joerg Sonnenberger wrote: What Operating System is that on? There might be a limit to the number of filesystem objects that can be cached and your tree just large enough to not fit into it. Another thing to try is forcing the _FOSSIL_ file into cache (e.g. cat _FOSSIL_> /d

Re: [fossil-users] Protection against timing attacks

2011-09-29 Thread Lluís Batlle i Rossell
On Thu, Sep 29, 2011 at 08:15:55PM +0200, Stephan Beal wrote: > On Thu, Sep 29, 2011 at 8:00 PM, Dmitry Chestnykh > wrote: > > > The more eyes the better, as it touches login code. > > ...COMPARE("A", "PASSWORD") returns FALSE in 0.1 msec, but > > COMPARE("P", "PASSWORD") returns F

Re: [fossil-users] Protection against timing attacks

2011-09-29 Thread Stephan Beal
On Thu, Sep 29, 2011 at 8:00 PM, Dmitry Chestnykh wrote: > The more eyes the better, as it touches login code. > ...COMPARE("A", "PASSWORD") returns FALSE in 0.1 msec, but > COMPARE("P", "PASSWORD") returns FALSE in 0.3 msec, because it did > two comparisons: > Given the relativel

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Joerg Sonnenberger
On Thu, Sep 29, 2011 at 11:31:19AM -0400, Jeff Slutter wrote: > There seems to be a minimum time of 6 seconds for my operations of > status, changes, and commit, and it would make sense that they all > have to do the same work at some point (that would be 'finding out > what files have changed') W

[fossil-users] Protection against timing attacks

2011-09-29 Thread Dmitry Chestnykh
Dear archeologists, Please review changes in the following branch: http://www.fossil-scm.org/index.html/timeline?r=dmitry-security The more eyes the better, as it touches login code. I try to protect against timing attacks on login, cookies, and sync by using the constant-time comparison funct

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Stephan Beal
On Thu, Sep 29, 2011 at 5:31 PM, Jeff Slutter wrote: > I don't know if that "6 seconds" can be improved on, but I am definitely > much happier than I was yesterday. > We all love success stories! Keep 'em coming! :) And thanks for having the patience to try to get to the bottom of the problem,

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Jeff Slutter
Some good news... I came in to work, disabled repo-cksum, on the copy of the repository at work and tested again. Single file commit took 6 seconds. I made a number of changes to files (11 files total, a collection of edits, adds and removes) and did a fossil commit (without specifying files o

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Stephan Beal
On Thu, Sep 29, 2011 at 3:18 PM, Jeff Slutter wrote: > Add was sub 1 second > Commit took 59 seconds > A few weeks ago someone posted about horrible performance in his BSD Ports repo - many tens of thousands of files. Richard explained (though i cannot find the post at the moment) something abou

Re: [fossil-users] Square brackets in filenames

2011-09-29 Thread Benoit Mortgat
Actually, FAT and NTFS both support brackets in file names. I ran into this issue of brackets when trying to version uncompressed Office 2007 documents (docx is just a zip that contains xml files, including a [Content-Types].xml at the root). On Thu, Sep 29, 2011 at 13:43, Richard Hipp wrote: >

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Jeff Slutter
On 9/29/2011 5:23 AM, Stephan Beal wrote: > > This _might_ be relevant, might not: > > (from within checkout folder root, on the Windows machine) fossil set repo-cksum off fossil set mtime-changes on dir * /s > test.txt fossil add test.txt fossil commit -m "test" test.txt Add was sub 1 second Com

Re: [fossil-users] Square brackets in filenames

2011-09-29 Thread Richard Hipp
On Thu, Sep 29, 2011 at 5:27 AM, Dmitry Chestnykh wrote: > What's the rationale for disallowing [ and ] in filenames? Are there any > systems that don't like them (maybe FAT)? Or maybe this is due to collision > with [wiki/ci links]? > Special characters like this are a minefield of potential bug

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Dmitry Chestnykh
> To open the repository to a new checkout it took Fossil about 26 > minutes. Roughly 13 minutes extracting the files into the directory, and > then 13 minutes of ... doing something, before it came back. > The equivalent command in Mercurial ("hg update null" to reset the > checkout then the timed

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Dmitry Chestnykh
On Sep 29, 2011, at 5:49 AM, Jeff Slutter wrote: > Initial 'add' of all the files took roughly 10 minutes. For comparison, > Mercurial took less than a minute. Let's start with this and work on low-hanging fruits. I have created ~7 tiny files (no directories). Adding all files takes 16 sec (

[fossil-users] Square brackets in filenames

2011-09-29 Thread Dmitry Chestnykh
What's the rationale for disallowing [ and ] in filenames? Are there any systems that don't like them (maybe FAT)? Or maybe this is due to collision with [wiki/ci links]? >From file_is_simple_pathname(const char *z): ** * Does not contain any of these characters in the path: "\*[]?" Brac

Re: [fossil-users] is a rebuild in http mode possible?

2011-09-29 Thread Stephan Beal
On Thu, Sep 29, 2011 at 1:40 AM, Richard Hipp wrote: > The Rebuild button on the shunning page > http://www.fossil-scm.org/fossil/artifact/4361efe5c919b1?ln=160 already > does this. > Great! i've never shunned, so i never saw that. Thanks :) -- - stephan beal http://wanderinghorse.net/hom

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Stephan Beal
On Thu, Sep 29, 2011 at 6:16 AM, Jeff Slutter wrote: > Interesting... > > I failed to mention in my post that my version of fossil was from 'trunk' > sometime this afternoon, build with MSVC 2008. I also made one minor change > to fix handling for repos > 2gig (MSVC build version only...patch wa

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Konstantin Khomoutov
On Thu, 29 Sep 2011 08:15:15 + ala...@snell-pym.org.uk wrote: > Just a thought - is there some virus-scanning software involved, that > feels a need to scan every file opened? The OP got the same results on Ubuntu which supposedy is not infested with antivirus software. ___

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread alaric
Just a thought - is there some virus-scanning software involved, that feels a need to scan every file opened? Sent from my BlackBerry® wireless device -Original Message- From: Lluís Batlle i Rossell Sender: fossil-users-boun...@lists.fossil-scm.org Date: Thu, 29 Sep 2011 09:55:45 To:

Re: [fossil-users] is a rebuild in http mode possible?

2011-09-29 Thread Dmitry Chestnykh
On Sep 29, 2011, at 1:40 AM, Richard Hipp wrote: > In the past there have been issues with output that rebuild sometimes > generates. I haven't pressed that button lately, so I don't know if those > issues are currently fixed or not. As of [2cd21f8dc2] (trunk: [ce354d0a9f]) there shouldn't be

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Lluís Batlle i Rossell
On Thu, Sep 29, 2011 at 01:45:29PM +0800, mjbmik...@gmail.com wrote: > It is the Windows version. > > I'm currently in the process of commiting a new 0 byte file to an existing > 2GB repo and Windows task manager says that the fossil process has read >3GB > of data since I issued the commit comm

Re: [fossil-users] Scalability, a single file commit and lots of disk reads

2011-09-29 Thread Jeff Slutter
I downloaded the Windows 1.18 version (supposedly build with mingw) from the website and tested it getting the same results as my previous post's timings ('delete' mode and 'wal' mode). Using Sysinternals' Process Monitor it was clear that fossil was reading all the way through the repository seve