[darcs-devel] patch: make get -q quieter

[Simon Marlow]

patch
Description: Binary data
___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] [issue124] performance of darcs changes

[Simon Marlow]

New submission from Simon Marlow <[EMAIL PROTECTED]>:

On a repository with a lot of patches, asking for darcs changes on a particular
file or directory can take a *long* time (30+ seconds of CPU time is common in
the GHC repository).

This is preventing us from using the darcs browser in Trac, because it
automatically runs darcs changes in this way.  Even the darcsweb browser isn't
really usable with GHC.

If I could wish for only one thing in darcs, faster 'darcs changes ' would
be it.  I imagine that in order to do this you have to store the files changed
along with each patch in the inventory, but once you have this a lot of other
operations could be sped up too (darcs annotate would be my #2 wish).  Also,
wouldn't this let you do 'darcs changes ' on a partial repository, as long
as you don't also say -v?

--
messages: 456
nosy: droundy, simonmar, tommy
priority: wishlist
status: unread
title: performance of darcs changes 


Darcs issue tracker <[EMAIL PROTECTED]>
<http://bugs.darcs.net/issue124>


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] [issue190] crash in function new_ur

[Simon Marlow]

New submission from Simon Marlow <[EMAIL PROTECTED]>:

We can supply failing repositories on request.

sh-2.04$ darcs --exact-version
darcs compiled on May 30 2005, at 11:54:13
# configured Mon May 30 11:48:37 USMST 2005
sh ./configure --disable-mmap --target=mingw

Context:

[update web page to reflect new stable release (1.0.3)
Tomasz Zielonka <[EMAIL PROTECTED]>**20050524225643] 

[TAG 1.0.3
Tomasz Zielonka <[EMAIL PROTECTED]>**20050524215127] 
sh-2.04$ darcs pull http://darcs.haskell.org/ghc
This is the GHC darcs repostory (HEAD branch)

For more information, visit the GHC developer wiki at
  http://hackage.haskell.org/trac/ghc
**

Wed May 17 16:42:04 GMT Daylight Time 2006  [EMAIL PROTECTED]
  * Improve pretty-printing slightly
Shall I pull this patch? (1/94) [ynWvpxqadjk], or ? for help: y


Wed May 17 16:43:04 GMT Daylight Time 2006  [EMAIL PROTECTED]
  * Comments only
Shall I pull this patch? (2/94) [ynWvpxqadjk], or ? for help: y


Wed May 17 16:43:49 GMT Daylight Time 2006  [EMAIL PROTECTED]
  * Improve pretty-printing
Shall I pull this patch? (3/94) [ynWvpxqadjk], or ? for help: y


Wed May 17 16:44:49 GMT Daylight Time 2006  [EMAIL PROTECTED]
  * Retain INLINE pragma information during indirection-shorting
  
  During indirection-shorting, we were dropping the InlinePragInfo,
  although were were carefully retaining strictness info etc.  
  I think this is a long-standing bug.
  
Shall I pull this patch? (4/94) [ynWvpxqadjk], or ? for help: y


Wed May 17 16:47:10 GMT Daylight Time 2006  [EMAIL PROTECTED]
  * Spelling correction
Shall I pull this patch? (5/94) [ynWvpxqadjk], or ? for help: a

darcs.exe: bug in darcs!
in function new_ur
Original patch:
merger 0.9 (
merger 0.0 (
hunk ./compiler/iface/IfaceSyn.lhs 633
--- gaw 2004
merger 0.0 (
hunk ./compiler/iface/IfaceSyn.lhs 589
-rhs= unfoldingTemplate unfold_info
-unfold_hsinfo |  neverUnfold unfold_info   -- The CoreTidy phase retains
unfolding info iff
- || has_worker = Nothing   -- we want to expose the 
unfolding, taking into account
-   -- unconditional NOINLINE, etc. 
 See TidyPgm.addExternal
- | otherwise   = Just (HsUnfold inline_prag (toIfaceExpr ext 
rhs))
+inline_hsinfo | isAlwaysActive inline_prag = Nothing
+ | no_unfolding && not has_worker = Nothing
+   -- If the iface file give no unfolding info, we 
+   -- don't need to say when inlining is OK!
+ | otherwise  = Just (HsInline inline_prag)
merger 0.0 (
hunk ./compiler/iface/IfaceSyn.lhs 577
-unfold_info = unfoldingInfo id_info
+unfold_info  = unfoldingInfo id_info
+rhs = unfoldingTemplate unfold_info
+no_unfolding = neverUnfold unfold_info
+   -- The CoreTidy phase retains unfolding info iff
+   -- we want to expose the unfolding, taking into account
+   -- unconditional NOINLINE, etc.  See TidyPgm.addExternal
+unfold_hsinfo | no_unfolding = Nothing 
+ | has_worker   = Nothing  -- Unfolding is implicit
+ | otherwise= Just (HsUnfold (toIfaceExpr ext rhs))
+   
+  Inline prag  --
merger 0.0 (
hunk ./compiler/iface/IfaceSyn.lhs 548
-  wrkr_hsinfo,  unfold_hsinfo] 
+  inline_hsinfo, wrkr_hsinfo,  unfold_hsinfo] 
merger 0.0 (
hunk ./compiler/iface/IfaceSyn.lhs 580
-unfold_hsinfo |  neverUnfold unfold_info 
- || has_worker = Nothing
+unfold_hsinfo |  neverUnfold unfold_info   -- The CoreTidy phase retains
unfolding info iff
+ || has_worker = Nothing   -- we want to expose the 
unfolding, taking into account
+   -- unconditional NOINLINE, etc. 
 See TidyPgm.addExternal
hunk ./compiler/iface/IfaceSyn.lhs 417
-\end{code}
-
-
-%
-%* *
-   Converting things to their Iface equivalents
-%* *
-%
-
-
-\begin{code}
-tyThingToIfaceDecl :: (Name -> IfaceExtName) -> TyThing -> IfaceDecl
--- Assumption: the thing is already tidied, so that locally-bound names
---(lambdas, for-alls) already have non-clashing OccNames
--- Reason: Iface stuff uses OccNames, and the conversion here does
---not do tidying on the way
-tyThingToIfaceDecl ext (AnId id)
-  = IfaceId { ifName   = getOccName id, 
- ifType   = toIfaceType ext (idType id),
- ifIdInfo = info }
-  where
-info

[darcs-devel] Darcs crashes on GHC repository

[Simon Marlow]

[ posting on behalf of [EMAIL PROTECTED], who tried to post the 
message earlier but it doesn't seem to have shown up, so here it is, 
just in case it got swallowed.  Also Simon submitted bugs to 
[EMAIL PROTECTED], which appear to have been lost, so I created tickets 
manually. ]


Dear Darcs developers

As you know, we switched GHC to Darcs some while ago, which makes Darcs 
absolutely mission-critical to GHC.  But I've encountered two serious 
Darcs crashes recently (two different internal consistency errors not 
seg-faults).  I reported the first to [EMAIL PROTECTED], but had no 
response.  I managed to work around it by abandoning that repository 
altogether, and manually applying patches to a fresh copy.  But it was 
scary.


The second one is only today; I've just sent off a bug report to the 
address above.  I would really appreciate advice about how to work 
around it.


The situation (in both cases) couldn't be simpler. I'm working on a 
variant of GHC.  I have accumulated a bunch of patches in my variant 
repository.  I wanted to pull patches from the main repository, but 
Darcs crashed.  'darcs check' reports that the repository is consistent.


I'd *really* appreciate some help with this.  Please.  Both are 100% 
reproducible, and are 100% show-stoppers.   Thanks!


Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: [darcs-conflicts] alternate (simplified) conflicts proposal.

[Simon Marlow]

I must admit, this formulation looked initially attractive, but having read up 
the dicsussion on IRC and the rest of this thread I now have some misgivings.


The problem (that I don't think was mentioned on the list, but was described by 
Igloo on IRC) is that sometimes two resolutions that should conflict don't.  eg. 
if A & B conflict, one developer resolves the conflict by killing A, the other 
by killing B, then the merge of the two repos kills both A & B.


To me, this illustrates a fundamental problem with the approach, and there's an 
easy way to describe why.  The resolution of a conflict is dependent on the 
conflicting changes: that is, it depends on both lines of development, not just 
the one beikng killed.  A resolution should say "I want to resolve these 
conflicting lines of development in the following way".  A resolution that just 
says "I want to kill these patches" doesn't express enough context, which leads 
to the problem above.


So to me, this idea seems just a little *too* simple.  But maybe there's an 
elboration that isn't too complicated?


Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] darcs patch: use System.Process on Windows, and a few more

[Simon Marlow]

Apologies for the attachment, I don't have a machine I can 'darcs send' from 
handy.

1.0.9rc2 seems to be working reasonably well on Windows.  For a while I had to 
use --disable-ssh-cm, but for some reason that problem went during the course of 
today, I'm not sure why.  Does control master work for anyone on Windows?


Cheers,
Simon

New patches:

[add explicit import list
Simon Marlow <[EMAIL PROTECTED]>**20061129160144] {
hunk ./AmendRecord.lhs 22
-import System
+import System.Exit ( ExitCode(..), exitWith )
}

[hFlush after "waiting for lock" message
Simon Marlow <[EMAIL PROTECTED]>**20061129160342
 On Windows, stdout isn't always in line-buffered mode, but we really
 want to see the message about waiting for a lock quickly.  Mostly
 because ^C isn't always caught properly on Windows and lock files are
 often left behind, but that's another storey...
 
] {
hunk ./Lock.lhs 39
-   IOMode(WriteMode, AppendMode) )
+   IOMode(WriteMode, AppendMode), hFlush, stdout )
hunk ./Lock.lhs 86
+  hFlush stdout -- for Windows
}

[catch exceptions in stdout_is_a_pipe
Simon Marlow <[EMAIL PROTECTED]>**20061129160620] {
hunk ./Compat.hs 33
+import Control.Exception (catchJust, ioErrors)
hunk ./Compat.hs 37
- = do stat <- getFdStatus stdOutput
-  return (isNamedPipe stat)
+ = catchJust ioErrors
+(do stat <- getFdStatus stdOutput
+return (isNamedPipe stat))
+    (\_ -> return False)
}

[Use System.Process on Windows
Simon Marlow <[EMAIL PROTECTED]>**20061129160710
 
 This was an attempt to address "[issue218] Patch bundle failed hash",
 but unfortunately it doesn't fix the problem.  Still, System.Process
 is a better way to invoke external commands these days.
 
 For now, the new code only replaces the Windows version of exec.  This
 means that GHC 6.4 will be required to build darcs on Windows.  Better
 would be to add a configure test, but I ran out of time here.
] {
hunk ./Exec.lhs 25
-import System
-import System.Cmd (rawSystem)
-import IO
+import System.Exit  ( ExitCode )
+import System.Cmd   ( rawSystem )
+import Control.Monad( liftM )
+
+#ifdef WIN32
hunk ./Exec.lhs 31
-#ifndef WIN32
+import System.Process
+import System.IO   ( IOMode(..), openBinaryFile, stdout )
+import Control.Exception ( bracketOnError )
+import GHC.Handle  ( hDuplicate )
+-- urgh.  hDuplicate isn't available from a standard place.
hunk ./Exec.lhs 37
+#else
+
+import System
hunk ./Exec.lhs 43
-import Monad ( liftM )
hunk ./Exec.lhs 73
+  deriving Show
hunk ./Exec.lhs 80
-  On Windows we call the system function with a command line
-  string. The string has the arguments in quotes, and contains
-  redirection operators.
+  On Windows we use System.Process, which does the necessary quoting
+  and redirection for us behind the scenes.  In the future, we can do
+  it this way on Unix too (we currently require to compile with GHC
+  6.2 on Unix, and 6.2 didn't have System.Process).
hunk ./Exec.lhs 86
-exec cmd args (inp,out,err) =
-  system $ cmd ++ " " ++ in_quotes_unwords args
-   ++ (redirect "<"  inp)
-   ++ (redirect ">"  out)
-   ++ (redirect "2>" err) -- order is important if err is Stdout
-  where redirect op value =
-  case value of
--- FIXME: are all these spaces necessary?
-AsIs  -> ""
-Null  -> " " ++ op ++ " " ++ "NUL"
-File "/dev/null" ->  -- safety catch
- " " ++ op ++ " " ++ "NUL"
-File fp   -> " " ++ op ++ " \"" ++ fp ++ "\"" -- fp in quotes
-Stdout-> " " ++ op ++ "&1"
-
-in_quotes_unwords :: [String] -> [Char]
-in_quotes_unwords (a:as) = "\""++a++"\" "++ in_quotes_unwords as
-in_quotes_unwords [] = ""
+exec cmd args (inp,out,err) = do
+  h_stdin  <- redirect inp ReadMode
+  h_stdout <- redirect out WriteMode
+  h_stderr <- redirect err WriteMode
+--  putStrLn (unwords (cmd:args ++ map show [inp,out,err]))
+  bracketOnError
+(runProcess cmd args Nothing Nothing h_stdin h_stdout h_stderr)
+(terminateProcess)
+(waitForProcess)
+  where
+redirect AsIs   _= return Nothing
+redirect Null   mode = Just `liftM` openBinaryFile "NUL" mode
+redirect (File "/dev/null") mode = redirect Null mode
+redirect (File f)   mode = Just `liftM` openBinaryFile f mode
+redirect Stdout _= Just `liftM` hDuplicate stdout
+-- hDuplicate stdout rather than passing stdout itself,
+-- because runProcess closes the Handles we pass it.
}

Context:

[TAG 1.0.9rc2
Tommy Pettersson <[EMAIL PROTECTED]>**20061116140351] 
Patch bundle hash:
064dfc2f9bc94d803ffbcb96d567e5c914d4d2b7
___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] darcs patch: Windows hard link support

[Simon Marlow]

Mon Dec  4 16:20:40 GMT Standard Time 2006  Simon Marlow <[EMAIL PROTECTED]>
  * Hard link support on Windows
  This works only on NTFS filesystems.  Also it requires Windows 2000 or
  later; this may or may not be acceptable, I'll leave that up to the
  darcs maintainers to decide.

New patches:

[Hard link support on Windows
Simon Marlow <[EMAIL PROTECTED]>**20061204162040
 This works only on NTFS filesystems.  Also it requires Windows 2000 or
 later; this may or may not be acceptable, I'll leave that up to the
 darcs maintainers to decide.
] {
hunk ./maybe_relink.c 31
+
+// Hard link support on Windows requires Windows 2000 or greater.
+// Therefore this rules out Windows 95/98/ME.
+#define _WIN32_WINNT 0x0500
+#include 
hunk ./maybe_relink.c 39
-return 0;
+BOOL result;
+result = CreateHardLink(src, dst, NULL);
+if (!result)
+return 0;
+else
+return -2;
}

Context:

[TAG 1.0.9rc2
Tommy Pettersson <[EMAIL PROTECTED]>**20061116140351] 
Patch bundle hash:
64791a5196ce126177af14504ad47edb77baec9d
___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: darcs patch: use System.Process on Windows, and a few more

[Simon Marlow]

Tommy Pettersson wrote:

I've reviewed and tested these patches on linux with ghc 6.4.


| Wed Nov 29 17:01:44 CET 2006  Simon Marlow <[EMAIL PROTECTED]>
|   * add explicit import list

| Wed Nov 29 17:03:42 CET 2006  Simon Marlow <[EMAIL PROTECTED]>
|   * hFlush after "waiting for lock" message

| Wed Nov 29 17:06:20 CET 2006  Simon Marlow <[EMAIL PROTECTED]>
|   * catch exceptions in stdout_is_a_pipe

These three looks good and works fine.



| Wed Nov 29 17:07:10 CET 2006  Simon Marlow <[EMAIL PROTECTED]>
|   * Use System.Process on Windows

This one I couldn't compile, but the following changes fixed it:

  hunk ./Exec.lhs 25
  -import System.Exit  ( ExitCode )
  +import System.Exit  ( ExitCode(..) )
  hunk ./Exec.lhs 39
  -import System
  +import System.IO ( stdin )
  +import Control.Exception ( bracket )

I can not test this patch since it's windows only, but I tried
to compile with #define WIN32, and got:

  Exec.lhs:33:27: Module `Control.Exception' does not export `bracketOnError'

Don't know if that's important, but I thought I'd mention it.


Thanks for taking a look.

These are due to changes in libraries between versions of GHC; bracketOnError 
only appeared in GHC 6.6, it seems.  So we need some configure tests to 
straighten things out.  Can I leave these up to you guys to sort out?


Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: summary of my recent spurt of patches, and request for suggestions

[Simon Marlow]

David Roundy wrote:


I've been working hard on getting support for the new hashed inventory
format into good shape.  If you aren't familiar with the benefits of the
new format (which I've talked about with at least some of you in person),
suffice to say that I see it as a precursor to working out the new way of
dealing with conflicts.


As an interested bystander, I'd really like to hear a brief description of what 
a "hashed inventory" is, and what benefits it brings.  Not a 12-page paper, just 
a quick outline will do fine, I don't want to distract you from the hacking 
frenzy :)


Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: summary of my recent spurt of patches, and request for suggestions

[Simon Marlow]

David Roundy wrote:

On Mon, Dec 04, 2006 at 04:34:48PM +, Simon Marlow wrote:


David Roundy wrote:



I've been working hard on getting support for the new hashed inventory
format into good shape.  If you aren't familiar with the benefits of the
new format (which I've talked about with at least some of you in person),
suffice to say that I see it as a precursor to working out the new way of
dealing with conflicts.


As an interested bystander, I'd really like to hear a brief description of 
what a "hashed inventory" is, and what benefits it brings.  Not a 12-page 
paper, just a quick outline will do fine, I don't want to distract you from 
the hacking frenzy :)



A hashed inventory is a modification of the darcs repository format, which
essentially replaces the _darcs/inventory file (which is human-readable, if
not human-modifiable, so if you're not familiar with it, you could take a
look) with a _darcs/hashed_inventory file.  The difference is that a hash
of the contents of each patch is stored, along with the identifier of the
patch, as is currently stored.  This hash is then used as the filename in
_darcs/patches/.  This has several benefits.

At the most obvious level, we've now got some extra information for
checking the consistency of a repository (helpful if, e.g. an http proxy
modifies files in transit).

The next advantage is that by cryptographically signing the hashed
inventory, you cryptographically sign the entire contents of the repository
(unless someone cracks sha1).  This is potentially valuable to high-profile
projects, or projects that use untrusted mirrors.

Next, because the filename for patches now depends on patch contents, all
darcs commands will be atomic (except with respect to the pristine
cache--but atomic with respect to remote access), including those that
currently aren't, such as amend-record and obliterate.

With hashed inventories it will be possible to implement "lazy" partial
repositories, in which darcs downloads patch files as needed to do the
commands you ask, since we'll have the hash with which to verify that the
patch files haven't been commuted (and therefore are still in the proper
context for our use).

Finally, as I mentioned above, the refactoring for this change should help
with our plans for new conflict handling, which will probably require that
we break the current picture of one patch file per named patch (which
wouldn't work in the current scheme where the patch filename is determined
by the name of the patch).


Great, thanks David!

Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: darcs patch: Windows hard link support

[Simon Marlow]

Juliusz Chroboczek wrote:

+#define _WIN32_WINNT 0x0500
+#include 
hunk ./maybe_relink.c 39
-return 0;
+BOOL result;
+result = CreateHardLink(src, dst, NULL);
+if (!result)
+return 0;
+else
+return -2;
}



I don't understand Windows -- but that looks completely bogus to me.
Does it obey the description in the comment just below?


Er.  Ok, I completely failed to understand what maybe_relink() was supposed to 
do, and didn't notice that it should compare the files before attempting to 
link.  Sorry about that, please ignore the patch, maybe I'll submit a fixed one 
at some point.


Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Add --show-external?

[Simon Marlow]

It'd be really useful when debugging problems with darcs' interaction with SSH 
for darcs to dump the external commands it is executing.  I have a temporary 
putStrLn that I uncomment in order to do this, but it would be better to have a 
flag, because then when a user reports some problem with darcs/SSH we can get 
them to show us the commands that their darcs is actually running (and perhaps 
the exit codes, too).


Should we add a flag --show-external that dumps the external commands?  I 
started to do this, but it means propagating [DarcsFlag] all over the place, and 
the patch is going to be pretty big.  Does anyone object to this?  In GHC when 
we have to propagate a flag everywhere we sometimes resort to a top-level 
unsafePerformIO instead (since the flag is always constant for a given run of 
darcs, this isn't so bad).  Thoughts?


Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

RE: [darcs-devel] Re: darcs patch: Windows hard link support

[Simon Marlow]

Juliusz Chroboczek wrote:
> Looking further at your patch, I'm not quite sure whether you're
> looking at the right place.  Please forgive me if the below is
> obvious for you.
>
> There are two places where Darcs will create hard links: (1) when
> doing``darcs get'' on a local filesystem, and (2) when runninng
> ``darcs optimize --relink''.
>
> The function maybe_relink, which is the one you tried to patch, is
> only used by (2).  Now (1) is implemented differently: it calls
> copyFilesOrUrls, which in turn calls copyLocals, which in turn calls
> copyLocal which is in External.hs and is implemented as so:
>
>   copyLocal fou out = createLink fou out `catchall` cloneFile fou out
>
>   cloneFile source dest = readFilePS source >>= writeFilePS dest
>
> So in order to get hard links on Windows when doing ``darcs get'',
> you've got two solutions -- you can either implement
> System.Posix.Files.createLink on Windows within the Ghc libraries
> (surely the preferred solution), or provide us with an implentation of
> createLink that works under Windows and that we'll substitute for
> Ghc's version within Darcs.
>
> (In case you find yourself confused by the package structure: there's
> an extra indirection through a module ``Workaround'' which is
> generated by configure.)

Thanks Juliusz - actually I did realise this after I submitted the patch.  One 
reason for the confusion was that I timed a 'darcs get' from a local repo using 
my installed darcs against the newly built one, and the new one was much 
faster, so I declared success :-)   I guess the speedup must be due to 
something else - maybe the System.Process switch that was also in my local copy.

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: darcs patch: Use System.Process on Windows (and 3 more)

[Simon Marlow]

David Roundy wrote:

On Sun, Dec 31, 2006 at 07:46:44PM +0100, Eric Y. Kow wrote:


Argh! I spoke too soon.  These patches make the external.pl script hang
on Unix.  The problem seems to be that the patch switches from the
bracket in the old IO module to the new Control.Exception.bracket.



The two brackest differ in what they are able to catch, since the two
catches differ.  IO.bracket (same as, I believe, System.IO.bracket) doesn't
catch asynchronous errors, I believe.  The semantics is confusing, to say
the least, and I've not got time to debug anything now.


The difference is indeed along those lines: IO.bracket only catches IO 
exceptions, because Haskell 98 only had IO exceptions; IO.bracket is a Haskell 
98 function, we keep it around for backwards compatibility only, I don't 
recommend using it except in a program that is supposed to be pure Haskell 98.


System.IO.bracket doesn't exist (maybe it did at some point in the past, I don't 
remember).


Control.Exception.bracket catches all kinds of exceptions, and additionally is 
safe in the presence of asynchronous exceptions.  That's the one to use.


I'm surprised if the difference would lead to anything hanging, though...

Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: darcs patch: Make annotate work on files with spaces in the name

[Simon Marlow]

Benedikt Schmidt wrote:

David Roundy <[EMAIL PROTECTED]> writes:



On Tue, Dec 26, 2006 at 03:35:26PM +0100, Benedikt Schmidt wrote:



There is some proof of concept code at http://beschmi.de/darcs-filecache/
and it really seems to make big difference for big repos. I hope to work
some more on it until and at the hackathon.


I'm not sure when I'll have time to look at this, but look forward to
checking it out!


I can send another mail when i've cleaned up the code a bit, this version
was mainly an exercise in how fast can i get something to compile to do
some timing to see if it's worth it.


I look forward to this in eager anticipation... it's about 3rd on my darcs 
wishlist.  The lack of this optimisation is why we can't offer access to GHC's 
repository via darcsweb, for example - every page view causes darcs to think for 
minutes.  Also I can't fully integrate GHC's darcs repository into our Trac.


I made a feature requst for this a while ago, in case you haven't seen it:

  http://bugs.darcs.net/issue124

Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] [issue391] GHC and flagging use of a given constant

[Simon Marlow]

New submission from Simon Marlow <[EMAIL PROTECTED]>:

David Roundy wrote:
> I'm cc'ing this to the bug tracker to create a bug for the feature request
> described below, which came up from an idea of Juliusz...
> 
> On Fri, Jan 12, 2007 at 01:19:36AM +0100, Josef Svenningsson wrote:
> 
>>On 1/11/07, David Roundy <[EMAIL PROTECTED]> wrote:
>>
>>>I think we'd really like to apply it not to the method, but to the instance
>>>itself.  So that use of the Show instance of Control.Exception.Exception
>>>would be deprecated.  This'd be more severe, since there might be classes
>>>that depend on a show instance, but don't actually use it.
>>>
>>>Another possibility for this particular problem would be to be able to
>>>rewrite the show instance.  If we could just remove the stupid "User
>>>error:" text from the show instance, I don't think it'd be a bug to use it
>>>anymore.
>>
>>I did a little testing and it turns out that you can actually achieve
>>what you want. Just write another Show instance for C.E.Exception and
>>import it wherever there is a possibility that a bad show might be
>>lurking. GHC will only complain about duplicate instances if you
>>actually use a method from the Show class with the particular type
>>which has several instances. So GHC is doing exactly what you want!
> 
> 
> Cool! (I would have expected it to cause trouble even when we don't use
> show.) I think this'd be a great little cleanup.  We could put the import
> in impossible.h, which is #included almost everywhere.  :)
> 
> For the benefit of the bug tracker, the idea is that we want to avoid using
> the Show instance of Control.Exception.Exception, which has the
> disadvantage of sometimes displaying "User error: xxx" when we failed with
> code such as error "xxx".  This isn't the user's fault, and we should never
> use this show instance (except if we've already checked that the exception
> is not a UserException or whatever it's called).  Adding this code to
> impossible.h will probably also require fixing a few (bad) uses of show.

I suggest not using userError, use a more descriptive exception type with 
dynamic exceptions (throwDyn, catchDyn).  This is what we do in GHC:

   http://darcs.haskell.org/ghc/compiler/utils/Panic.lhs

To avoid the use of IO.bracket, there are a couple of options:

   - Use -hide-package haskell98.  This probably works in 6.4.x.  No doubt
 you'll have to change a lot of code to import the base-package version
 of a lot of haskell98 modules, but you get to avoid a few of these legacy
 issues in one go.

   - the other way is to have a local 'IO' module that will override the
 haskell98 one.  This only works with GHC 6.6, however.

   - or just search for 'import IO' and replace them all with
 'import System.IO'.

Cheers,
Simon

--
messages: 1407
nosy: EricKow, beschmi, droundy, simonmarhaskell, tommy
status: unread
title: GHC and flagging use of a given constant


Darcs issue tracker <[EMAIL PROTECTED]>
<http://bugs.darcs.net/issue391>


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: Building darcs

[Simon Marlow]

Zachary P. Landau wrote:


When I'm working on the darcs code, I'm generally rebuilding the code
fairly often.  Unfortunately this seems to take a long time.  Certain
files, because of dependencies, have to build a lot of files again.  A
clean build on my machine, a 3.4ghz machine with 1 gig of ram, takes
somewhere around 15 minutes.  Each individual file takes a certain
amount of time, and then the final link at the end takes quite a bit by
itself.


When GHC compiles files with optimisation on, the .hi files contain a lot of 
information required for cross-module optimisation (the definitions of small 
functions, strictness properties, etc.).  This has the effect that more 
recompilation will be required when something changes: it's very likely that 
small changes to a module will cause the optimisation annotations in the .hi 
file to change, forcing more recompilation.  However when you compile without 
optimisation, the .hi file only contains the types of exported functions and 
datatypes, which is much less likely to change when you modify the file.  So for 
fast edit/compile cycles, you really want to compile with optimisation off.



The tips under 'Faster edit-compile cycle' on
http://darcs.net/DarcsWiki/DeveloperTips don't seem to be valid anymore.
I don't see O2 under GHCFLAGS.  I did try the ghci line but it fails
with an error when trying to link in curl.  I may try to fight with this
later, as it seems like a good option.  I'll update that page once I'm
sure I'm not just doing something stupid.

I also tried playing with the -j option with make, but that did a fairly
good job of bringing my system to its knees.


I would try -j2, or more depending on how many cores in your machine.  Reduce 
the number if you're running out of memory.


Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://www.abridgegame.org/cgi-bin/mailman/listinfo/darcs-devel

[darcs-devel] Re: patches accepted 2007-03-17

[Simon Marlow]

Eric Y. Kow wrote:

On Sat, Mar 17, 2007 at 17:32:37 -0400, Zachary P. Landau wrote:

Rejected (1)

Sat Mar 10 09:31:37 CET 2007  Eric Kow <[EMAIL PROTECTED]>
  * Use System.Cmd.system to launch SSH control master.

Just out of curiosity, why is this being rejected?  There was only one
reply to the email, but one that said the problem wasn't showing up with
the patch.


I used it with one of my other repositories, some other combination of
patches, and I still saw the problem (or so I thought).  The other
reason is that I still don't have any idea why it works and I suspect
that for this kind of thing it'll be useful to do so.  I basically need
(someone to) look deeper into the Exec stuff and come up with a more
global explanation and fix, but I don't really have the time for that,
so I thought it'd be better to just reject this outright rather than
keep people hanging.

If you do want to continue using the cm functionality, I am interested
to see how things work out over time, using this patch and enabling
--ssh-cm in your defaults.


Another reason to be suspicious of this patch is that it bypasses the existing 
infrastructure for running external processes in darcs.


If using System.Cmd makes a difference, then the right thing to do is to use 
System.Process in exec, because System.Cmd is just a thin wrapper around 
System.Process.


Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] Re: patch for lazy partial repos

[Simon Marlow]

Max Battcher wrote:

This may be completely off the mark, I'm entirely ignorant of how
partial repositories work (and I've not had the reason to work with
one thus far), but does darcs need the "url symlink" patches anyway?

Why can't darcs just take on the lazy behavior whenever a patch just
doesn't exist or is perhaps an empty file or some other similar
thing...  ie, in that changes -v when it needs the information in a
patch and the patch isn't available couldn't it just attempt an
auto-pull of that patch from _darcs/prefs/defaultrepo, unless some
alternative is specified in an argument?


This sounds like much more reasonable behaviour than auto-downloading from the 
original location that was specified with 'darcs get'.


I find it hard to pin down exactly what I dislike about --lazy, but it's 
something like this: I think of 'darcs get' as like 'cp', but with --lazy it 
would be like 'ln -s'.  You're specifying exactly what the requirements are -- 
that the source repository doesn't move -- but to me that's an unreasonable 
requirement.  The 'cp' semantics are pure, but the 'ln -s' semantics put your 
repository in the IO monad!


And I don't have the option to retarget the link later, so if the source repo 
does move, I'm hosed.  In all other things, darcs is completely agnostic about 
the location of a repo, so it seems strange to create a fixed link.


There are other things that could go wrong.  For example, I'm used to pulling 
from a --partial repo to get a new --partial repo (e.g. the ghc-6.6 repo used to 
be partial, it isn't any more).


Personally I'd be happy if darcs said something like:

  cannot complete this operation because the following patch is not
  available:
"blah blah ..."
  please use --full-repo P to specify where to fetch the patch from.

Cheers,
Simon

___
darcs-devel mailing list
[EMAIL PROTECTED]
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] stable branch needs hashed_inventory?

[Simon Marlow]

I built the stable branch today on my Windows machine, and it fails to push to 
an existing repository, with this error:


> c:/builds/darcs/darcs push [EMAIL PROTECTED]:/home/darcs/ghc-wind
ows-builds
darcs failed:  Not a repository: [EMAIL PROTECTED]:/home/darcs/ghc-wind
ows-builds ((scp) failed to fetch: [EMAIL PROTECTED]:/home/darcs/ghc-wi
ndows-builds/_darcs/hashed_inventory)

Is this the correct behaviour, or should it be backwards compatible?  Do I need 
to upgrade the remote repository?


Nice to see this hashed inventory stuff going in, BTW.  I hope this will make 
life a lot easier with partial repos.


Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] problem in the stable repo?

[Simon Marlow]

Pulling the stable branch seems to hang for me.  One of my local copies of the 
darcs stable repo is 151 patches behind, and when I try to pull it darcs hangs - 
I gave it 20 mins or so before giving up.


Having got a copy of the stable branch with a fresh get, I tried to get back to 
1.0.9 using 'darcs unpull --from-tag=1.0.9', and darcs hangs again.  Perhaps 
there are some problem patches in the repo post-1.0.9?


Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] Re: problem in the stable repo?

[Simon Marlow]

Simon Marlow wrote:
Pulling the stable branch seems to hang for me.  One of my local copies 
of the darcs stable repo is 151 patches behind, and when I try to pull 
it darcs hangs - I gave it 20 mins or so before giving up.


Having got a copy of the stable branch with a fresh get, I tried to get 
back to 1.0.9 using 'darcs unpull --from-tag=1.0.9', and darcs hangs 
again.  Perhaps there are some problem patches in the repo post-1.0.9?


Update: pulling the patches in small batches helped.  There was one particular 
patch that took 5 minutes to pull:


Thu Apr  5 01:06:16 BST 2007  David Roundy <[EMAIL PROTECTED]>
  * add support for partial and lazy downloading of hashed repos.

but after that the rest pulled without any problems.

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] Re: stable branch needs hashed_inventory?

[Simon Marlow]

David Roundy wrote:

On Tue, Jul 03, 2007 at 02:10:05PM +0100, Simon Marlow wrote:
I built the stable branch today on my Windows machine, and it fails to push 
to an existing repository, with this error:



c:/builds/darcs/darcs push [EMAIL PROTECTED]:/home/darcs/ghc-wind

ows-builds
darcs failed:  Not a repository: 
[EMAIL PROTECTED]:/home/darcs/ghc-windows-builds ((scp) failed to fetch: 
[EMAIL PROTECTED]:/home/darcs/ghc-windows-builds/_darcs/hashed_inventory)


Is this the correct behaviour, or should it be backwards compatible?  Do I 
need to upgrade the remote repository?


No, this is not expected behavior, we should be backwards compatible.  :(
I'm not sure what causes this, but would definitely like to fix it.  It is
clearly a bug in the repository-identification code, but I'm not sure what
the bug is.


Mystery solved.  There seems to be a bug in darcs, but it's a minor one.

I pointed darcs at the wrong place; the location I gave is not actually a 
repository.  In this situation, darcs used to say "not a repository", in 
the current stable branch it gives the above error about failing to load 
hashed_inventory.  Does that give you enough to go on?  You should be able 
to reproduce it.


Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] Re: Example Conflicts

[Simon Marlow]

David Roundy wrote:

On Wed, Aug 01, 2007 at 01:58:29AM +0100, Ian Lynagh wrote:



And the "cancel A" patch doesn't have any sort of reference to B in it?


Right.


I seem to remember there was a problem with this approach.  Perhaps not a 
technical problem, but a conceptual one.


Something like this:

Suppose patches A and B conflict, and David and Ian both have repositories 
containing A and B.


Ian resolves in favour of A, records cancel(B).
David resolves in favour of B, records cancel(A).

Ian pulls from David, and now has both cancel(A) and cancel(B).  At this 
point we expect a conflict, because both David and Ian have resolved the 
original conflict in different ways; but cancel(A) and cancel(B) commute. 
Don't they?


Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] [issue556] wishlist: default repository for darcs pull and push

[Simon Marlow]

Juliusz Chroboczek wrote:
>>>   ALL no-set-default
>>>   ALL no-test
>>>   send unified
> 
>> I'd rather leave the default
>> --test for record, but making send default to unified or switching to make
>> --no-set-default the default wouldn't be a problem--except we'd want to
>> ensure that there was some sort of consensus, and that we wouldn't
>> immediately have folks clamoring for a return to the old behavior.
> 
> Send --unified looks like a no-brainer to me.  I'd be more cautious
> about the no-set-default, as it's at least easy to explain, which
> makes it suitable as the default.

FWIW, my .darcs/defaults contains

changes --summary
diff --unified
pull --no-set-default
push --no-set-default

(and I find it quite annoying that when doing 'darcs changes -v' I have to 
add --no-summary, why is that?)

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre1, the first prerelease for darcs 2

[Simon Marlow]

David Roundy wrote:

> === Creating a repository in the darcs-2 format ===
> 
> Converting an existing repository to the darcs-2 format is as easy as
> 
> darcs convert oldrepository newrepository

I did this for GHC's repository.  I left it running last night, and I'm not 
sure whether it completed successfully - I certainly have a repository, but 
it had a _darcs/lock file left in it.  It seems to have all the patches in 
_darcs/patches, and the last one is dated about 3.5 hours after I started 
the conversion.

$ darcs2 query repo
   Type: darcs
 Format: hashed, darcs-2-experimental
   Root: /64playpen/simonmar/ghc-darcs2
   Pristine: HashedPristine
  Cache: thisrepo:/64playpen/simonmar/ghc-darcs2
Num Patches: 17532

A few quick performance tests.  The darcs2 repository is on a local filesystem:

$ time darcs2 whatsnew -s
No changes!
2.25s real   2.04s user   0.18s system   98% darcs2 w -s

In a darcs1 GHC repository mounted over NFS:

$ time darcs whatsnew -s
No changes!
0.13s real   0.03s user   0.05s system   58% darcs w -s

"darcs changes" seems to have a big performance regression:

$ time darcs2 changes --last=10 >/dev/null

I killed it after 3 minutes of CPU time and the process had grown to 1.4Gb. 
  darcs1 does this in 0.05 seconds using 2Mb.  Perhaps the repository is 
corrupted somehow?

I've tarred up the repo and put it here:

   http://darcs.haskell.org/ghc-darcs2.tar.gz

It is also online here:

   http://darcs.haskell.org/ghc-darcs2

---
Documentation nits

The 'darcs show' documentation appears in two places, under "Seeing what 
you've done" and "Advanced examination of the repository".

The docs still say that two patches making the same change are considered 
to be in conflict.

I can't find any docs about using lazy patch downloading and the 
~/.darcs/sources file.

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre1, the first prerelease for darcs 2

[Simon Marlow]

Simon Marlow wrote:

> It is also online here:
> 
>   http://darcs.haskell.org/ghc-darcs2

Getting a lazy partial repository over http isn't particularly quick:

   $ time darcs2 get http://darcs.haskell.org/ghc-darcs2 --darcs-2
   Finished getting.
   495.19s real   2.08s user   1.12s system   0% darcs2 get 
http://darcs.haskell.org/ghc-darcs2 --darcs-2

Compared to a --partial get of the GHC repository:

   $ time darcs get --partial http://darcs.haskell.org/ghc
   Copying patch 179 of 179... done.
   Applying patch 178 of 178... done.
   Finished getting.
   181.93s real   9.43s user   1.42s system   5% darcs get --partial 
http://darcs.haskell.org/ghc

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre1, the first prerelease for darcs 2

[Simon Marlow]

A small UI issue:

$ darcs2 get http://darcs.haskell.org/ghc-darcs2

darcs failed:  Incompatibility with repository 
http://darcs.haskell.org/ghc-darcs2:
Cannot mix darcs-2 repositories with older formats


Since I'm trying to get a darcs-2 format repository, I would expect it to 
just work, and give me a darcs-2 copy of it, without having to specify 
--darcs-2.

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre1, the first prerelease for darcs 2

[Simon Marlow]

David Roundy wrote:

> darcs check should work to indicate the conversion went fine.

Just fired one off, I'll let you know if it finishes before I've written 
this email :-)

>> $ darcs2 query repo
>>Type: darcs
>>  Format: hashed, darcs-2-experimental
>>Root: /64playpen/simonmar/ghc-darcs2
>>Pristine: HashedPristine
>>   Cache: thisrepo:/64playpen/simonmar/ghc-darcs2
>> Num Patches: 17532
>>
>> A few quick performance tests.  The darcs2 repository is on a local 
>> filesystem:
>>
>> $ time darcs2 whatsnew -s
>> No changes!
>> 2.25s real   2.04s user   0.18s system   98% darcs2 w -s
>>
>> In a darcs1 GHC repository mounted over NFS:
>>
>> $ time darcs whatsnew -s
>> No changes!
>> 0.13s real   0.03s user   0.05s system   58% darcs w -s
> 
> The difference here is that I haven't implemented the time-stamp
> synchronizing feature for hashed repositories.  I wasn't sure it was
> still needed (and would be nice to drop, as it's a bit hackish), since
> for the darcs repository whatsnew is pretty fast.  Will have to add it
> to the TODO list.

2s or so for a 'darcs whatsnew' doesn't seem much, but it affects the 
perceived responsiveness quite a bit.  I'm used to doing lots of whatsnew 
operations in emacs, and the fact that they're really snappy even on the 
whole GHC tree is something I really like about darcs.

> It's also possible that you're getting hurt by the cost of checking
> the sha1 hashes, which we currently do in a rather paranoid way (I
> like being paranoid, except when it hurts).  If this is the case, we
> could speed things up by using a faster sha1 hash function.  Right now
> we use on written in Haskell, but it wouldn't be hard to bind to a
> well-optimized implementation (openssl or something).

I presume you can profile using the repository I uploaded?

> But I guess I've been running on local disks long enough that I've
> forgotten the cost of opening up a file over nfs... I'd best go ahead
> and make this change.  It's potentially a little painful, as
> synchronizing the modification time of files in the pristine cache
> doesn't interact well with hard linking between files in the pristine
> caches of different repositories.
 >
> Which means either we live with a
> performance cost to hard linking of pristine caches, or we store
> modification times in the file contents of the pristine cache, so that
> we could have multiple modification times per file.  :(

So while the pristine files can be shared between repositories, the 
modification times cannot, so I guess that means you need to store them 
separately, perhaps in another file?

>> "darcs changes" seems to have a big performance regression:
>>
>> $ time darcs2 changes --last=10 >/dev/null
>>
>> I killed it after 3 minutes of CPU time and the process had grown to 1.4Gb.
>>   darcs1 does this in 0.05 seconds using 2Mb.  Perhaps the repository is
>> corrupted somehow?
> 
> Yikes! That's actually a very surprising bug.  I'd be interested in
> hearing if it shows up if you run a darcs2 optimize first? Either way,
> of course, it's a serious bug, but that'd give a hint where the
> trouble is.

darcs2 check has nearly finished... yup, the repository is consistent.

Trying optimise now...

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre1, the first prerelease for darcs 2

[Simon Marlow]

Simon Marlow wrote:
> David Roundy wrote:
>> Yikes! That's actually a very surprising bug.  I'd be interested in
>> hearing if it shows up if you run a darcs2 optimize first? Either way,
>> of course, it's a serious bug, but that'd give a hint where the
>> trouble is.
> 
> darcs2 check has nearly finished... yup, the repository is consistent.
> 
> Trying optimise now...

darcs2 optimize was killed by the OS after 5 minutes for using up all my 
memory :-(

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre1, the first prerelease for darcs 2

[Simon Marlow]

David Roundy wrote:
> On Wed, Dec 12, 2007 at 01:45:13PM +0000, Simon Marlow wrote:
>> "darcs changes" seems to have a big performance regression:
>>
>> $ time darcs2 changes --last=10 >/dev/null
>>
>> I killed it after 3 minutes of CPU time and the process had grown to 1.4Gb. 
>>   darcs1 does this in 0.05 seconds using 2Mb.  Perhaps the repository is 
>> corrupted somehow?
> 
> Okay, it turns out that it was indeed bad strictness causing the trouble.
> For some reason, I had made the PatchInfoAnd data type strict in both its
> components, which meant that every time we read a patch ID, we also needed
> to parse the patch itself.  Very foolish.  There may be some further
> regressions (I'm still running an optimize with profiling enabled.  But
> darcs changes --last 10 (with profiling running) now takes me just a bit
> over a minute, and not too much memory (I don't quite recall).

Ok, that is certainly an improvement:

  $ time darcs2 cha --last=10
  ...
  60.60s real   59.83s user   0.21s system   99% darcs2 cha --last=10

But this is still 1000 times slower than darcs1 for the same operation. 
Doesn't darcs changes just dump the contents of the inventory?

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre1, the first prerelease for darcs 2

[Simon Marlow]

David Roundy wrote:
> On Fri, Dec 14, 2007 at 10:04:57AM +0000, Simon Marlow wrote:
>> David Roundy wrote:
>>> Okay, it turns out that it was indeed bad strictness causing the trouble.
>>> For some reason, I had made the PatchInfoAnd data type strict in both its
>>> components, which meant that every time we read a patch ID, we also needed
>>> to parse the patch itself.  Very foolish.  There may be some further
>>> regressions (I'm still running an optimize with profiling enabled.  But
>>> darcs changes --last 10 (with profiling running) now takes me just a bit
>>> over a minute, and not too much memory (I don't quite recall).
>> Ok, that is certainly an improvement:
>>
>>   $ time darcs2 cha --last=10
>>   ...
>>   60.60s real   59.83s user   0.21s system   99% darcs2 cha --last=10
>>
>> But this is still 1000 times slower than darcs1 for the same operation. 
>> Doesn't darcs changes just dump the contents of the inventory?
> 
> If you run darcs optimize first, this drops to 1s for me.  Still a bit
> slow, but not so bad (and that's most of why darcs1 is faster).

Ok, confirmed.

However, I never use optimize, and only use tag when I need to.  This is 
mainly because I'm paranoid and I don't fully understand what optimize 
does, and perhaps also because I'd like to understand what goes wrong if 
you don't use it.

I guess I don't understand why optimize is exposed to the user at all. if 
there's an optimal state for the repository, why can't it be maintained in 
that state?

> The problem is that --last isn't at all tuned for efficiency, and instead
> uses the same code that can handle --from-tag, and this could require
> reordering (--from-tag could), so there are O(N^2) operations going on,
> where N is the number of patches since the last known-to-be-in-order tag.
> 
> This has never been a problem (that I'm aware of), and simplifies the code
> since we only have to deal with one case.  Reusing the same code also
> ensures that performance improvements for one command are leveraged for
> other commands.  Which comes down to: I'd rather not optimize changes
> --last for the case of 17k patches and no tags (or not running optimize).
> But I could certainly be convinced, because we are indeed taking a very
> roundabout approach.  But then again, darcs1 uses exactly the same
> approach, so if we could gain another factor of ten without losing this
> abstraction, I'd rather know how--particularly as the improvement is likely
> to benefit all other darcs commands.

Sure, code re-use is definitely a good thing, and I agree that optimising 
this operation in ways that darcs1 does not would be premature, given that 
there is still a factor of 20 difference between darcs1 and darcs2 
unaccounted for.

Thanks for the quick response to my feedback so far... things are 
definitely heading in the right direction!

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre2

[Simon Marlow]

David Roundy wrote:
> I am pleased to announce the availability of the second prerelease of darcs
> two, darcs 2.0.0pre2.

Thanks!

Continuing my performance tests, I tried unpulling and re-pulling a bunch 
of patches in a GHC tree.  I'm unpulling about 400 patches using 
--from-tag, and then pulling them again from a local repo.  Summary: darcs2 
is about 10x slower than darcs1 on unpull, and on pull it is 100x slower in 
user time but only 20x slower in elapsed time.

In both cases, the repository was on an NFS filesystem.  In the darcs2 
case, the repository I was pulling from was on the local disk, and I'm also 
using a cache (NFS-mounted).  The darcs2 repository has been optimized, but 
the darcs1 repository has not (at lesat, not recently).  I did all of these 
a couple of times to eliminate the effects of cache preloading etc., the 
times reported are from the second run.

--- darcs 1:

$ time darcs unpull --from-tag 2007-09-25 -a
Finished unpulling.
35.17s real   5.77s user   1.00s system   19% darcs unpull --from-tag 
2007-09-25 -a

$ time darcs pull ~/ghc-HEAD -a
Pulling from "/home/simonmar/ghc-HEAD"...
33.51s real   3.62s user   1.05s system   13% darcs pull ~/ghc-HEAD -a

--- darcs 2:

$ time darcs2 unpull --from-tag 2007-09-25 -a
Finished unpulling.
385.22s real   52.18s user   12.62s system   16% darcs2 unpull --from-tag 
2007-09-25 -a

$ time darcs2 pull /64playpen/simonmar/ghc-darcs2 -a
Finished pulling and applying.
668.75s real   290.74s user   15.03s system   45% darcs2 pull 
/64playpen/simonmar/ghc-darcs2 -a

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] Problem building darcs 2 on WinXP

[Simon Marlow]

Claus Reinke wrote:
>> I've installed MinGW, MSYS (with msysDTK, autoconf and automake), GHC,
>> zlib, libcurl and OpenSSL. I can run ./configure --target=mingw and
>> compile the source, but I get this linking error:
>>
>> Linking darcs ...
>> src/win32/System/Posix.o(.text+0x550):fake: undefined reference to `SleepEx'
>> collect2: ld returned 1 exit status
>> make: *** [darcs] Error 1
> 
> it might be a case of adding -fvia-C (used to be the default
> until recently). if that is true, you might want to file a ticket
> for ghc (change in default route gives failures and unhelpful
> error messages).
> 
> also, as a naive ffi user, i'd have expected
> 
> foreign import ccall "winbase.h SleepEx" c_SleepEx :: DWORD -> BOOL -> IO 
> DWORD
> 
> to match the
> 
> WINBASEAPI DWORD WINAPI SleepEx(DWORD,BOOL);
> 
> in winbase.h, rather than the current
> 
> foreign import ccall "winbase.h SleepEx" c_SleepEx :: CULong -> CUInt -> 
> IO CInt
> 
> but i don't know whether that is any issue at all.

Looks like it should be

foreign import stdcall "winbase.h SleepEx" c_SleepEx ...

(i.e. stdcall rather than ccall)

this would cause it to work with -fvia-C but fail with -fasm, because the C 
compiler can see the prototype.  Arguably a bug in -fvia-C, but it's been 
that way for a very long time.

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] [darcs-users] Problem building darcs 2 on WinXP

[Simon Marlow]

David Roundy wrote:
> On Tue, Dec 18, 2007 at 09:20:22AM +1100, Rob Moss wrote:
>> On 18/12/2007, David Roundy <[EMAIL PROTECTED]> wrote:
>>> On Mon, Dec 17, 2007 at 03:55:20PM +, Simon Marlow wrote:
>>>> Looks like it should be
>>>>
>>>> foreign import stdcall "winbase.h SleepEx" c_SleepEx ...
>>>>
>>>> (i.e. stdcall rather than ccall)
>>>>
>>>> this would cause it to work with -fvia-C but fail with -fasm, because the C
>>>> compiler can see the prototype.  Arguably a bug in -fvia-C, but it's been
>>>> that way for a very long time.
>>> I'm not sure I understand what's gone wrong here.  Do either of you have a
>>> recommendation for something our fearless user can try? or a patch for us
>>> to apply?
>> Thanks a lot everybody! By changing the import from ccall to stdcall,
>> I was able to successfully compile darcs! The HTML manual is also
>> compiled, although there must be some small problem (maybe in the
>> makefile?) as make rebuilds the manual every time I run it. That
>> doesn't matter at all to me, but if you'd like I can try to figure out
>> what the problem might be.
>>
>> Anyway, I'm off to test darcs 2! Thanks again for your help :)
> 
> Great!
> 
> But I'm still confused as to what change (if any) I should make.  I thought
> I understood that changing to stdcall would fail on older versions of ghc,
> or is that always the correct way to go? If so, should we perhaps put in a
> configure test to determine which is the proper way to call this function?

stdcall is the correct calling convention for SleepEx, and it should work 
with all versions of GHC.  Older versions of GHC used -fvia-C by default, 
which had the effect of hiding the fact that the calling convention was 
wrong, -fasm is less forgiving.

Summary: just change it to stdcall.

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] faster darcs whatsnew

[Simon Marlow]

David Roundy wrote:

> I've just pushed a change that should make darcs whatsnew as fast with
> darcs 2 on hashed repositories as it is with darcs 1.  If you could try
> this out on your large repositories, that would be great!
> 
> Incidentally, I went with the "sloppy" approach, which means that if you
> use a cache and have multiple copies of the same repository, this
> optimization may not work so well for both of your repositories that share
> contents of their pristine cache.  I doubt this will be a bit issue, but
> it's something to keep in mind (and to complain about, if you anticipate it
> causing trouble).

Hmm, this worries me a bit.  I typically have many GHC trees, and I 
anticipate using a cache with darcs 2, so will this mean that whatsnew will 
  sometimes miss changes?  Or will it just be slower?

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] http pipelining

[Simon Marlow]

Stefan O'Rear wrote:

> How about something like:
> 
> data Cookie = C { getData :: L.ByteString }
> fetchURL :: URI -> Cookie   -- sends request when forced, reads 
> responce when deeply forced
> 
> {- sample implementation, ignores the existance of multiple servers and
> the Connection: close header -}
> 
> sock = unsafePerformIO ...
> 
> queue fun = newChan >>= \q -> forkIO (forever (readChan q >>= fun)) >>
> return q
> 
> writer = unsafePeformIO $ queue $ \ req -> writeRequest sock req
> reader = unsafePeformIO $ queue $ \ var ->
>   readRequest sock (\ bit -> writeChan var (Just bit)) >>
>   writeChan var Nothing
> 
> 
> fetchURL url = unsafePerformIO $ do
>   c <- newChan
>   writeChan writer (fetch url)
>   writeChan reader c

A single queue seems nicer, given that you would otherwise need some 
locking to prevent races between multiple threads (I know this doesn't 
apply to darcs, but still).

>   return $ C $ unsafePerformIO $ getData c

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] faster darcs whatsnew

[Simon Marlow]

David Roundy wrote:
> On Dec 18, 2007 4:58 AM, Simon Marlow <[EMAIL PROTECTED]> wrote:
>> Hmm, this worries me a bit.  I typically have many GHC trees, and I
>> anticipate using a cache with darcs 2, so will this mean that whatsnew will
>>   sometimes miss changes?  Or will it just be slower?
> 
> Well, using the file-modification-times optimization always means that
> whatsnew will sometimes miss changes, since it's always possible that
> the pristine cache will have the same timestamp as the working
> directory file, but this is unlikely (and no more likely when
> interacting with multiple repositories, it's actually in a single
> repository that you're more likely to modify a file in the working
> directory within a second of when a darcs record is performed).  The
> main issue would be that it'd be slower.  This shouldn't be a big
> issue, since whichever darcs repository you last recorded in (or
> pulled into, etc) will have a fast whatsnew.
> 
> One other possibility would be that we could disable global caching of
> the pristine cache (bad confusion of the term "cache" here).

Ok.  Though it seems the right thing to do is to store the modification 
times in a file under _darcs, the file just needs to contain a list of 
(,) pairs that you read into a Data.Map (you're really just 
using the filesystem as a Data.Map at the moment).  The question is how 
much hacking you'd have to do to make this fast enough... I don't know.  It 
shouldn't be too hard to beat stat() with Data.Map, but there's the 
overhead of populating the Data.Map in the first place.

I guess this isn't a high priority now.  But it would be good if it were 
possible to disable globally caching the pristine copy, so we can get the 
darcs1 behaviour.  Perhaps have separate 'cache:' and 'cachepristine:'?

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre2

[Simon Marlow]

David Roundy wrote:

> Anyhow, could you retry this test with the above change in methodology, and
> let me know if (a) the pull is still slow the first time and (b) if it's
> much faster the second time (after the reverse unpull/pull)?

I think I've done it in both directions now, and it got faster, but still 
much slower than darcs1:

$ time darcs2 unpull --from-tag 2007-09-25 -a
Finished unpulling.
58.68s real   50.64s user   6.36s system   97% darcs2 unpull --from-tag 
2007-09-25 -a
$ time darcs2 pull -a ../ghc-darcs2
Pulling from "../ghc-darcs2"...
Finished pulling and applying.
53.28s real   44.62s user   7.10s system   97% darcs2 pull -a ../ghc-darcs2

This is still an order of magnitude slower than darcs1 for the same 
operation.  (these times are now on the local filesystem, BTW)

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre2

[Simon Marlow]

David Roundy wrote:
> On Thu, Jan 03, 2008 at 11:11:40AM +0000, Simon Marlow wrote:
>> David Roundy wrote:
>>> Anyhow, could you retry this test with the above change in methodology, and
>>> let me know if (a) the pull is still slow the first time and (b) if it's
>>> much faster the second time (after the reverse unpull/pull)?
>> I think I've done it in both directions now, and it got faster, but still 
>> much slower than darcs1:
>>
>> $ time darcs2 unpull --from-tag 2007-09-25 -a
>> Finished unpulling.
>> 58.68s real   50.64s user   6.36s system   97% darcs2 unpull --from-tag 
>> 2007-09-25 -a
>> $ time darcs2 pull -a ../ghc-darcs2
>> Pulling from "../ghc-darcs2"...
>> Finished pulling and applying.
>> 53.28s real   44.62s user   7.10s system   97% darcs2 pull -a ../ghc-darcs2
>>
>> This is still an order of magnitude slower than darcs1 for the same 
>> operation.  (these times are now on the local filesystem, BTW)
> 
> Is this with the latest darcs-unstable? I made some improvements shortly
> before Christmas (or was it after Christmas?) that ought to improve the
> speed of pulls dramatically.  We were doing O(N^2) operations in our
> handling of "pending" changes, which I fixed (I think).  So I'll wait on
> investigating this until you've confirmed which version this was tested
> with.  And thanks for the testing!

This is using a binary I compiled up from the latest sources yesterday, so 
it should have those improvements.

Cheers,
Simon


___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] darcs patch: test: Exibit a falling test about rollback.

[Simon Marlow]

David Roundy wrote:
> On Thu, Jan 10, 2008 at 06:23:56PM +0100, Nicolas Pouillard wrote:
>> Excerpts from David Roundy's message of Thu Jan 10 18:08:59 +0100 2008:
>>> On Mon, Jan 07, 2008 at 03:43:40PM +, Nicolas Pouillard wrote:
 Mon Jan  7 16:02:24 CET 2008  [EMAIL PROTECTED]
   * test: Exibit a falling test about rollback.
   Indeed  the only test about rollback was br0ken by a prior test that 
 creates a
   directory  and  remove  read  permissions  to  it.  The  rollback test 
 do some
   records  that  silently  fail  by lack of permissions, finally the 
 rollback is
   cancelled since the named patch doesn't exist.
   This shows that rollback need some care.
>>> Thanks for the patch!
>>>
>>> I've actually been debating the idea of removing the rollback command.
>>> It's poorly implemented, and has been a source of confusion and problems.
>>> What do you think?
>> The source of problems was about hidden conflicts, right?
>> It's no longer a problem in darcs2, right?
>>
>> It's mainly a common use case when can no longer use amend-record.
>>
>> I  think that's also a great tool to temporarily revert a patch without 
>> having
>> two repositories.
>>
>> Moreover  this  kind of operation is waited when one know that patches must 
>> be
>> invertible.
> 
> The problem is that it's a pretty limited and counterintuitive command.
> You can't (currently) rollback a patch if there is a patch that depends on
> it which has been rolled back already.  And it doesn't affect the working
> directory, which makes certain things much easier (e.g. no need to deal
> with conflicts), but doesn't match what most folks actually want to do.
> Also, you can't add a note indicating *why* a patch was rolled back, which
> is a pretty big downside.
> 
> Having just chatted on this subject with a friend who walked by my office,
> I think what I'll do is implement a modified rollback that will allow you
> to undo more than one named patch at a time, and will make those changes in
> the working directory as well as recording them, and will allow you to
> provide a description of why you're rolling the change back... and will
> also (maybe not in the first draft) allow you to roll back just a subset of
> the primitive changes in those patches.  I think this'll be more useful and
> also easier to implement.

How about just applying the inverse of the patch (suitably commuted) to the 
working copy, leaving the user to record it, or record only parts of it, or 
whatever.  That seems like the most flexible solution.

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] [issue573] corrupted repository -- indelible "conflicting local changes"

[Simon Marlow]

Tommy Pettersson wrote:
> Tommy Pettersson <[EMAIL PROTECTED]> added the comment:
> 
> On Sat, Jan 12, 2008 at 10:00:10PM -, Tommy Pettersson wrote:
>> An alternative is to switch to the current darcs-stable, which I
> 
> Darn, that is not true.
> 
> The bug is actually a known bug. A backport exists (which is why
> I wrongly thought darcs-stable was ok, but I haven't pushed it
> yet). The reason I haven't pushed the backport is (beside I
> wasn't aware it could be triggered this way) it conflicts badly
> with darcs-unstable, and the required conflict resolution is of
> course even worse.
> 
> I think I'm ready to wrap up a release of 1.1.0 tomorrow (well
> rc1, and make it final within a week), but I'm not sure that's a
> good idea when everything is focused on darcs2.

I tried the latest darcs-stable recently on Windows, and had some problems. 
  Perhaps I missed something, but it seemed that 'darcs get --partial' was 
getting *all* the patches, ignoring the checkpoint.  And due to the known 
case-insensitive-filesystem issue that caused the get to fail.

I had to back off to 1.0.9 to get it to work again.  I still have the 
1.1.0pre binary if you need any more details, but perhaps the above will be 
enough to go on.

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre2

[Simon Marlow]

David Roundy wrote:
> On Thu, Jan 03, 2008 at 11:11:40AM +0000, Simon Marlow wrote:
>>> Anyhow, could you retry this test with the above change in methodology,
>>> and let me know if (a) the pull is still slow the first time and (b) if
>>> it's much faster the second time (after the reverse unpull/pull)?
>> I think I've done it in both directions now, and it got faster, but still 
>> much slower than darcs1:
>>
>> $ time darcs2 unpull --from-tag 2007-09-25 -a
>> Finished unpulling.
>> 58.68s real   50.64s user   6.36s system   97% darcs2 unpull --from-tag 
>> 2007-09-25 -a
>> $ time darcs2 pull -a ../ghc-darcs2
>> Pulling from "../ghc-darcs2"...
>> Finished pulling and applying.
>> 53.28s real   44.62s user   7.10s system   97% darcs2 pull -a ../ghc-darcs2
>>
>> This is still an order of magnitude slower than darcs1 for the same 
>> operation.  (these times are now on the local filesystem, BTW)
> 
> I've recently found the problem leading to this slowdown (I believe) and
> get about an order-of-magnitude improvement in the speed of a pull of 400
> patches in the ghc repository.  It turned out to be an issue that scaled
> with the size (width) of the repository, not with the number of patches
> (which had been the obvious suspect), which was causing trouble when
> applying to the pristine cache.
> 
> At this point, darcs-2 outperforms darcs-1 on most tests that I've tried,
> so it'd be a good time to find some more performance problems, if you
> can... and I don't doubt that there are more out there.

Certainly a lot faster, nice work!  Though it's still not as fast as 
darcs-1 here.  New figures:

$ time darcs2 unpull --from-tag 2007-09-25 -a
Finished unpulling.
18.83s real   15.27s user   1.53s system   89% darcs2 unpull --from-tag 
2007-09-25 -a
$ time darcs2 pull ../ghc-darcs2-other -a
Finished pulling and applying.
10.38s real   7.69s user   1.50s system   88% darcs2 pull ../ghc-darcs2-other -

I repeated the darcs-1 timings for comparison:

$ time darcs unpull --from-tag 2007-09-25 -a
Finished unpulling.
8.04s real   7.14s user   0.90s system   99% darcs unpull --from-tag 
2007-09-25 -a
$ time darcs pull ~/ghc-HEAD -a
Finished pulling and applying.
7.90s real   4.90s user   0.98s system   74% darcs pull ~/ghc-HEAD -a

In this case darcs-1 is pulling more patches (530 vs. 400), because I'm 
using the latest GHC HEAD repo.  Also the darcs-1 repository being pulled 
from is on a different, NFS mounted, filesystem, whereas the darcs-2 
timings were made using repos on the same local filesystem.  In all cases I 
tried things a few times to let caches etc. fill up.

Can you repeat these?

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] [Revctrl] DARCS correctness question

[Simon Marlow]

David Roundy wrote:

> The conflict marking does depend on the order of changes in the repository,
> but this doesn't really matter, since conflict-marking is not fundamental
> to how darcs works.  It's something that's done to the working directory
> for the convenience of the user.  We could remove this feature and darcs
> would be just as correct (although rather more awkward to use).

Which reminds me - conflict markers are rather hard to use, because it's 
not clear which change comes from which patch, and unless I'm mistaken the 
order can vary from one file to another (perhaps even between hunks?). 
Also, you don't get to see the original version of the file before either 
patch.  I'd like to see something like

v v v v v v (original)
foo
=== (patch A)
bar
=== (patch B)
baz
=== (patch C)
wibble
^ ^ ^ ^ ^ ^

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] darcs patch: Libwww new API: waitUrl, copyUrlFirst added; copyUrls ...

[Simon Marlow]

David Roundy wrote:
> On Mon, Jan 21, 2008 at 08:03:22PM +0300, Dmitry Kurochkin wrote:
>> 2008/1/21, David Roundy <[EMAIL PROTECTED]>:
>>> Alas, I am now getting segfaults when I do a darcs pull over http with
>>> libwww.  :(  I get this pretty reliably if I do
>>>
>>> darcs get http://darcs.net/repos/unstable
>>> cd unstable
>>> darcs obliterate --last 1000 -a
>>> darcs pull -a
>>>
>>> (I've been running these slightly-crazy-length commands to test the
>>> progress code.)
>> I will try to reproduce and fix it.
>> BTW How can i build darcs with debug symbols?
> 
> I don't know.  :( I see that ghc has a -debug option, but am unsure what it
> does.

It links the program with a debug version of the runtime, which adds 
assertions and other sanity-checking, some debug output options (+RTS 
-D) and also has debug symbols.

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] [issue604] Windows builds of darcs-2 don't work for me

[Simon Marlow]

Zooko wrote:
> New submission from Zooko <[EMAIL PROTECTED]>:
> 
> Folks:
> 
> I tried the executables from
> 
> http://www.cs.mu.oz.au/~rgm/darcs/
> 
> but they don't do anything on my Windows XP system, not even pop up a  
> crash dialog.  "./darcs.exe --version", for example, returns silently  
> to the prompt.

This often indicates a missing DLL.  IIRC, the exit code is 128 in this 
case.  If so, try http://www.dependencywalker.com/ to find out what is missing.

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] darcs patch: Libwww new API: waitUrl, copyUrlFirst added; copyUrls ...

[Simon Marlow]

David Roundy wrote:
> On Wed, Jan 23, 2008 at 01:31:30PM +0000, Simon Marlow wrote:
>> David Roundy wrote:
>>> On Mon, Jan 21, 2008 at 08:03:22PM +0300, Dmitry Kurochkin wrote:
>>>> I will try to reproduce and fix it.  BTW How can i build darcs with
>>>> debug symbols?
>>> I don't know.  :( I see that ghc has a -debug option, but am unsure what it
>>> does.
>> It links the program with a debug version of the runtime, which adds 
>> assertions and other sanity-checking, some debug output options (+RTS 
>> -D) and also has debug symbols.
> 
> Ah good, that does sound like it could be a real help (particularly the
> debug symbols).  Perhaps we should add a --enable-debug option to our
> configure script, and maybe even make that the default for prerelease
> builds?

-debug slows things down: GC by as much as a factor of 2, and other things 
to a lesser extent.  Apart from that there should be no downsides.

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] announcing darcs 2.0.0pre3

[Simon Marlow]

David Roundy wrote:
> We are happy to announce the third prerelease version of darcs 2! Darcs 2
> features numerous improvements, and it seems that we have fixed most of the
> regressions, so we're looking for help, from users willing to try this
> release out. Read below, to see how you can benefit from this new release,
> and how you can help us to make the final darcs 2 release the best ever!
> 
> The third prerelease features (apart from numerous bug and performance
> regression fixes) a completely rewritten rollback command and new
> progress-reporting functionality. If darcs takes more than a couple of
> seconds for a given operation and provides you with no feedback as to what
> it's up to, let us know, so we can fine-tune the progress-reporting output!

The progress reporting is fantastic!  It's worth upgrading to darcs2 just 
for that :-)  Although the progress reporting doesn't appear to work quite 
as well on darcs-1 repositories as it does on darcs-2 repositories - is 
that expected?

There are still times when I see nothing happening, for example in the 
unpull test on the GHC repo (see previous messages), the last progress 
message I get is

Reading patches in /64playpen/simonmar/ghc-darcs2 17040

and it sits there for 7-8 seconds before completing.  Does this maybe shed 
any light on why this unpull is 2 times slower than darcs1?

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] next stable release?

[Simon Marlow]

zooko wrote:
> Dear Tommy Pettersson:
> 
> What remains to be done to make the darcs 1.0.10 release?
> 
> I've been switching more and more of my machines over to darcs-2, and  
> it occurs to me that now is probably a good time to release darcs  
> 1.0.10 and mention the upcoming darcs-2 in the release announcement.
> 
> I would be willing to help with darcs-1.0.10, at least in my  
> traditional role of making a zipfile with it plus putty executables  
> for Windows and Cygwin...

I have two requests, if there is to be a 1.0.10 release:

  - make it understand what a darcs2 repository looks like, and emit a
helpful error message explaining where to get darcs2 from.  Currently
the error is something like "not a repository", which is likely
to confuse people in the shift to darcs2.  Doing a new point release
is a good opportunity to ease the upgrade path to darcs2.

  - look carefully at 'darcs get --partial'.  When I last tried 1.1.0pre
on Windows it was getting a full repository.

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] New performance regression?

[Simon Marlow]

I just updated my darcs2 to try out the new ByteString changes and see if I 
could reproduce Erik's results, and I'm seeing a bit regression in the 
performance of unpull:

$ darcs2 unpull --from-tag=2007-09-25 -a
134.18s real   128.56s user   2.32s system   97% darcs2

previously this was ~15 seconds.  The performance of pull seems about the 
same.  I've tried it several times, and also done a 'darcs optimize' just 
in case.  Can anyone reproduce this?

There's a darcs2 repository of ghc here, BTW:

  http://darcs.haskell.org/ghc-darcs2

Incedentally, I'm doing a get on this repository right now, and darcs2 has 
been silent (no progress info at all) for a couple of minutes, even though 
I can see it downloading stuff... oh, now it says "Identifying repository".

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] New performance regression?

[Simon Marlow]

David Roundy wrote:
> On Thu, Feb 07, 2008 at 03:26:09PM +0000, Simon Marlow wrote:
>> I just updated my darcs2 to try out the new ByteString changes and see if I 
>> could reproduce Erik's results, and I'm seeing a bit regression in the 
>> performance of unpull:
> 
> I'll try to take a look at this tomorrow... I've spent my darcs time for
> the day working out zooko's performance regression.  Who knows, maybe I've
> fixed yours?  :)

I just updated and tried again, and now pull has regressed too, by an order 
of magnitude :-(

You should be able to reproduce this pretty easily with a darcs2 ghc 
repository.  I get no progress messages at all from pull for quite a while, 
and the 400-patch pull now takes over 100s where it previously took 10. 
Unpull is also taking about 100s on this machine, where previously it took 15.

This happens to be with --disable-bytestring, if that matters (probably 
not, I guess).

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] New performance regression?

[Simon Marlow]

David Roundy wrote:
> On Fri, Feb 08, 2008 at 10:03:52AM -0500, David Roundy wrote:
>> On Fri, Feb 08, 2008 at 10:53:57AM +0000, Simon Marlow wrote:
>>> You should be able to reproduce this pretty easily with a darcs2 ghc 
>>> repository.  I get no progress messages at all from pull for quite a while, 
>>> and the 400-patch pull now takes over 100s where it previously took 10. 
>>> Unpull is also taking about 100s on this machine, where previously it took 
>>> 15.
>> Oddly enough in my quick darcs2-format test, I don't see the slowdown, but
>> on the hashed-format test I do.  This may be some sort of hysteresis
>> effect.  I'll try using your darcs2 repository, but first I want to debug
>> the darcs get slowness (or at least lack of progress).  :(
> 
> Okay, I've got a data point:  darcs obliterate --last 580 -a is fast (well,
> the old speed) while darcs obliterate --from-tag ... -a is slow (where on
> my test case there are 580 patches after the tag).  So it seems to be in
> the patch selection code.

Indeed, I can confirm that.

> On the other hand, I still see now slowness in pull!  :(

It's very strange - certainly this morning I saw very long pull times in 
both directions between my two repositories, and now I'm only seeing it in 
one direction.

To summarise, with my two repos called A and B:

   oblit in either A or B slow with --from-patch, fast with --last
   pull A -> B  fast
   pull B -> A  *slow*

Ah, but I have clues.  repo A has some strange corruption.  Spaces in 
filenames have been replaced by (literally) \32.

$ \ls InstallShield
Component\32\DefinitionsREADME   Setup\32\Files
decyg.plRegistry\32\Entries  Shell\32\Objects
File\32\Groups  runexe.c String\32\Tables
Glasgow\32\Haskell\32\Compiler.ipr  Script\32\Files  Text\32\Substitutions

this is what it should look like:

$ \ls InstallShield
Component Definitions READMESetup Files
decyg.pl  Registry Entries  Shell Objects
File Groups   runexe.c  String Tables
Glasgow Haskell Compiler.ipr  Script Files  Text Substitutions

I have to go now so I can't investigate any more, but my best guess is that 
this happened when I did 'darcs repair' yesterday using a darcs2 that was 
built with bytestring support turned on.

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] New performance regression?

[Simon Marlow]

David Roundy wrote:
> On Thu, Feb 07, 2008 at 03:26:09PM +0000, Simon Marlow wrote:
>> Incedentally, I'm doing a get on this repository right now, and darcs2 has 
>> been silent (no progress info at all) for a couple of minutes, even though 
>> I can see it downloading stuff... oh, now it says "Identifying repository".
> 
> This is rather a tricky scenario:  the hashed_inventory of this repository
> takes 36s to download on my computer using wget.  Our current assumption in
> the progress-reporting code is that any given download will be fast, and
> the result is that this particular operation is going to take a minimum of
> 36 seconds with no change in progress output.  As it turns out, we download
> hashed_inventory twice when we only need download it once, so that slows
> things down dramatically.
> 
> I can easily get rid of one of these downloads (which is just checking the
> format of the repository, something that can more easily be done with a
> check of _darcs/format, which always exists for hashed and darcs-2
> repositories).  I'd actually also like to cache the contents of
> hashed_inventory, in a way that would enable us to guarantee that it's only
> downloaded once, which is good both for efficiency and for atomicity (to
> make sure we don't use two versions of the remote repository but assume
> they're the same).
> 
> I'm testing (and planning to push) a patch now that'll cut 36 seconds on
> this get for me, by avoiding one download of hashed_inventory, but I don't
> see how to avoid a 36s wait with no new progress report without very
> dramatically rewriting our libwww/libcurl bindings.

Ok, sounds reasonable.  But perhaps there should be a progress message 
before the download starts, something like "getting 
http://darcs.haskell.org/ghc-darcs2/_darcs/hashed_inventory...";?

Cheers,
Simon

___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] [issue700] darcs2 cannot check the ghc-darcs repository

[Simon Marlow]

New submission from Simon Marlow <[EMAIL PROTECTED]>:

I built a darcs-unstable from today's sources (18 Feb 2008), and it hangs 
eating all my memory whan I try to 'darcs check' a GHC repository (darcs2 
format).

The reason I'm trying this is because I have a darcs2 from a couple of 
weeks ago built with bytestring support, that fails to 'darcs check' the 
GHC repository, claiming an inconsistency.  I'll report this separately, as 
the issue may still exist.

--
messages: 3530
nosy: beschmi, droundy, kowey, simonmarhaskell, tommy
status: unread
title: darcs2 cannot check the ghc-darcs repository

__
Darcs bug tracker <[EMAIL PROTECTED]>
<http://bugs.darcs.net/issue700>
__
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

Re: [darcs-devel] [issue659] Release Darcs 2.0

[Simon Marlow]

Mark Stosberg wrote:
> 
> Here's the report I made for the Darcs 2 release-critical issues. I used the
> feature to save it so it appears in "Your Queries" in the sidebar. 
> 
> It shows that 2 of the 7 items on the list are resolved now. 
> 
> http://bugs.darcs.net/[EMAIL 
> PROTECTED],id,activity,status,assignedto&@sort=status&@group=priority&@filter=topic&@pagesize=50&@startwith=0&topic=24&@dispname=Fix%20For%20Darcs%202.0

How about 699?  Unless I'm doing something completely bogus (don't think 
so), darcs2 has developed a new performance problem recently.

What's the status of the bytestring support?  I believe it has a nasty 
problem with filenames containing spaces, but I'm stuck testing that due to 
issue 699.  Maybe if someone has a spare moment they could play around with 
some filenames with spaces in and see if they can provoke any problems - 
just a 'darcs check' was enough to show a problem for me, and 'darcs 
repair' left me with a corrupt repository.

Cheers,
Simon
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel

[darcs-devel] [issue706] Filenames with spaces issue

[Simon Marlow]

New submission from Simon Marlow <[EMAIL PROTECTED]>:

I mentioned there might be a problem with filenames containing spaces.  I 
just tried a few things and managed to reproduce some strange behaviour.

'darcs' is darcs 1.0.9
'darcs2' is darcs-unstable pulled yesterday

$ mkdir foo
$ cd foo
$ darcs init
$ touch 'A B'
$ darcs add 'A B'
$ darcs rec -a
What is the patch name? fo
Do you want to add a long comment? [yn]
$ ls
A B  _darcs/
$ darcs check
Applying patch 1 of 1... done.
The repository is consistent!
$ ~/darcs/darcs-unstable/darcs check
The repository is consistent!
$ cd ..
$ ~/darcs/darcs-unstable/darcs convert foo foo2
Finished converting.
$ cd foo2
$ ls
A B  _darcs/
$ ~/darcs/darcs-unstable/darcs check
Looks like we have a difference...
Difference:  rmfile ./A

Inconsistent repository!
zsh: 20274 exit 1 ~/darcs/darcs-unstable/darcs check

This behaviour is *not* fixed by recompiling with --disable-bytestring, 
incedentally.

Also, darcs2 fails to check the GHC darcs2 repository, giving this error:

rmfile ./WindowsInstaller/Glasgow\92\32\92\Haskell\92\32\92\Compiler.ism

Inconsistent repository!

Cheers,
Simon

--
messages: 3663
nosy: beschmi, droundy, kowey, simonmarhaskell, tommy
status: unread
title: Filenames with spaces issue

__
Darcs bug tracker <[EMAIL PROTECTED]>
<http://bugs.darcs.net/issue706>
__
___
darcs-devel mailing list
darcs-devel@darcs.net
http://lists.osuosl.org/mailman/listinfo/darcs-devel