On 21 September 2011 21:02, Dave Tapley <duked...@gmail.com> wrote: > On 21 September 2011 17:33, Jeremy O'Donoghue > <jeremy.odonog...@gmail.com> wrote: >> Hi Dave, >> >> On 20 September 2011 17:58, Dave Tapley <duked...@gmail.com> wrote: >>> >>> On 16 September 2011 23:30, Dave Tapley <duked...@gmail.com> wrote: >>>> >>>> I presume everyone has a very long compile time when building wxcore? >>>> Specifically rebuilding everything under src/cpp/ every time.. >>>> >>>> Has anyone ever looked into avoiding this complete rebuild? >>> >>> I've just spent a few hours looking deeper in to this and come across two >>> issues: >>> >>> 1. There is a very informative blog post[1] written by Jeremy, which deals >>> with the subject of "Compiling C or C++ code from within Cabal". >>> Unfortunately I can't find any of the code mentioned in the post in the >>> project, specifically I tried to find a "myBuildHook" in "./wxcore/Setup.hs" >>> (I also looked in previous revisions using a trackdown grep[2]) but I didn't >>> find anything. Perhaps someone with better knowledge of the project can >>> comment on if/where/when the code in the post was used? >> >> The code sits on my development machine. It works, but with a fairly serious >> limitation. >> >> The limitation is that I do not do any dependency tracking. It works by >> checking if an object file is older than its corresponding source file. This >> works fine for .cpp files but breaks if you change a header. Short of >> writing a complete dependency tracker, this is hard to fix, and in truth I >> think it belongs in Cabal. >> >> If Cabal wants to say that it can compile C/C++ code, it should be the one >> to do so correctly, especially as dependency tracking for C/C++ is vile and >> compiler dependent(*). > > Firstly welcome back, and thanks for taking the time to reply :) > > Ah, this all makes more sense now. > Do you know if there is any documentation on cabal's C/C++ compiling > abilities? > I still have no idea how the "confHook" stuff actually results in code > being built, because I don't see any reference to the C++ code (i.e. > wxcore/src/cpp/) in Setup.hs, only the calling of 'wxdirect' to > generate them, and use of 'libBuildInfo' to set up GCC. > >> >> I would be very happy to put the code out there as a GitHub gist or similar >> - it's only in one or two files, and quite easy to follow, but I don't feel >> it is ready for prime time due to the limitations noted above. > > Well, for me, the compile time of the C++ component is around four > minutes, which proves to be a real pain when you miss one semi-colon. > I'd certainly like to give your code a go, and I shall do so whilst > heeding your warning :) > >> >> (*) one option might be to do this only for GCC, as in practice we only >> really support GCC anyway. >> >>> 2. Inspecting the wxdirect code you can see that "System.IO.writeFile" is >>> used to write all the generated code[3], but no test is performed to see if >>> the output file has actually changed. Thus the file is always opened for >>> write, and so its modification time is changed, and so everything is >>> recompiled every time wxcore is built. I have have written a local patch >>> which replaces the "writeFile" function with one which first checks whether >>> the string to be written differs (aside from date/time stamp) to the current >>> one; it only performs the "writeFile" if there has been a change. >>> Using this patch none of the Haskell code is re-built, but unfortunately >>> all the C++ code is. >> >> This is a nice patch - I think we should apply it. > > Excellent, well at the moment it's a rather 'quick' implementation, > comprising of the following function: > writeFileIfRequired :: FilePath -> String -> IO () > writeFileIfRequired f str = readFile f >>= evaluate >>= \str' -> when > (not $ and $ zipWith isSame (lines str) (lines str')) (writeFile f > str) > where isSame l1 l2 = ("UTC" `isInfixOf` l1) || ("UTC" `isInfixOf` > l2) || (l1 == l2) > > You'll note the the use of ("UTC" `isInfixOf`), this is my nasty hack > to get around the fact that the time of generation is written in to > the file, and this obviously changes every time. Unless anyone has a > good reason not to, I'd like to suggest we remove this "generated > on..." lines (but leave in the "don't edit this" blurb), if someone > really wants to know they can always check the file modification time? > > Dave, > >> >> Jeremy >> >
Hmm, I was just revisiting this and happened on the following: The Setup.hs for wxcore uses "simpleUserHooks", overriding only "confHook". However there is also "cleanHook", which is defined by simpleUserHooks to be: cleanHook = \p _ _ f -> clean p f, If you consult the source for clean[1] you'll see that it tries to "remove the whole dist/ directory rather than tracking exactly what files we created in there". I presume that's why we have to do a full re-build every time? To try and circumvent this I modified the definition of "main" in Setup.hs to this: main = defaultMainWithHooks simpleUserHooks { confHook = myConfHook, cleanHook = (\_ _ _ _ -> return ())} Unfortunately it still seems to re-build all the C++ on each 'install' from cabal. Not sure why? Dave, [1] Taken from http://www.haskell.org/ghc/docs/6.10.4/html/libraries/Cabal/src/Distribution-Simple.html#simpleUserHooks -- Cleaning clean :: PackageDescription -> CleanFlags -> IO () clean pkg_descr flags = do let distPref = fromFlag $ cleanDistPref flags notice verbosity "cleaning..." maybeConfig <- if fromFlag (cleanSaveConf flags) then maybeGetPersistBuildConfig distPref else return Nothing -- remove the whole dist/ directory rather than tracking exactly what files -- we created in there. chattyTry "removing dist/" $ do exists <- doesDirectoryExist distPref when exists (removeDirectoryRecursive distPref) -- these live in the top level dir so must be removed separately removeRegScripts -- Any extra files the user wants to remove mapM_ removeFileOrDirectory (extraTmpFiles pkg_descr) -- If the user wanted to save the config, write it back maybe (return ()) (writePersistBuildConfig distPref) maybeConfig where removeFileOrDirectory :: FilePath -> IO () removeFileOrDirectory fname = do isDir <- doesDirectoryExist fname isFile <- doesFileExist fname if isDir then removeDirectoryRecursive fname else if isFile then removeFile fname else return () verbosity = fromFlag (cleanVerbosity flags) ------------------------------------------------------------------------------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 _______________________________________________ wxhaskell-devel mailing list wxhaskell-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wxhaskell-devel