I'm gonna push these tomorrow unless someone objects.
On Tuesday, September 23, 2014 05:55:47 PM Dylan Baker wrote: > For a long time piglit has had a considerable problem with resuming > interrupted runs. This problem comes from the recovery tool, a very > fragile piece of code that relies on the formatting of the json to walk > backwards throwing away code. The problem with this approach are two > fold, first if the indentation changes the tool breaks. Second, it can > only handle corruption at the end of the file. > > There is a second problem with our resume method, and that is that the > json file is read, truncated, and completely re-written on every resume, > this can be quite expensive. > > Initially I began writing a complicated tool that actually could parse > json and clean it, but this quickly became a significant amount of code, > and the complexity was overwhelming, it was a complete project in > itself. > > What I have implemented is a much simpler idea, that I borrowed from > databases, Atomic writes. By atomic we mean that a transaction should > either completely succeed or completely fail. Using a file backend there > are two basic ways to go about doing this. One method Copy the file to a > temporary location, write to the temporary file, verify the results, and > copy it back over the original. This has a couple of distinct drawbacks. > First it requires a lot of copying, second it means that the file needs > to be locked for long periods, and finally it requires reading the file. > > What I implemented instead is a system that writes each test to it's own > file using built-in methods for both json and xml. At the end the runner > reads in each of these files, and creates a combined file from the > results. This has advantages and drawbacks. Because each test gets it's > own file there is no locking required at all, and flushing and syncing > is cheaper. It also means that there is no need to verify writes, when > the final file is being assembled json.load and xml.ElementTree.parse > are called on each file, and if they fail the result is considered > failed and ignored. > > This series also takes a few more steps toward abstracting backends, by > creating a separate package for backends, and including a small subset > of the classes and functions from the package in the __init__ module, > allowing them to be accessed as if backends itself was a module. This is > sufficient for most applications, with mainly unit tests needing to > bypass this and access the modules directly. > > Patch 1 removes a broken executable script that has been broken for some > time > Patch 2 does most of the abstraction > Patches 3-6 do some code cleanup > Patches 7 and 8 implement atomic writes in the two backends > Patch 9 does some additional refactoring that is possible becasue of > patches 7 and 8 >
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit