On Fri, Aug 14, 2009 at 12:29 AM, Ralph Versteegen<teeem...@gmail.com> wrote:
> 2009/8/11 David Gowers <00a...@gmail.com>:
>> With James' permission, I've created a git import of OHRRPGCE
>> subversion, mainly because development of ambitious features such as
>> are found in
>> http://gilgamesh.hamsterrepublic.com/wiki/ohrrpgce/index.php/Plan_for_dynamic_types_in_HamsterSpeak
>> benefit a lot from inexpensive, pain-free branching such as is
>> available in git (and well, git is just nicer to use by about 3 orders
>> of magnitude :)
>>
>> Details can be found here:
>> http://gilgamesh.hamsterrepublic.com/wiki/ohrrpgce/index.php/Git
>>
>>
>> In other news, I am releasing my project 'nohrio' now!
>> This is the (Python package) project to convert OHRRPGCE data between
>> occasionally-bizarre-binary-format, YAML, and Python (dicts +lists).
>> The API is far from final but it's effective.
>>
>> URL: http://gitorious.org/nohrio
>> browse git repository: http://gitorious.org/nohrio/nohrio/trees/master
>>
>> Further info is in the README file (accessible through the git repo link
>> above)
>>
>> Sample YAML showing the human-editable format is available in
>> tests/milestone1.yaml
>> (http://gitorious.org/nohrio/nohrio/blobs/master/tests/milestone1.yaml
>> )
>> It's designed so that it's possible to unambiguously mix records of
>> different binary dtypes in the same YAML document.
>>
>> Currently, dtypes are found in the ohrrpgce module -- that is,
>> nohrio.ohrrpgce.dtypes
>> you can, for example, memmap something with the help of nohrio.ohrrpgce.mmap,
>> then dump part or all of this array to YAML using dump (array2py(arr))
>> (dump and array2py are from nohrio.ohryaml module)
>>
>> Dumping is currently restricted to a few dtypes, but that is
>> essentially because of something very simple: I haven't yet put the
>> NAME: dtypename aliases into the nohrio.ohryaml.lookup dictionary.
>> Patches to improve this situation would be appreciated; if you just
>> want to try out dumping some particular type, it's dead easy to do.
>> (see http://gitorious.org/nohrio/nohrio/blobs/master/nohrio/ohryaml.py#line90
>> )
>>
>> Other candidates for improvement are:
>> * milestone1.yaml doesn't contain tests for all formats, only many of
>> them. This is pretty easy to do; you use mktest to generate most of
>> the entry, and use mmap, dump and array2py to generate the actual
>> records. I should put up an ipython log showing how that works in
>> detail)
>> * api should be nicer. In particular, we should be able to very easily
>> get a memmap or normal array by asking for a particular dtype and
>> slice of records by name; and converting records in an OHRRPGCE binary
>> file to YAML or python should also be dead easy (one step)
>> * SAV support
>>
>> There are all sorts of interesting uses to put this to. for example,
>> once YAMLized, you can use diff tools to see exactly how records
>> differ (and use merge tools like xxdiff to do interactive merging!);
>> you can find records by any field or by complex criterion..
>>
>> I await your comments :)
>
> This is not a full response, just some quick feedback (and also, I'll
> get around to asking about git as well):
>
> After failing miserably to understand its use for an hour, I finally
> found array2py, which I initially overlooked because I thought it had
> something to do with yaml. I assume that you will want to run all
> memmaps through this to get a directly usable object.
No; an array is directly usable. It just can't be translated to yaml.
It was never the intent to manipulate large amounts of objects using
yaml -- the array interface is close to 95% efficient, and is more
powerful for simple manipulations.
I can see I need to provide more documentation -- and perhaps a better
README. I'll work on that.
Array2py needs a better name. what it does is translate between an array and
{python lists and dicts (with appropriate scalar valuess in them)},
which is easy to serialize to YAML. Hence it's in ohryaml.py
> Then why put it
> in ohryaml.py? However, I immediately ran into a problem: array2py
> takes forever on big lumps, so I guess I need a proper wrapper.
big lumps? lumps that have many records, or lumps where each record is
big? (or lumps with a complex dtype?)
(logs are a big timesaver here -- consider using
http://ipython.scipy.org and turning logging on so you can post logs
of what you did.)
A predicted typical usage pattern does not involve translation of
large amounts of records. Clearly this is inaccurate, can you tell me
what exactly you were aiming to do and how you went about it?
>
> Is 'old-ohrio-tests.rst' still any use as documentation? That's what I
> attempted to read. Also, is it necessary to use linearize and
> spliceattack on an RPG before being able to load those files?
old-ohrio-tests.rst is only really useful as documentation on things
like starting offsets (BLOAD etc).. once knowledge of that is
incorporated in nohrio, old-ohrio-tests.rst will go away.
Currently nohrio only accepts linearized data (in cases where it was
planar) and nohrio accepts only spliced attack data.
This is mainly to do with issues relating to representing a single
record as a single record (rather than 200 x, 200 y, etc). In future
it will probably happen transparently
>
> I'll also mention that I had problems with installing it: I ended up
> with a nohrio-0.33dev-py2.6.egg folder in my site-packages, which was
> empty aside from EGGINFO and nohrio subdirectories. Python wouldn't
> import ohrrpgce until I moved everything in nohrio up a directory. I
> don't know a thing about installing python packages.
In that case, I'll guess that you didn't have setuptools (or it's fork
Distribute) installed (as the directory structure you describe is
normal and standard). Setuptools is fairly standard these days.
http://pypi.python.org/pypi/setuptools/
I'll need to add that to the dependencies; it never really occurred to
me that someone might not have it.
Thanks for all the feedback :D
David
_______________________________________________
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
