On 14 May 2012 18:36, David Gowers (kampu) <00a...@gmail.com> wrote: > Hi TeeEmCee, > > On Mon, May 14, 2012 at 3:37 PM, Ralph Versteegen <teeem...@gmail.com> wrote: >> I've been working on a crude nohrio program to import and export data >> (enemies, heroes, attacks, items, textboxes) to csv files. It's >> totally generic, working by inspecting the fields of the lump's dtype, >> but there weren't any other interesting lumps to handle (aside from >> shop items, but nohrio can't memmap .stf directly because of the odd >> length). > > I think that must be a Windows-only constraint. I can memmap with any > offset and any length.
I forgot to try manually specifying the length. Wow, numpy lets you grow files by writing off the end of a memmap? I didn't know; that's quite nice. But RPGDir needs a special case for that. It seems LibreOffice can't handle spreadsheets with 1300 columns though! Would need to change the format to have 'shop id' and 'item number' key columns. >> >> What I learnt is that this is totally the wrong approach, and it needs >> to be rewritten. You really need to write lump-specific code which >> knows how to handle things like enumerations, off-by-ones, arrays that >> are split into two pieces, and the like. > > Yeah, that's what I decided. A more conventional record interface > implemented in Python, which uses ndarray subclassing to provide > enumeration, badstring, off-by-one, and split record handling. It > needs to address two distinct needs : arr.fieldname needs to > behave like `recarray`, but it should determine the subclass of the > returned array from the dtype metadata associated with the field. I was trying to do that by subclassing np.void and returning instances of that from OhrData.__getitem__, because IIRC even recarray records return np.void instances instead of np.records if you index a compound subfield of a record (eg. stat 'bonuses' in ITM). But I didn't get it working. > To that end I started working on improving the datatype definitions > (we can store such additional info in the 'metadata' field of the > dtype object.). The format is compilable (ie. you can produce actual > dtypes with metadata attached from it, and vice versa), but incomplete > (off-by-one isn't handled. not sure if badstring is handled). > > I thought I'd pushed this work, but no.. Well, that's remedied now. This looks a lot like what nohrio needs, though the encoding is a bit too compact imo :) Originally I wanted to create a metadata file format similar to editedit definitions, except describing binary record (and RELOAD) based file formats, and try to split that off from editedt menu definitions (which would then specify much less: mostly just the order in which fields are presented and conditions on each appearing/being enabled). But clearly this would just create a whole lot of work, so I'm not so keen on it anymore. >> I think nohrio itself should >> know what's in a data field, rather than just the dtype. For example, >> I even had to recognise strings by their dtype. My eventual goal for >> nohrio is to do better, and wrap all lumps in classes (preferably >> ndarray subclasses) to cleanup the interface. >> >> So little of this code has any use, aside from the csv >> handling/escaping functions. However, just as I finished, I discovered >> that Python actually has a module for CSV files. That's to be >> expected, but I didn't even think to look. >> >> I did however fix several nohrio bugs and implement missing >> functionality, though. >> >> http://gitorious.org/~tmc/nohrio/tmc/commits/rpgbatch > > Cool, I'll check that out when I have time.. Right now, I'm both sick > and caught up in moving. _______________________________________________ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
