Claimer: the following is more or less rambling, without necessarily a specific point.
On Sat, Oct 17, 2009 at 7:27 PM, Ralph Versteegen <teeem...@gmail.com> wrote: > (ED: I originally wrote the following in November 2008, but never sent > it. Finding it in my draft emails folder, I've dusted it off and am > sending it for amusement.) > > FINE THEN. Look what you made me do! I wrote a utility to export > attack data to xml just to end up something I could test! I took the > dt6 page off the wiki, wrote a crude script to process it, added lots > of hand editing, and wrote a program around it. Near the end I > realised that xml doesn't allow spaces in tag names, so I renamed the > output format to NOTXML. > > I also realised that I could probably have used the code for the > actual flexmenu attack menu code which specifies the data in the lump > in the nearly same way, for a really-good time saving. > > Here's an example attack: > > <attack id="146"> > <Animation picture>39<\Animation picture> > <Animation palette>52<\Animation palette> The usual solution here is like <AnimationPalette>52<\AnimationPalette> (CamelCase?) Which seems to work quite well, in conjunction with readable XML indentation. Please note, however, that I am not in favor of XML for data storage; as I've observed, it's a Markup Language, not a Data Language. JSON is a fairly nice fit; YAML is near ideal but I realize that it's much more involved to wrap or implement. > <Target Class>Ally excluding self<\Target Class> > <Target Setting>Focused<\Target Setting> > <Damage Equation>No Damage<\Damage Equation> > <Aim Math>Never misses<\Aim Math> > <Base Attack Stat>Atk<\Base Attack Stat> > <MP Cost>30<\MP Cost> > <Attack Animation>Projectile<\Attack Animation> > <Number of hits>1<\Number of hits> > <Target Stat>HP<\Target Stat> > <bit>Element 1 Damage<\bit> > <bit>Cannot target enemy slot 1<\bit> I would expect these to be like element-1-damage cant-target-eslot1 for memorability (no care needed for letter case) > <bit>Cannot target enemy slot 2<\bit> > <bit>Cannot target enemy slot 3<\bit> > <bit>Cannot target hero slot 1<\bit> > <bit>Cannot target hero slot 2<\bit> > <bit>Cannot target hero slot 3<\bit> > <bit>Cannot target hero slot 4<\bit> > <bit>Store Target<\bit> > <bit>Automaticaly choose target<\bit> > <bit>Show attack name<\bit> > <Name>FireBurst<\Name> > <Caption Display Time>30<\Caption Display Time> > <Caption>Cloak Combustion: Now for judgment...<\Caption> This is another reason why this is not XML: OHR Strings are not ascii. Of course you can escape them (I took this approach with the YAML support in NOHRIO). > <bit>Mutable<\bit> > <\attack> > > I could have grouped related items like tag id/condition/check and > damage options, but didn't care to go further. > > > So here's the results for Adventures of Powerstick Man (158 attacks): > DT6 + ATTACK.BIN: 31916 bytes > DT6 + ATTACK.BIN compressed (.zipped using Windows XP which isn't very > good, but that's what most users would probably use): 4254 bytes You may have shot yourself in the foot here somewhat because using zlib you don't have to depend on the user to do anything with subpar utilities -- you can get something pretty close to gzip -9 compression level with no particular effort. With zlib and an XML format, you can do streaming decompression (which can help when searching). With a 'collection of records' paradigm like OHR, it may be smart to compress each record individually (then you'd be able to load it fast.. and with a bit of thought, it even helps you save it fast..) At the moment I think SQLite is a fairly good fit for OHRRPGCE, including for a save format, since it matches the record-based, fairly optimized approach, while being self-describing (and particularly, being relational -- with the amount of cross-referencing of resources that goes on in OHRRPGCE data files, the automatic verification that they actually are pointing somewhere valid is helpful. this is planned for the next significant version of SQLite, though. Had I always the absolute choice of data format/framework, I would always use either HDF5 or SQLite for 'binary' (large volume, predefined fields and no others), preferring HDF5 for its 'attach-arbitrary-metadata-which-can-itself-be-structured' capability; and YAML for 'textual' (structurally varying; large amount of typically-omitted data (ie. fields whose value is implied by their absence)) data. > powerxe_attacks.notxml: 103127 bytes > powerxe_attacks.notxml compressed: 6727 bytes > > Hey, what! That wasn't fair! I just noticed that most of the attack > lumps are actually taken up by 3 string fields: name, description and > caption. Most attacks in most games (unlike powerxe) have 2 of these > blank, and most data fields are left to defaults. > > > Wandering Hamster (151 attacks): > DT6 + ATTACK.BIN: 30,502 bytes > DT6 + ATTACK.BIN compressed: 5,397 bytes > wander_attacks.notxml: 97,922 bytes > wander_attacks.notxml compressed 7,353 bytes > > Gerania (22 attacks): > DT6 + ATTACK.BIN: 4,444 bytes > DT6 + ATTACK.BIN compressed: 1,183 bytes > ohrrpgce_attacks.notxml: 16,068 bytes > ohrrpgce_attacks.notxml compressed 1,638 bytes > > Vikings (213 attacks): > DT6 + ATTACK.BIN: 43,026 bytes > ? > viking_attacks.notxml: 162,363 bytes > viking_attacks.notxml compressed: 10,690 bytes > > > (ED: This is where I was going to write a message conceding defeat to David. Thank you. I'd already done several tests on this over the years.. I've basically concluded that the savings are analogous to the difference between a set of PNGs encoding a movie and an AVI, FLV or whatever movie format encoding the same thing -- you can get significantly better compression with binary formats, but only if you are willing to sacrifice relaxedness (ie. it makes your code 'tense' -- highly tuned/dependent on small factors) Some general thoughts on appropriate behaviour WRT data formats: * Be as relaxed as you can reasonably be (eg. low-volume things in a textual, non-strict format like YAML or JSON, high-volume things in the simplest self-describing binary format you can choose) * Avoid format proliferation (eg. using PTx rather than PNGs for sprite frames/framesets) in order to maximize inspectability. This usually means accepting some overhead (obvious example is SQLite database versus pure data) * maximize orthogonality (eg. the fact that the HS lump is of the same format as an RPG file is a win; the fact that old record-based lumps tend to have BLOAD headers while new record-based lumps don't is a loss.). Self-descriptive formats really help this. I think a self-descriptive format is almost vital for the new save game format. > > The utility is attached in case you want to play around with it, but > it's not meant to be useful, or instructive... but it wasn't hard to > make the code simpler than the attack flexmenu :P! So it might be > interesting.) May have a look whenever I get a sane FB install. Or possibly hack up something that works similarly based on the dictionary format currently used for generic access and YAML I/O in NOHRIO. _______________________________________________ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
