Re: [Ohrrpgce] SVN: james/5480 Attacks base stat is now allowed to be max stats,
I thought the meaning of 'splitting the attack stat' was to reduce the number of enumerated values; to have only two related options in base stat, 'attacker stat' and 'target stat', with the selection of which stat to use in a separate field. On Sun, Dec 9, 2012 at 12:48 PM, James Paige wrote: > I am glad you like it :) > > I am confused about what you mean by splitting the attack stat. Can you > explain? > > As for increasing the number of stats, I was thinking that switching to > reload format would help. That way we could store base attack stat in > some slightly more structured way, rather than a list of sequential > magic numbers > > --- > James > > On Sat, Dec 08, 2012 at 11:24:34PM +1300, Ralph Versteegen wrote: > > The base stat browser menu is neat! > > > > A couple months ago I discussed improvements to damage calculation and > > couldn't decide on handling max stats. I'm happy with the way you've > > done this (though I think I'll split the attack stat up into two > > field, eg. "Attacker Max Stat" and HP. That'll make it easy to > > increase the number of stats in future.) > > > > On 8 December 2012 08:13, wrote: > > > james > > > 2012-12-07 11:13:46 -0800 (Fri, 07 Dec 2012) > > > 90 > > > Attacks base stat is now allowed to be max stats, > > > or the targets stats (both cur and max) > > > --- > > > U wip/bmodsubs.bas > > > U wip/flexmenu.bas > > > U wip/testgame/turntest.rpg > > > ___ > > > Ohrrpgce mailing list > > > ohrrpgce@lists.motherhamster.org > > > http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org > > ___ > > Ohrrpgce mailing list > > ohrrpgce@lists.motherhamster.org > > http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org > > > ___ > Ohrrpgce mailing list > ohrrpgce@lists.motherhamster.org > http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org > ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New nohrio RELOAD api feedback
On Mon, Oct 1, 2012 at 1:46 AM, Ralph Versteegen wrote: > I see. But what would this stream api actually look like (just when > and next)? I think either iterating, indexing by child name, or elif > chain would comfortably and naturally handle all the document > structures we currently have (would have to review though). Well, the idea is that it wouldn't look like anything -- you could just node['subnodename'] and expect to get subnodename if it's available at the current level. ie. no actual difference in API (aside from having to ask for streaming when you open the document), just a difference in whether the loading is lazy or not. when() is probably going to go away in favor of the elif-chain pattern you pointed out. next() is actually just a standard Python iterating function, which in most cases you wouldn't want to call manually (but maybe in some cases where nodes are big you would.) (In Python3. to manually get the next item from an iterator, you use `next(iterator)`. In Python2 it was `iterator.next()`) .. Ah, so basically like I said -- no additional API. I just scrubbed all the remaining streaming API :) I'll start working on implementing this whole API soon if no other points of clarification come up, cause it's hitting all the right spots -- simple both in front and backend, unsurprising, easy things are easy, complex things are possible. Yeah, I'm feeling pretty happy with it :) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New nohrio RELOAD api feedback
On Sun, Sep 30, 2012 at 12:20 AM, Ralph Versteegen wrote:
> On 29 September 2012 18:23, David Gowers <00a...@gmail.com> wrote:
>> Good idea CCing the list, I initially thought it was a topic of
>> limited general relevance, but I realized at least Mike might also
>> have something to say here..
>>
>>
>> On Sat, Sep 29, 2012 at 2:31 AM, Ralph Versteegen wrote:
>>> On 29 September 2012 00:38, David Gowers <00a...@gmail.com> wrote:
>>>
>>> I found that doing seeks on a file while writing to it is very slow on
>>> Windows XP; apparently the libc implementation flushes its cache when
>>> you seek. glibc doesn't do this. So I actually wrote a file write
>> ... Wow, they like to punish databases?. I'd guess that nobody hosts
>> websites on WinXP machines, but that'd be a wrong guess >_<.
>
> I'm sure professional databases don't use libc buffered files for IO.
Point. Your clarification below makes the policy seem a bit saner.
Flushing disk cache on seek is a lot crazier than writing buffered
data on seek.
>
> (By "flush" I mean sending the buffer to the OS (a write() call). OS
> call overhead is really large though. The a few weeks ago I timed
> write, read, and lseek, and found they took about 2us, 1us, 0.5us
> respectively on my 64bit dual core Linux 3.2 machine. I also compiled
> the test as 32 bit to compare, but can't remember the difference.)
>> Anyway I'm curious, so I'll write a PlanarRecordManager subclass that
>> caches the entire content of the relevant super-record, then they can
>> be swapped in and out to compare.
>
> What's a super-record?
Just a way of referring to the next-dimensional record index.
[0] -> N records
[1][2] -> single record, the third record in the second subrecord.
It may come up in relation to sprites, since some of PTx have
2-dimensional record indexing:
(spriteset, frame).
That still has to be worked out.
It was going to be used for map indexing too, but I realized that was silly.
Anyway due to changes of plan, PlanarRecordManager is destined for the
bitbucket.
I'll just load the entirety of the planar lumps at once, this
minimizes silliness since they're small.
>
> Anyway, I advise against premature optimisation. The buffering I
> implemented was probably a waste of time, as we aren't expecting any
> 30MB RELOAD lumps. Writing incrementally rather building the whole
Yeah, I saw 'INT' as the size type on the RELOAD wiki page when I was
reviewing it again, and I thought.. "doesn't that mean RELOAD lumps
are limited to 64k?.. Actually, that's kind of good.". But of course
it doesn't, the RELOAD page is just inconsistent with everything else
on the wiki by calling a 32bit signed integer an INT, and the link to
Formal Specs just confuses things further.
(I've just fixed it.)
> But you seem to be saying that there is an obligation to close nodes
> using either close() or with? If so I'm complaining that it's
> unneeded.
There's no requirement to (also like filelike objects). I just will
find the code harder to read if they're missing.
Though why you wouldn't use 'with' blocks when they help in multiple
ways, I don't know.
>>
>> statsby = parentnode.when('stats_by')
>> if statsby:
>> # do stuff with statsby
>
> Wouldn't an elif chain be simpler?
You're right. Even though I want to annihilate long elif chains on
general principle.
>> * precache node info on the current level. This allows to process
>> nodes 'out-of-order' -- if you request a node that is after the
>> 'current' node, you can still get it.. and process the other one
>> whenever it occurs in your code.
>
> So basically this would be an optimisation for code looking like...
>
> for foonode in parentnode.iter('foo'):
> #...
> for barnode in parentnode.iter('bar'):
> #...
It wasn't intended for that, but it would help in that case too.
It was intended for the case where.. say you're scanning for multiple
simple nodes -- flag nodes are an apt example --, and they occur but
not in the expected order. It would be inane to raise an error just
because what you expect is not what you immediately get, esp. since
simple nodes are usually tiny in size.
>
> Hmm, maybe you could tack something like the following onto the end of that:
>
> for node in parentnode.iter_unhandled():
> print("Unexpected node " + str(node))
I like this!
I'd like to make it a decorator.
@show_unhandled
def load_hero(rootnode):
# do stuff
# unexpected nodes are automatically printed,
# with the added context of the name of the function that
# they were found with, when load_hero() finishes.
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New nohrio RELOAD api feedback
Good idea CCing the list, I initially thought it was a topic of
limited general relevance, but I realized at least Mike might also
have something to say here..
On Sat, Sep 29, 2012 at 2:31 AM, Ralph Versteegen wrote:
> On 29 September 2012 00:38, David Gowers <00a...@gmail.com> wrote:
>
> I found that doing seeks on a file while writing to it is very slow on
> Windows XP; apparently the libc implementation flushes its cache when
> you seek. glibc doesn't do this. So I actually wrote a file write
... Wow, they like to punish databases?. I'd guess that nobody hosts
websites on WinXP machines, but that'd be a wrong guess >_<.
> buffering layer used when writing RELOAD documents to avoid this.
> Loading RELOAD documents was also considerably slower in Win XP than
> in Linux; I don't why, but it was still perfectly acceptable.
I'm currently researching this, as far as I know Python does some
buffering on its side of things, so how much of an impact this has
needs to be determined. My net access is kind of FUBAR, I'll get there
eventually though.
...
Looks like Python definitely does do its own buffering:
http://docs.python.org/library/io.html#io.BufferedIOBase
The only reference I've been able to find to slow seek() in python is
this one guy who was performing the inanity:
for i in range(100):
f.seek(0)
/-_-\ .
Anyway I'm curious, so I'll write a PlanarRecordManager subclass that
caches the entire content of the relevant super-record, then they can
be swapped in and out to compare.
>> (the function that 'with' blocks otherwise perform, but for whenever a
>> 'with' block is inappropriate). Much like you can do with file
>> handles.
>
> OK, but littering code with lots of close()s isn't too pretty.
In practice so far close() doesn't get much usage, versus with blocks.
It won't go away (the context management used by 'with' will use it)
but there's no obligation to. I only use explicit close()s where
indentation levels would otherwise get excessive.
(again, just like would be the case with filehandles)
> Fine, it is clearer, ever so slightly!
If you don't like, don't use. It's no problem.
I designed it because I much prefer it.
> Sketching DSLs can be a lot of fun (but turns into frustration for me
> due to perfectionism. They usually can't be the silver bullet you
> want).
Yeah, even more so the more powerful the source language is. You end
up realizing that you're looking at implementing features that are
already done well in the host language, better than any quick hack
could be.
>> You're right here. I actually went with the 'sub' idea initially
>> because it distinguished subsections clearly, but another convention
>> can probably handle that.
>
> That could be a useful convention, but there's no gain in restricting
> it to just RELOADWriter.
Hm. Point.
>
>>>
>>> The ZoneMap class should definitely have name and extra/extradata
>>> members rather than obscuring that in an 'info' tuple.
>>
>> So what, you want a name[N_ZONES] dict and an extradata[N_ZONES]
>> dict instead? Eh, okay.
>
> I didn't realise .info was indexed by zone id. That explains why you
> did it that way. In that light, slightly prefer .info, as long as you
> make .info elements namedtuples instead of a tuples.
Sure, a lot of that stuff is provisional anyway, I just put things in
containers that seem
reasonable, and wait for usage to tell me if adjustments are needed.
Updated status:
* 10 classes are done (most of them written from scratch this month)
* 3 are WIP/ need more testing
* 20 are not started yet.
...(tile|foe|wall)maps got finished quickly.
I've been looking at how to do streaming RELOAD.
the general pattern I'm currently looking at:
statsby = parentnode.when('stats_by')
if statsby:
# do stuff with statsby
and to explicitly get the next node, just iterate:
node = next(parentnode)
Related ideas:
* No need for another class, the difference in behaviour is minor.
* precache node info on the current level. This allows to process
nodes 'out-of-order' -- if you request a node that is after the
'current' node, you can still get it.. and process the other one
whenever it occurs in your code.
* precaching based on a size threshold: if the size specified on a
parent node is smaller than N bytes, precache all its children.
* saving needs some thought, mainly we need to be able to create
'detached' nodes and 'attach' them later when dealing with complex
structures.
In general this describes a 'mixed' model, where we stream when we
must, and otherwise act like we have random access. I haven't decided
on the quality of this model yet. It would raise an error when sizes
are drastically out of wack (eg. a container node you expect to be
1000 bytes is 10 instead)
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Mac bug: Maybe BINARY ACCESS READ rather than LOF is the culprit?
On Sat, Jun 2, 2012 at 6:43 AM, James Paige wrote:
>> Actually, I can't believe I missed this before, but it looks like ALL
>> files < 4096 bytes are affected. The filenames that have smaller sizes
>> that were not affected were actually the names of directories.
>>
>> I am not sure why OPEN works at all on directories, but it does. The LOF
>> when opening a directory appears to be nonsense.
>>
>
> When using the Linux version of FreeBasic, you can always successfully
> OPEN directories, and LOF is always 4096.
This is rather academic, but.. is that true even for directories that
aren't 4096 bytes in size (which is the default size for ext*) ?
( after running the following code in a Python shell,:
---
import os
def spamfiles (directory, nfiles = 256):
try:
os.mkdir(directory)
except:
pass
for i in range (nfiles):
f = open(os.path.join(directory, str(i)), 'wb')
f.write (b'\x00')
f.close()
---
you can type in, for example, "spamfiles ('/tmp/foo8')" which creates
a directory with 256 files inside. ('ls' tells me that this directory
is 5160 bytes. larger nfiles produce larger directory size, for
example 512 makes the directory 10280 bytes, 1024 -> 20520)
SHORT VERSION: I suspect FreeBasic is just returning the actual
directory size, which may vary according to the number of items
inside. I guess the only alternative behaviour they could have chosen
here is to return an invalid value like -1 for directories. But then
you'd have problems if you really wanted to check the actual directory
size (for example if you want to defer investigating large directories
during a tree walking operation)
I'm not sure what use it is to OPEN a directory, though ;)
[using Python, opening a directory for reading is possible with the
low-level descriptor-based access tools in 'os' module; AFAICS this
seems to be only useful for chdir/chmod/chown, and stat. Nothing
remarkable except being faster than doing these operations by name.]
At the moment, I wonder whether this bug is a problem in how FB sets
up file buffering on Mac. Buffer size == 4096?
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] nohrio-based tool for importing/exporting data
On Fri, May 18, 2012 at 1:18 AM, Ralph Versteegen wrote: >> One thing I'd like to mention at this point is that some parts of >> ndarray subclassing are non-obvious; for instance recarray-style >> access is not as good as it sounds because arrays come with a great >> many (100+?) methods, which will make things quite confusing when >> debugging (Tab-complete -> truckload of options even if there are only >> two subfields). I concluded that most of these methods need to be >> hidden so when you tab-complete or dir(), you get a useful result. >> That's complex to implement correctly. > > Why is it? Do you mean that only a hand selected set of methods should be > shown? Well, you could even show NONE. Although some are very common and useful. The point was more that with attribute-style access, completion will not only show, say, arr.x and arr.y, but arr.sum, arr.take, arr.copy, arr.T, arr.all, arr.any,. On the base ndarray class, there are 69 methods. On subclasses, of course you can't just delete these methods (though 99% of them can be achieved equivalently using identically named functions from the numpy module).. after all, they don't exist in your subclass, only in the parent class. And MRO resolution will always find them. So you end up having to write your own __hasattr__ and __getattr__ (or was that __getattribute__? I forgot which does what), which have to: * do MRO lookup for all __* attributes, but not themselves / * do fake attribute lookup when handed a field name (eg. 'x'). / * do MRO lookup IFF the attribute name is whitelisted. The main problem is in those last two steps, and the potential for a method to become hidden by a field name. "all" is an obvious candidate there; "take" is another; view, size, max, min, base, data are other candidates (I'm selecting only from the list of attributes that I think are pretty essential to leave accessible as methods). It's wrong to say it's complex, actually; 'Irresolvable' is a much more accurate description. > >> So for now I opted to go for >> x['y'] notation, with an extension allowing pseudo-dot-access -- >> x['y.z.foo.bar'] >> >> Looks like that got left out of the previous commit, so updating will >> get you that set of files (nohrio/dtypes/* (but mainly 'general.py' >> right now; the others are somewhat experimental and really only >> demonstrate a possible subclassing/field access model)) > > Looking again, I see that there's still something missing. ohrtype and > readrecords are defined anywhere. No, they aren't. I haven't written them yet. Probably cause that code's out of date. That submodule's kind of a mess of different ideas how to handle rich data type definition. Not all of them will even run. > > BTW you wrote '3188' instead of '3118' several times. Grep says twice; I've pushed a fix for those two. > >> Oh, also, there's machinery for mutable dtypes in nohrio. Useful for >> split-records like the attack data, BINSIZE controlled records, or >> where valid enumeration values vary across versions. You just freeze() >> it after you're done editing. >> >>> >>> Originally I wanted to create a metadata file format similar to >>> editedit definitions, except describing binary record (and RELOAD) >>> based file formats, and try to split that off from editedt menu >>> definitions (which would then specify much less: mostly just the order >>> in which fields are presented and conditions on each appearing/being >>> enabled). But clearly this would just create a whole lot of work, so >>> I'm not so keen on it anymore. >> >> (dtype objects are pickleable, and pickling preserves metadata, BTW) >> >> I eventually decided that this kind of idea was overengineering, >> because the only application that could do anything useful with that >> kind of information would be a generic data editor/filterer (there are >> commercial hex editors that provide this functionality). >> Despite that, I had a go at writing one; Hopefully it got into the >> recent commit. Though it's notably got problems creating the right >> widgets for nested fields, it's a decent prototype. > > It's not there, but that sounds very cool! SDHawk wrote something Actually it's in my i/o centered project here, not in nohrio at all: https://gitorious.org/bits/bits (in subdirectory 'tools') I can't currently run it, because my Python 3 GObject-Introspection installation seems somehow broken. Ironic. > similar (pygtk I think) for user-definable data structures for the ika > engine (described as Python classes or nested dictionaries; I don't > remember) > . I'd actually like to do the same thing for the OHR, but for > RELOAD documents. I think that's something Mike intended from the > beginning. That's kind of different, right? Procedurally generated structures instead of fixed but possibly overlapping structures? I know an implementation of that exists in a commercial hex editor.. They provide a plug-in system that requires you to write in a c-ish language, haha. > >> Eve
Re: [Ohrrpgce] nohrio-based tool for importing/exporting data
On Mon, May 14, 2012 at 5:27 PM, Ralph Versteegen wrote:
> On 14 May 2012 18:36, David Gowers (kampu) <00a...@gmail.com> wrote:
>> I think that must be a Windows-only constraint. I can memmap with any
>> offset and any length.
>
> I forgot to try manually specifying the length. Wow, numpy lets you
> grow files by writing off the end of a memmap? I didn't know; that's
> quite nice. But RPGDir needs a special case for that.
>
> It seems LibreOffice can't handle spreadsheets with 1300 columns
> though!
..
..
..
who would have ever imagined THAT?
> Would need to change the format to have 'shop id' and 'item
> number' key columns.
>
>>>
>>> What I learnt is that this is totally the wrong approach, and it needs
>>> to be rewritten. You really need to write lump-specific code which
>>> knows how to handle things like enumerations, off-by-ones, arrays that
>>> are split into two pieces, and the like.
>>
>> Yeah, that's what I decided. A more conventional record interface
>> implemented in Python, which uses ndarray subclassing to provide
>> enumeration, badstring, off-by-one, and split record handling. It
>> needs to address two distinct needs : arr.fieldname needs to
>> behave like `recarray`, but it should determine the subclass of the
>> returned array from the dtype metadata associated with the field.
>
> I was trying to do that by subclassing np.void and returning instances
> of that from OhrData.__getitem__, because IIRC even recarray records
> return np.void instances instead of np.records if you index a compound
> subfield of a record (eg. stat 'bonuses' in ITM). But I didn't get it
> working.
No, np.void is a special snowflake.. an arrogant, uncooperative one.
Work to improve this situation in NumPy is ongoing (and will probably
emerge by Numpy 2.0 IIRC).
AFAICS, once you have received an np.void, an np.void it shall forever remain.
One thing I'd like to mention at this point is that some parts of
ndarray subclassing are non-obvious; for instance recarray-style
access is not as good as it sounds because arrays come with a great
many (100+?) methods, which will make things quite confusing when
debugging (Tab-complete -> truckload of options even if there are only
two subfields). I concluded that most of these methods need to be
hidden so when you tab-complete or dir(), you get a useful result.
That's complex to implement correctly. So for now I opted to go for
x['y'] notation, with an extension allowing pseudo-dot-access --
x['y.z.foo.bar']
>
>> To that end I started working on improving the datatype definitions
>> (we can store such additional info in the 'metadata' field of the
>> dtype object.). The format is compilable (ie. you can produce actual
>> dtypes with metadata attached from it, and vice versa), but incomplete
>> (off-by-one isn't handled. not sure if badstring is handled).
>>
>> I thought I'd pushed this work, but no.. Well, that's remedied now.
>
> This looks a lot like what nohrio needs, though the encoding is a bit
> too compact imo :)
There's a few formats. The one that I currently prefer is a good
balance between compact and explicit:
gen = DType ("""
> Originally I wanted to create a metadata file format similar to
> editedit definitions, except describing binary record (and RELOAD)
> based file formats, and try to split that off from editedt menu
> definitions (which would then specify much less: mostly just the order
> in which fields are presented and conditions on each appearing/being
> enabled). But clearly this would just create a whole lot of work, so
> I'm not so keen on it anymore.
(dtype objects are pickleable, and pickling preserves metadata, BTW)
I eventually decided that this kind of idea was overengineering,
because the only application that could do anything useful with that
kind of information would be a generic data editor/filterer (there are
commercial hex editors that provide this functionality).
Despite that, I had a go at writing one; Hopefully it got into the
recent commit. Though it's notably got problems creating the right
widgets for nested fields, it's a decent prototype.
Every other kind of application pretty much needs to have some
hardcoded understanding of the data fields and their type, if not
their structure; So I concluded this functionality doesn't belong in
OHRRPGCE itself.
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] nohrio-based tool for importing/exporting data
Hi TeeEmCee, On Mon, May 14, 2012 at 3:37 PM, Ralph Versteegen wrote: > I've been working on a crude nohrio program to import and export data > (enemies, heroes, attacks, items, textboxes) to csv files. It's > totally generic, working by inspecting the fields of the lump's dtype, > but there weren't any other interesting lumps to handle (aside from > shop items, but nohrio can't memmap .stf directly because of the odd > length). I think that must be a Windows-only constraint. I can memmap with any offset and any length. > > What I learnt is that this is totally the wrong approach, and it needs > to be rewritten. You really need to write lump-specific code which > knows how to handle things like enumerations, off-by-ones, arrays that > are split into two pieces, and the like. Yeah, that's what I decided. A more conventional record interface implemented in Python, which uses ndarray subclassing to provide enumeration, badstring, off-by-one, and split record handling. It needs to address two distinct needs : arr.fieldname needs to behave like `recarray`, but it should determine the subclass of the returned array from the dtype metadata associated with the field. To that end I started working on improving the datatype definitions (we can store such additional info in the 'metadata' field of the dtype object.). The format is compilable (ie. you can produce actual dtypes with metadata attached from it, and vice versa), but incomplete (off-by-one isn't handled. not sure if badstring is handled). I thought I'd pushed this work, but no.. Well, that's remedied now. > I think nohrio itself should > know what's in a data field, rather than just the dtype. For example, > I even had to recognise strings by their dtype. My eventual goal for > nohrio is to do better, and wrap all lumps in classes (preferably > ndarray subclasses) to cleanup the interface. > > So little of this code has any use, aside from the csv > handling/escaping functions. However, just as I finished, I discovered > that Python actually has a module for CSV files. That's to be > expected, but I didn't even think to look. > > I did however fix several nohrio bugs and implement missing > functionality, though. > > http://gitorious.org/~tmc/nohrio/tmc/commits/rpgbatch Cool, I'll check that out when I have time.. Right now, I'm both sick and caught up in moving. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Multiline string constants
On Tue, Apr 24, 2012 at 2:42 PM, Seth Hetu wrote:
> Huh, I never knew that Python included the newlines and spaces in
> multi-line strings (probably because I only use them for doc comments,
> which are post-formatted anyway).
>
> Hmm I can think of a few cases where it's useful to keep the
> spaces, and a few cases where it's useful to strip them. Would a
> function wrapper work with the current syntax? E.g.,
>
> new textbox(remove_leading_ws("""
> Buck Star:
> Yes sensai Danielsan.""")
> , 4, align:center, align:bottom)
>
IMO it's simpler (from a user's POV) to support implicit string
compile-time string concatenation, than to have these stripping
operators.
For example,
new textbox("Buck Star:\n"
"Yes sensai Danielsan", 4, align:center,
align:bottom)
This is also a property of Python. for example,
>>> s = ("foo"
"bar")
>>> s
'foobar'
(hopefully the above shows up normally, rather than as a triple email
quotation :)
TL;DR : Docstring style is neat if you really need to write a many
lined string; if you just need a few lines, auto-concatenation is
easier and more predictable to work with.
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Teach me some git
On Sat, Apr 21, 2012 at 3:26 AM, Ralph Versteegen wrote: > > I found this pretty confusing too, and I'm still not entirely > comfortable working with remotes. > > But I'll point out (though this is getting into advanced territory) > that there are usually *three* copies of a remote branch: the actual > branch on the remote machine (don't be fooled, unlike in svn you can > not directly access these, first you have to copy them to your > machine), your copy of the remote branch (named remotename/branchname > e.g. origin/master), and your personal copy of the branch (e.g. > 'master'), (called a 'remote tracking' branch) which is the one you > modify. Is this really true? The model I learnt includes: * the actual branch on the remote machine (eg. master; which as you mentioned, cannot be modified; you can only directly modify local branches) * one or more 'aliases' that map local branch names to remote+branch combinations (these are modifiable, but only on the alias level or via git push/pull, not on the individual file level). * local branch (this is called a 'remote tracking' branch if it originates from one of the 'aliases' (aka 'remotes')) > > But any of these three copies might not exist, and you can actually > modify 'origin/master' -- I've done that by accident a few times. This is true. I'm guessing that you accidentally created a *local branch* named 'origin/master' (not a remote; a remote is the usual situation in which you'd see an identifier like 'origin/master'). The reason you can do that is that '/' is a character which is allowed in branch names, TL;DR : The branch you really modified, is it relly the remote branch 'origin/master', or is it a local branch? Cause AFAICS the former is directly and intentionally prevented by Git, following the 'all changes are local unless explicitly pushed/pulled' principle. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] dealing with a corrupted palettes.bin file
On Sun, Apr 8, 2012 at 2:38 PM, Ralph Versteegen wrote: > Anyway, while I was looking at the relumping code I realised it was > quite delicate. If something goes wrong, you'll probably just get a > silently corrupted RPG file. It should be possible to add some safety > checks. Of course I have trouble coming up with examples of something > going wrong other than 'you started deleting working.tmp while trying > to relump it' * 'the filesystem that working.tmp is on was unmounted before or during relumping' (functionally equivalent to your example) * 'The filesystem that the relumped file is supposed to be written to has been remounted read-only due to IO errors' aka 'destination location is read-only' * 'Read permissions have been removed from working.tmp (or >=1 of the files in it)' ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
[Ohrrpgce] 64bit dependencies
Hi, Just dropping a note, since I didn't find this in README-source.txt, that on 64bit Linux platforms, 32bit versions of libs are needed for GCC to pick them up when linking ohrrpgce-(game|custom). This usually involves installing a separate package -- for example (on Arch Linux), I already had 'sdl_mixer' installed, but needed to install 'lib32-sdl_mixer' before the executables would link. (FWIW, this is with FB 0.23) Currently trying to verify bitset-numbering endianness, David ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] 16-color Palette manipulation with scripts
On Sat, May 28, 2011 at 1:17 AM, James Paige wrote: > > This makes a lot of sense to me now. I wrote up a summary of it here: > http://rpg.hamsterrepublic.com/ohrrpgce/Types_of_Palettes > "There are four types of palettes.. (describes three)" Are you trying to make a Monty Python reference? ;) 'Type four, there IS NO type four..' ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Build system pruning
Instead, you have 'scons -c' """ Specifying "cleanup" targets in SConscript files is not usually necessary. The -c flag removes all files necessary to build the specified target: scons -c . to remove all target files, or: scons -c build export """ -- scons man page On Tue, Apr 26, 2011 at 6:57 PM, Seth Hetu wrote: > From what I read in the manual, scons doesn't have a default "clean" > option, but you can (of course) make that target yourself. > > I never really understood why they didn't include clean by default. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: teeemcee/4084 Ugh, should have incremented RPG Format version when I added .DT0 record
On Thu, Jan 6, 2011 at 6:16 AM, James Paige wrote: > On Thu, Jan 06, 2011 at 08:02:20AM +1300, Ralph Versteegen wrote: >> However f you don't build out of an svn or git repository (for >> example, if you download the nightly source) then you'll get a >> revision number of 0, which is awfully inconvenient. It would be nice >> the the build system cached the last know revision number somehow. >> Maybe this could also solve the problem of 'git svn info' taking >> longer than the actual compile under Windows! git-svn is written in >> fork-crazy perl, and spawning a process is something like thousands of >> times slower than it is on any Unix. just for comparison: 'git svn info' on the git-svn mirror of OHRRPGCE takes about 4 seconds. >> >> Anyway, if that problem is solved, I'm much in favour of it. what about 'git log --grep=git-svn-id |grep -oEe 'ohrrpgce/.+@([0-9]+)'|head -n 1|grep -oEe '[0-9]+' ' ? (as a demo) outputs '4085' on my current checkout. Designed to handle the case where you have one or more commits that haven't yet been committed to SVN, only locally, as well as the case where you don't. The idea, if you don't speak grep-ese, is simply to find the log entries with 'git-svn-id' on some line, take the first one, and extract the bit of text following 'ohrrpgce/SVNBRANCHNAME@' (stopping at the next space) Writing a little FB program to do that would probably be the most portable solution. .. it's been ages since I actually wrote any BASIC. pseudo-ish code follows: ' ** get the output of "git log --grep=git-svn-id" (could be done in advance by the build system) ** ' ** read lines until INSTR (thisline, "git-svn-id") > 0 ** ' and now find the part of the line we want svnspecstart = INSTR (thisline, "svn://") svnspecnumstart = INSTR (thisline, '@', svnspecstart) + 1 svnspecnumend = INSTR (thisline, ' ', svnspecnumstart) version = MID (thisline, svnspecnumstart, svnspecnumend - svnspecnumstart) PRINT version ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] string.bi
Hmm, I thought I fixed that before (it's about the dependency detection -- SCons is detecting string.bi as a dep, and not finding it.).. wait.. Adding 'string.bi' to the list on line 10 of ohrbuild.py should fix it. Testing.. yep, works. Commited :) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Instant updates to Game from Custom, through backends
On Sun, Nov 7, 2010 at 2:07 PM, James Paige wrote: > On Sat, Nov 06, 2010 at 05:16:03AM -0700, Jay Tennant wrote: managed by the >> backend, but doing so would allow edits in Custom to be immediately >> visible in Game. >> >> I'm not sure of the practicality of that, but it's an idea. > > While I can see the appeal of being able to test in real-time, I think > the complexity of that task is waaay to large in comparison to the > benefit it would provide. > > And remember, it is not just about updating graphics. What about maps > updates? Hero enemy attack and item updates? Running scripts? Displaying > of text boxes? I think Jay was talking about graphics because (providing their size does not change, as is currently implicitly enforced) they have no side effects -- if a graphic changes, reloading it doesn't require anything else to be reloaded. I haven't dug into the OHR sprite caching code recently, but IIRC this should be true (unsure what the situ is regarding tilemaps). Anyway I agree it's inappropriate to implement merely *some* refreshing -- if there is refreshing at all it should be on all possible data types, which is exceedingly complex even with a good framework (like recording an md5sum for each record so you can tell when it actually changes) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Thu, Oct 28, 2010 at 1:45 AM, James Paige wrote: > On Wed, Oct 27, 2010 at 04:34:35PM +1030, David Gowers (kampu) wrote: >> On Wed, Oct 27, 2010 at 1:47 AM, James Paige >> wrote: >> >> > Hypotheticlly suppose I have a side-scroller where the hero has a >> >> > 4-frame walk animation. When I press and hold the attack button while >> >> > walking, I want her to switch to to the sprite set where she charges an >> >> > energy canon, and when it is fully charged, I want to change her sprite >> >> > set to the one with the fully-charged canon. >> >> > I want to keep my place in >> >> > the 4-frame walk animation when these changes occur. >> >> >> >> It sounds like >> >> >> >> stored := push animation(current) >> >> set animation gfxset (current, charging) >> >> set animation timeout (current, chargeticks) >> >> >> >> # per tick, detect current anim timeout and.. >> >> set animation gfxset (current, charged) >> >> set animation timeout (-1) >> >> # or detect button release and... >> >> replace animation (current, pop animation(stored)) >> > >> > Interesting. With an interface like that I would probably not use the >> > feature at all. I would probably just use "replace animation" and >> > nothing else, since the push/pop commands and the timeout commands since >> > those both kinda volate Mode-View-Controller by putting "model" data >> > into the "view" which is going to have to be duplicated in the "model" >> > anyway for other reasons. >> >> I don't 100% get MVC, however I had the impression that the design I >> describe in the lumpdefs.yaml didn't have any view stuff, only model >> and (with the above api description) controller. I suppose an active >> animation could be considered a view.. in some sense. >> What do you mean? That Timeout should be an op in the animation? (in >> this case a way to prime an animation with a 'local default' timeout >> seems moderately important) > > Well, I don't strictly have to stick to MVC when I am making a > sidescroller, it is just a general guideline. > > What I mean about the timeout, is that in a sidescroller, every > animation should represent something that is happening in the game's > model. So say I know that charging takes chargeticks ticks > > set animation gfxset (current, charging) > set animation timeout (current, chargeticks) > > ...but am I actually going to read the animation state to see whether or > not I have finished charging? No, I am going to have some other timer > in my player model that keeps track of how I am charging. So I am > storing the timer in two places. > > That is not a terrible problem, not at all. I am just bringing this up > to draw attention to the fact that I am uncertain about how I would > actually use this proposed animation interface in a script. > > The fact that I don't know that yet doesn't mean I think it is a bad > proposal. > > In the same way, storing the previous animation state with a push/pop > interface doesn't make sense to me because when an animation ends you > want to switch to whatever is appropriate for the sprite *now* not what > was appropriate for the sprite back before the animation started. Yeah, that was poorly thought out. I'll scrap the push/pop idea and just go with the idea of being able to invoke child animations. The timeout is supposed to be complimentary to the other kind of looping, but it seems dubious now. > >> > But for non-sidescrollers... maybe just making a hero or an NPC do an >> > "emote" in a regular RPG, I can see that interface being convenient. >> I am now unsure. > > I guess another thing to think of is how might animations be used in a > non-scripted way. > > Continuing on the idea of hero and NPC emotes, what if we want to be > able to create a text box, and say that the box should trigger the > "talking" animation of the NPC that triggered the box. Or what if we > want to make a text box that runs one loop of the portrait "scorn" > animation before settling on the "frown" state of the portrait, > regardless of who's portrait is actually being displayed? a) TMC and I were talking about this earlier, and concluded that in some places, animations need to be selectable by name rather than number. Textboxes seem to be one. Would attacks be another? (Working out how these animations interact with the existing 'attacker motion' will be interesting o) b) that sounds like a meta-animation (which would have to be started with a specific gfxset). Trouble is, images in a gfxset are not currently(in the spec) identifyable by name, so that would require all portrait gfxsets to have matching numbers and ordering of images. Actually! That sounds like a rather helpful convention. Empty images could just be stored as 1x1 sprites filled with #0. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Wed, Oct 27, 2010 at 4:34 PM, David Gowers (kampu) <00a...@gmail.com> wrote: > On Wed, Oct 27, 2010 at 1:47 AM, James Paige wrote: >>> > Hypotheticlly suppose I have a side-scroller where the hero has a >>> > 4-frame walk animation. When I press and hold the attack button while >>> > walking, I want her to switch to to the sprite set where she charges an >>> > energy canon, and when it is fully charged, I want to change her sprite >>> > set to the one with the fully-charged canon. >>> > I want to keep my place in >>> > the 4-frame walk animation when these changes occur. >>> >>> It sounds like >>> >>> stored := push animation(current) >>> set animation gfxset (current, charging) >>> set animation timeout (current, chargeticks) >>> >>> # per tick, detect current anim timeout and.. >>> set animation gfxset (current, charged) >>> set animation timeout (-1) >>> # or detect button release and... >>> replace animation (current, pop animation(stored)) >> >> Interesting. With an interface like that I would probably not use the >> feature at all. I would probably just use "replace animation" and >> nothing else, since the push/pop commands and the timeout commands since >> those both kinda volate Mode-View-Controller by putting "model" data >> into the "view" which is going to have to be duplicated in the "model" >> anyway for other reasons. > > I don't 100% get MVC, however I had the impression that the design I > describe in the lumpdefs.yaml didn't have any view stuff, only model > and (with the above api description) controller. I suppose an active > animation could be considered a view.. in some sense. > What do you mean? That Timeout should be an op in the animation? (in > this case a way to prime an animation with a 'local default' timeout > seems moderately important) > >> >> But for non-sidescrollers... maybe just making a hero or an NPC do an >> "emote" in a regular RPG, I can see that interface being convenient. > I am now unsure. > After doing some more research, it seems that 'view' would simply pertain to the sprite displayed during an animation. Is this correct? What configuration of components are you imagining? ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Wed, Oct 27, 2010 at 1:47 AM, James Paige wrote: >> > Hypotheticlly suppose I have a side-scroller where the hero has a >> > 4-frame walk animation. When I press and hold the attack button while >> > walking, I want her to switch to to the sprite set where she charges an >> > energy canon, and when it is fully charged, I want to change her sprite >> > set to the one with the fully-charged canon. >> > I want to keep my place in >> > the 4-frame walk animation when these changes occur. >> >> It sounds like >> >> stored := push animation(current) >> set animation gfxset (current, charging) >> set animation timeout (current, chargeticks) >> >> # per tick, detect current anim timeout and.. >> set animation gfxset (current, charged) >> set animation timeout (-1) >> # or detect button release and... >> replace animation (current, pop animation(stored)) > > Interesting. With an interface like that I would probably not use the > feature at all. I would probably just use "replace animation" and > nothing else, since the push/pop commands and the timeout commands since > those both kinda volate Mode-View-Controller by putting "model" data > into the "view" which is going to have to be duplicated in the "model" > anyway for other reasons. I don't 100% get MVC, however I had the impression that the design I describe in the lumpdefs.yaml didn't have any view stuff, only model and (with the above api description) controller. I suppose an active animation could be considered a view.. in some sense. What do you mean? That Timeout should be an op in the animation? (in this case a way to prime an animation with a 'local default' timeout seems moderately important) > > But for non-sidescrollers... maybe just making a hero or an NPC do an > "emote" in a regular RPG, I can see that interface being convenient. I am now unsure. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Fri, Oct 22, 2010 at 1:40 AM, James Paige wrote: >> > Then the question arises of what to do when chaining to another >> > animation: should the saved state be overwritten? Both possibilities >> > could be useful; maybe another parameter to 'end' would even be >> > justified. >> This seems hard. I'll poke around a few different models and tests to >> see what comes out. > > Are we talking about support for chaining of animations built right into > the animation itself? Mainly this. > > Or are we talking about switching animations as a result of an external > trigger like a plotscripting command? Not sure what that would look like; that had been part of my model of game animations for a while, but I haven't figured out any remotely elegant implementation. The best I deciphered there was a message-waiting mechanism (on anim side) and message sending (on script side) > > Hypotheticlly suppose I have a side-scroller where the hero has a > 4-frame walk animation. When I press and hold the attack button while > walking, I want her to switch to to the sprite set where she charges an > energy canon, and when it is fully charged, I want to change her sprite > set to the one with the fully-charged canon. > I want to keep my place in > the 4-frame walk animation when these changes occur. It sounds like stored := push animation(current) set animation gfxset (current, charging) set animation timeout (current, chargeticks) # per tick, detect current anim timeout and.. set animation gfxset (current, charged) set animation timeout (-1) # or detect button release and... replace animation (current, pop animation(stored)) The current GIT head of nohrio shows (somewhat) what might be required to support this: http://gitorious.org/nohrio/nohrio/blobs/master/tools/lumpdefs.yaml > > Right now, I would just script the frame-changes for the animation > manually. (in fact I already do exactly this in Bell of Chaos for the > walk/crawl animations, but because they are only 3-frame and involve a > walking-speed-cap-change I doubt it is even noticeable) > > Would the chaining of animations you are talking about be relevant to > that situation? Or is it some other purpose entirely? That was mainly TMC's idea, but as he hasn't responded.. It was mainly internal, but I've begun adjusting it to suit external interaction better. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Mon, Oct 18, 2010 at 11:11 PM, Ralph Versteegen wrote: >>> >>> Wondering why you noted reset as only resetting gfxset - you don't >>> think the rest is useful? >> I hadn't thought how to derive the defaults yet. >> I've partially amended it now. > > I intended to just save the state at the start of the animation, and > restore it when resetting. Fair enough. So we need an 'original_state' pointer? > > Then the question arises of what to do when chaining to another > animation: should the saved state be overwritten? Both possibilities > could be useful; maybe another parameter to 'end' would even be > justified. This seems hard. I'll poke around a few different models and tests to see what comes out. > >>> >>> I'm wondering how much trouble it would be to unify 256 and 16 colour >>> palettes. Either allow palettes to have a variable number of colours, >>> or upgrade all palettes to 256 colours so that there is no >>> distinction. It would make a lot of things easier. >> I totally agree. The only question is, how? >> >> You are proposing sharing one master palette between all graphics of >> the same class? >> so 8*256 = 2048 in total. > > No... I don't think so. I thought I left specification of master > palettes open, but (changing topic slightly) perhaps it's dubious to > not associate a master palette with each 16 or 256 colour (more > correctly: index) palette, if you allow more than one master palette > at once. I think it's important, especially given the low overhead of doing so > >> I'd like to extend this by requiring a 'master palette #' as part of >> the on-disk image format. This can > > This sounds good, unless master palettes are instead associated with > 16/256 index palettes. Not sure what you mean but the following might answer it The current scheme associates a masterpalette number and an indexpalette with each gfxset. This works out nicely for existing gfx types, particularly tilesets and fonts. > >> a) double as a way to indicate unpalettedness, if this ever seems >> appropriate in the future. >> b) allow for 'peripheral' graphics that aren't precisely part of any >> graphics class. > > I'm not sure what this has to do with master palettes. I thought we > would add one additional class for misc graphic sets. Or better, allow > users to define multiple, so that they can group their sets as they > wish. Yeah, I reread what I'd written above several times, and I have no idea what I meant by b) >> I don't seem to grok that. Doesn't it need to have an 'first >> applicable tile' property in order to be a range (or to make sense at >> all in the context of old TAP)? > > I meant that the animation would be defined for the first tile in the > range (implicit in the animation itself), and the n-th animating tile > would display as the animation's current tile/frame + n. One animation runs, and you add eg ((TILENUMBER - 160) % 48) to it to get the actual tile number to display? >> >>> >>> Storing each tile as a separate frame is a slight nuisance. I'm just >>> going to load them all into a single frame as an optimisation. >> SPECIALCASING UGH (this would require tile animation to run off a >> variant animation engine instead of a universal one. >> But I understand why. > > Actually, no, as the animation engine doesn't need to access the > sprite data, aside from number of frames, as far as I can see. right, good point. > It > would instead require tiles to be drawn using a variant sprite drawing > routine -- and they already are. I think that this allows a noticeable > speed-up in drawing the map due to improved memory locality and > reduced processing of hundreds of frame structures. > > I should have talked about strings instead. Fonts are also > special-cased, yet the overhead of drawing each character is still > about as large as actually blitting the 64 pixels (largely due to FB > vs. C optimisation differences). > > So, (I think) I forgot to mention fonts. We'll want to store them in > the same way as other graphics, minus animations. I had thought about fonts earlier. You propose to store them as 256-color graphic sets? This makes sense to me, particularly for multicolored fonts. > >>> > >> * nor does my proposition address partial recoloring. With 256 color >> sprites, it would be far easier to standardize on recoloring, so, for >> example, you could fill a 256-color palette via sets of 16, and just >> swap in/out sets of 16 to recolor things easily. > > I really like the idea of treating a 256-colour palette as 16 > 16-colour palettes. But swapping 16-colours individually doesn't sound > simple to support. Do you mean from a UI perspective? >>> >>> Partially; I was thinking of associating with each sprite an array of >>> palette indices, rather than giving each sprite its own copy of its >>> palette. Using copies of the palette will . It's my nature to be >>> uneasy about the extravagance of a 256 byte array per sprite, but we >>> c
Re: [Ohrrpgce] better palette options
Just a quick reply now, the rest when I do not need to go to bed: On Mon, Oct 18, 2010 at 11:11 PM, Ralph Versteegen wrote: >> >> <- various updates, still considering other updates as a consequence >> of Mike's input. > > Initially I thought tag2ctr was quite nice, but I wonder whether some > people will find the extra layer of indirection required to use tags > in their animations as they currently can in TAP to be confusing. I resisted adding a 'checktag' opcode, because it seemed to me that doing that would make it easy for people to make complicated nonsensical loop structures (overloading the animation system far beyond 'run anim, which could go along branch A or branch B, and possibly do a little variation here or loop for a time there, until it ends.'* (completely incidentally, it gives better cache locality.). However that might just be paranoia, I'll look at adding such an opcode. * which IME is about the limits of what 98% of existing anims do anyway. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Sprites, animation food for thought
On Mon, Oct 18, 2010 at 9:58 PM, Ralph Versteegen wrote: > On 14 October 2010 13:45, David Gowers (kampu) <00a...@gmail.com> wrote: >> On Wed, Oct 13, 2010 at 7:20 PM, David Gowers (kampu) <00a...@gmail.com> >> wrote: >>> On Mon, Oct 11, 2010 at 4:07 AM, Mike Caron wrote: >>>> First off, a disclaimer: This is not an actual proposal for a data format >>>> for the OHR. This is something I've cooked up for another project which may >>>> or may not ever see the light of day (hopefully, it will, but it's >>>> immaterial to this discussion). >>>> >>>> This is how I've implemented sprites and animations in this other project: >>>> >>>> First, all pixel data is stored in sprite sheets. These are PNGs that >>>> contain all the sprites for a given on-screen entity. >>> I wanted to use a sprite-sheet based system in my proposition, >>> but so far I don't see how to avoid dumping a load of complexity on >>> the user. (as a user, arranging images on a sprite sheet is something >>> I don't want to deal with. Defining some nominal 'default dimensions' >>> and padding to that when adding new sprites would help.) >> Of course, depending on how clever it's okay to be, we could actually >> encode all sprite data as a solid block of pixels with 0 wasted bytes. > > That's actually a really small change to the file format, isn't it? > You're only suggesting concatenating the pixel data lumps together, > aren't you? (I think I'd prefer it that way, as it's a bit simpler to > handle, but less self-describing) yes, it would require reading individual images, not the 'whole image' - which would look like gibberish. OTOH if we ever use hardware accel, being able to read spritesheets as one big image would definitely help performance. > >> This would require a rewrite of the gfxset every time a sprite changed >> dimensions, and we would need to do a little work to export: >> 1. Take the maximum of the dimensions of the sprites in the gfxset and >> designate that the tile size. >> 2. Guess an appropriate WxH of tiles (ie how many tiles across and how >> many down, in order to attain an aspect ratio near 4:3, which will fit >> the most sprites on screen.) >> 3. Place the sprites sequentially (converted to masterpalette indices, >> rather than private-palette indices) within the tiles according to >> default-offset info (clamped so the sprite doesn't escape tile bounds) >> 4. Save a matching metadata file, specifying sprite bounds, default offsets, >> masterpalette->private palette mapping > > And (optionally) animations? Animations associated with the gfxset? (ie normally any animations in the correspondingly numbered animset) Good idea, seems pretty useful to me if you're writing a filter or such. (I'd bet I could hack up a PyGIMP plugin which built GIMP-viewable animations from animation+frameset. That could be pretty handy esp. given GIMP's nice time-zooming animation player.) > >> 5. Save the actual tile-based bitmap. > > This sounds almost exactly like how I wanted to export sprite sets. > > I also thought about drawing two-colour X- and Y-axis guides (line > segments) along the edges of the tiles to show the size/position of > the frame and of the srcrect/"bounding box". It does not matter that > the (along with background) three colours used won't be unique, if the > edges of the tiles can be found on import. This could be done even > without any metadata. I think I know what you mean. If we add 1 to the w/h of each tile, we can use that for said annotation. > >> This would require: >> a) the user to export frames individually if they wanted to resize(not >> scale -- just change the dimensions) them (or else provide resizing in >> the sprite editor). > > Of course we'll provide resizing in the sprite editor. > >> b) them not to 'lose' the metadata file >> (IME this kind of thing is a bit of a pain, however hopefully being >> able to do a fresh export of a sprite set would mitigate it) >> c) we would have to autodetect added indices and do something with >> them (prompt the user whether to add them to the private palette?). >> This would require scanning every frame which the metadata says uses >> the same private palette. >> >>> I'm a bit ambivalent now. TMC's proposition of attaching a set of >>> palette indices to each sprite "makes my common-sense tingle" :) I >>> think we need to look at how frequently color swapping is used vs
Re: [Ohrrpgce] better palette options
I believe I've solved the palette problems, see the other thread if interested. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Sprites, animation food for thought
On Wed, Oct 13, 2010 at 7:20 PM, David Gowers (kampu) <00a...@gmail.com> wrote: > On Mon, Oct 11, 2010 at 4:07 AM, Mike Caron wrote: >> First off, a disclaimer: This is not an actual proposal for a data format >> for the OHR. This is something I've cooked up for another project which may >> or may not ever see the light of day (hopefully, it will, but it's >> immaterial to this discussion). >> >> This is how I've implemented sprites and animations in this other project: >> >> First, all pixel data is stored in sprite sheets. These are PNGs that >> contain all the sprites for a given on-screen entity. > I wanted to use a sprite-sheet based system in my proposition, > but so far I don't see how to avoid dumping a load of complexity on > the user. (as a user, arranging images on a sprite sheet is something > I don't want to deal with. Defining some nominal 'default dimensions' > and padding to that when adding new sprites would help.) Of course, depending on how clever it's okay to be, we could actually encode all sprite data as a solid block of pixels with 0 wasted bytes. This would require a rewrite of the gfxset every time a sprite changed dimensions, and we would need to do a little work to export: 1. Take the maximum of the dimensions of the sprites in the gfxset and designate that the tile size. 2. Guess an appropriate WxH of tiles (ie how many tiles across and how many down, in order to attain an aspect ratio near 4:3, which will fit the most sprites on screen.) 3. Place the sprites sequentially (converted to masterpalette indices, rather than private-palette indices) within the tiles according to default-offset info (clamped so the sprite doesn't escape tile bounds) 4. Save a matching metadata file, specifying sprite bounds, default offsets, masterpalette->private palette mapping 5. Save the actual tile-based bitmap. This would require: a) the user to export frames individually if they wanted to resize(not scale -- just change the dimensions) them (or else provide resizing in the sprite editor). b) them not to 'lose' the metadata file (IME this kind of thing is a bit of a pain, however hopefully being able to do a fresh export of a sprite set would mitigate it) c) we would have to autodetect added indices and do something with them (prompt the user whether to add them to the private palette?). This would require scanning every frame which the metadata says uses the same private palette. > I'm a bit ambivalent now. TMC's proposition of attaching a set of > palette indices to each sprite "makes my common-sense tingle" :) I > think we need to look at how frequently color swapping is used vs a > number of sprites sharing the same palette. If it's high, that favors > individually defined frames. If it's low, blobs of glued-together > frames. I suspect it to be low. regardless, spritesheets have a definite limitation WRT palettes: they must be shared across all sprites in a sheet in order for the spritesheet to be human editable. If we postulate that all sprites in a gfxset share a palette, this is not a problem. As I suggested above, we could convert to master palette and that could be the 'shared palette' (with private palette data stored in the metadata file) That might actually work quite nicely. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Sprites, animation food for thought
On Mon, Oct 11, 2010 at 4:07 AM, Mike Caron wrote: > First off, a disclaimer: This is not an actual proposal for a data format > for the OHR. This is something I've cooked up for another project which may > or may not ever see the light of day (hopefully, it will, but it's > immaterial to this discussion). > > This is how I've implemented sprites and animations in this other project: > > First, all pixel data is stored in sprite sheets. These are PNGs that > contain all the sprites for a given on-screen entity. I wanted to use a sprite-sheet based system in my proposition, but so far I don't see how to avoid dumping a load of complexity on the user. (as a user, arranging images on a sprite sheet is something I don't want to deal with. Defining some nominal 'default dimensions' and padding to that when adding new sprites would help.) > Technically, as I am > writing it in .NET, nothing is stopping them from being BMP or JPG or > something, but I prefer PNG for its true-colour and transparency support. > > Then, with each sprite sheet, there is an XML document that describes it: > > http://mike-caron.com/za"; name="link1" file="link.png"> > name="stand-down" /> > > name="walk1-down" /> > name="walk2-down" /> > > > > Each frame has a name, a position and size on the sheet, and an origin. The > origin is aligned up with the actual entity's origin (which, ATM is always > their (x,y) position, but that may be customizable later on) > > Then, after the frame definitions, we have the animation definitions: > > > > stand-down > > > stand-up > > > stand-right > > > stand-left > > > > This is mostly self explanitory. Every frame has either one or four > directions defined, depending on if the sprite has meaningful directions (an > item on the ground, for example, probably only has one direction. An NPC, on > the other hand, will have all four). > > Each frame in a direction can have a time parameter, which describes how > many ticks the frame is displayed for. deftime is just the default such > value. Obviously, as well, if loop is yes, the animation will start from the > beginning. If no, it will just stop on the last frame forever. > > Finally, every entity in the game references both a sprite sheet and an > animation. The idea is that you could change one or the other. For example, > if you get a better shield, the sprite sheet changes from "link1" to > "link2". And, obviously, the animation changes based on what is happening in > the game. Ugh, you're giving me all these awesome ideas :) I'll have to cut down, there are already too many ops. > > A few notes on this structure: > > - There is no accounting for palettes. If you wanted to use a paletted > sprite, then by all means you are able, but don't expect to be able to > change that palette in game. The canonical way to add palette swaps is to > have a different sprite sheet with the same animations (and, I may add some > way to automate this. Eg, "this sprite sheet is exactly the same as this > oth er sprite sheet, so use those definitions") I'm a bit ambivalent now. TMC's proposition of attaching a set of palette indices to each sprite "makes my common-sense tingle" :) I think we need to look at how frequently color swapping is used vs a number of sprites sharing the same palette. If it's high, that favors individually defined frames. If it's low, blobs of glued-together frames. > > - There is also no way to get any fancier animations. I.e, there's no way > to automatically oscillate or go backwards or whatever. The way to achieve > these effects would be to duplicate the frames in the animation. > - I may or may not add sprite flipping. > > - The way the images are laid out on the sheet has absolutely nothing to do > with how they are animated or displayed. If you wanted the same frame, but > offset by 5 pixels or whatever, you just modify the origin. No muss, no > fuss. > > - Also, there are no inherent limitations on how large or small the sprite > is. If you want to make a 2d version of Shadow of the Colossus, then be my > guest! (just, don't expect the video card to necessarily play along ;) > > So... yeah. Make of that what you will, I'm just adding ideas to the pot. I > am aware that this format, as is, is unsuitable for the OHR. It's just my > two cents. > ___ > Ohrrpgce mailing list > ohrrpgce@lists.motherhamster.org > http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org > ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Sun, Oct 10, 2010 at 9:34 PM, Ralph Versteegen wrote: >> RELOAD was not part of it (well, honestly RELOAD vaguely annoys me, >> the way it replicates XML's weird data model. So it's not the first >> thing I would think of.) >> >> But it is now. > > I wasn't actually requesting it. RELOAD as container for animation > data, as in your proposal, seems like a fairly good idea. RELOAD as > container for pixel data may be overly heavy. RELOAD annoys me a > little too, due to much increased complexity of read/writing. > >>> >>> I notice that separate image size and bounding box size/offset are >>> missing. Intentionally? >> >> Missing? >> I understood the base bounding box size and location as being >> immutable (and based off say, a manually definable box - see slice >> editor). While this was a bit annoying, it made some amount of sense. >> I'm not sure how to avoid confusing the user if they are not >> immutable. >> >> Please, explain further how you conceive of the interaction of >> bounding boxes and image offsets? > > It's a conflation of two (or three) different ideas. Perhaps they > should be separated. A "srcrect" is just a reference point (offset) > plus width and height equal to the existing frame size limits. I > called these "bounding boxes" because in the current tile-based > movement system (and also somewhat in in-battle movement and > positioning) the frame size is effectively a bounding box. But if/when > we add pixel-based movement one may well want the bounding box to > depend on the current frame. after reading Mike's email and recalling my own experiences with bounding boxes, I'm -1 on "frames should have their own bounding boxes" for the 'single image' sense of 'frame', and +1 on "frames should be able to have their own bounding boxes" in the 'unit of animation' sense of 'frame'. (implemented as an opcode, since while the need for this is definite, it's rare.) > >>> >>> I don't understand the difference between the 'animset' command, and >>> chaining to another animation using 'end'. >> animset != anim. >> it doesn't set the anim, it sets the animset. >> Anyway I removed that by allowing varlength tuples. > > What varlength tuples? one opcode + N parameters. I see I changed the proposition to not use the 'tuple' term though, so my mistake. > >>> > always be appropriate. Quoting from the 'text tags' thread, where I > proposed embedding tags in textboxes which trigger portrait > animations: > >> > Maybe animations should get both a name and a number. The name can be >> > used everywhere that it makes sense to use a name, and the number can be >> > used in whatever small number of places a name isn't practical (like >> > here) >> >> OK, sounds practical. But I wonder whether the ID number should be an >> index into the big global table of animation names (shared by all >> spritesets of any sort), or refer to the n-th animation currently >> defined for this spriteset. I'd prefer the first, since it might make >> sense to sort animations, say alphabetically. The first is obviously better IMO. > > > >>> >>> The check/setctr codes seem to be very inefficient for creating loops: >>> >>> setctr ctr0 = x >>> label 1 >>> ...loop contents >>> setctr ctr0 -= 1 >>> checkctr ctr0 == 0 >>> goto label 2 >>> goto label 1 >>> label 2 .. wait, what? checkctr is crazy. Def changed to '!= 0' >>> >>> Those last 5 instructions could be replaced with a loop(ctr,label) >>> code: decrement ctr and goto label if ctr > 0. A relative frame change >>> code would also be very useful for loops. >> oh right, ctr and label can fit into 16 bits easily(3+8) >> Good idea. (note that I'm leaving in the original ops -- they have a >> specific use that cannot be readily attained by a loop command : >> interleaving instructions, like 'wait 1' that only execute eg on >= >> second loop through a particular area >> of code. > > Your definition of loop isn't correct. Indeed. > > The actual encoding of the bytestring, bits per opcode and argument, > isn't specified. Making them all 16 bit signed is easy, but we if want > to cram multiple arguments into the same short, maybe it would just be > simpler to same that opcodes are 8 bit, and arguments are either 8 or > 16 bit signed (or a string (wouldn't be necessary if we numbered > animations internally?)). > >>> >>> I suggest a 'reset' command. Changing the sprite state after the end >>> may be desired (ie. a transition animation), and sometimes not. >>> 'reset' would return to the original gfxset, frame, palette, and >>> offset. > > Wondering why you noted reset as only resetting gfxset - you don't > think the rest is useful? I hadn't thought how to derive the defaults yet. I've partially amended it now. > > I'm wondering how much trouble it would be to unify 256 and 16 colour > palettes. Either allow palettes to have a variable number of colours, > or upgrade all palettes to 256 colours so that there is no > distinction. It would make a lot of things easier. I totally agre
Re: [Ohrrpgce] better palette options
On Mon, Oct 11, 2010 at 3:36 AM, Mike Caron wrote: > What about RELOAD (or, I guess, XML's) data model don't you like? That nodes with children can also have values. That's a pretty extraordinary behaviour for a tree data model, IMO. Unless you expect people to XSLT their XML source data, I think a model like YAML or JSON is much simpler while still supporting the basic data model. (you can currently convert YAML or JSON to RELOAD with a bit of work. I'm just arguing that the "nodes with children can also have values" is a specific, and probably unneeded, accommodation for XML's strange behaviour.) I'm not > trying to defend it, I honestly want to know what you don't like. Perhaps > something could be improved? > >>> But it is now. >> >> I wasn't actually requesting it. RELOAD as container for animation >> data, as in your proposal, seems like a fairly good idea. RELOAD as >> container for pixel data may be overly heavy. > > I agree that pixel data should not be stored in a RELOAD container. > > Rather, all the animation data should be stored in a single RELOAD document, > and the images separate from that. > >> RELOAD annoys me a >> little too, due to much increased complexity of read/writing. > > But, surely, you agree that the benefits of having a strongly typed and > named data format outweigh the complexity, right? > > Honestly, I think the lump resizing scheme for older lumps is a lot more > complex than RELOAD... > > > I think that's a really bad idea. The point of bounding boxes is that they > are independent of the graphic on screen. Case in point: > > What if, on a frame with a small bounding box, a character moves up against > a wall. Then, on the next frame, he has a large bounding box, which now > intersects with the wall. Which of the following happens? > > 1. The character can now move into the wall? > 2. The character is ejected (a la super mario bros)? > 3. The character can only move out from the wall? > 4. The character can't move at all? > 5. The character is disintegrated since two particles cannot share the same > quantum state? > > If the bounding boxes are fixed (or, at least, independent of sprites), > ..?? Hello sentence which is not there. I don't understand the difference between the 'animset' command, and chaining to another animation using 'end'. >>> >>> animset != anim. >>> it doesn't set the anim, it sets the animset. >>> Anyway I removed that by allowing varlength tuples. >> >> What varlength tuples? >> You're also missing animation names. I think naming animations is better than numbering them. Then you can add a attackee flinch animation to an enemy, say, just by adding an animation and naming it "hurt". >>> >>> +1. I was thinking about this problem and couldn't figure out how to >>> do it nicely, but this is excellent. >> >> But there's a complication, referring to animations by name may not >> always be appropriate. Quoting from the 'text tags' thread, where I >> proposed embedding tags in textboxes which trigger portrait >> animations: >> Maybe animations should get both a name and a number. The name can be used everywhere that it makes sense to use a name, and the number can be used in whatever small number of places a name isn't practical (like here) >>> >>> OK, sounds practical. But I wonder whether the ID number should be an >>> index into the big global table of animation names (shared by all >>> spritesets of any sort), or refer to the n-th animation currently >>> defined for this spriteset. I'd prefer the first, since it might make >>> sense to sort animations, say alphabetically. >> >> >> The check/setctr codes seem to be very inefficient for creating loops: setctr ctr0 = x label 1 ...loop contents setctr ctr0 -= 1 checkctr ctr0 == 0 goto label 2 goto label 1 label 2 Those last 5 instructions could be replaced with a loop(ctr,label) code: decrement ctr and goto label if ctr> 0. A relative frame change code would also be very useful for loops. >>> >>> oh right, ctr and label can fit into 16 bits easily(3+8) >>> Good idea. (note that I'm leaving in the original ops -- they have a >>> specific use that cannot be readily attained by a loop command : >>> interleaving instructions, like 'wait 1' that only execute eg on>= >>> second loop through a particular area >>> of code. >> >> Your definition of loop isn't correct. >> >> The actual encoding of the bytestring, bits per opcode and argument, >> isn't specified. Making them all 16 bit signed is easy, but we if want >> to cram multiple arguments into the same short, maybe it would just be >> simpler to same that opcodes are 8 bit, and arguments are either 8 or >> 16 bit signed (or a string (wouldn't be necessary if we numbered >> animations internally?)). >> I suggest a 'reset' command. Changing the sprite state after the end may be desired (ie. a transitio
Re: [Ohrrpgce] better palette options
On Sat, Oct 9, 2010 at 5:07 AM, Ralph Versteegen wrote: > On 9 October 2010 02:48, David Gowers (kampu) <00a...@gmail.com> wrote: >> On Fri, Oct 8, 2010 at 11:42 PM, David Gowers (kampu) <00a...@gmail.com> >> wrote: >>> http://bpaste.net/show/10001/ >>> ^ >>> The format proposition >>> > > Generally this looks good. Aside from the following long string of criticisms > ;) > > I'm not sure, but I believe you're suggesting using a plain binary > lump instead of Reload for animation? If Reload is actually part of > it, I don't see much point in a separate lump per animation. RELOAD was not part of it (well, honestly RELOAD vaguely annoys me, the way it replicates XML's weird data model. So it's not the first thing I would think of.) But it is now. > > I notice that separate image size and bounding box size/offset are > missing. Intentionally? Missing? I understood the base bounding box size and location as being immutable (and based off say, a manually definable box - see slice editor). While this was a bit annoying, it made some amount of sense. I'm not sure how to avoid confusing the user if they are not immutable. Please, explain further how you conceive of the interaction of bounding boxes and image offsets? > > I don't understand the difference between the 'animset' command, and > chaining to another animation using 'end'. animset != anim. it doesn't set the anim, it sets the animset. Anyway I removed that by allowing varlength tuples. > > You're also missing animation names. I think naming animations is > better than numbering them. > Then you can add a attackee flinch > animation to an enemy, say, just by adding an animation and naming it > "hurt". +1. I was thinking about this problem and couldn't figure out how to do it nicely, but this is excellent. > > The check/setctr codes seem to be very inefficient for creating loops: > > setctr ctr0 = x > label 1 > ...loop contents > setctr ctr0 -= 1 > checkctr ctr0 == 0 > goto label 2 > goto label 1 > label 2 > > Those last 5 instructions could be replaced with a loop(ctr,label) > code: decrement ctr and goto label if ctr > 0. A relative frame change > code would also be very useful for loops. oh right, ctr and label can fit into 16 bits easily(3+8) Good idea. (note that I'm leaving in the original ops -- they have a specific use that cannot be readily attained by a loop command : interleaving instructions, like 'wait 1' that only execute eg on >= second loop through a particular area of code. > > I suggest a 'reset' command. Changing the sprite state after the end > may be desired (ie. a transition animation), and sometimes not. > 'reset' would return to the original gfxset, frame, palette, and > offset. > >> Note that I intentionally made it as unrestricted as possible. >> Imposing additional constraints could allow the removal of certain >> tuple types and runtime data fields. Whether additional constraints >> are appropriate is a matter for discussion. >> >> Comments on PNG: >> * I saw your suggestion, Mike, and I thought : though it isn't the >> only option, it has a lot going for it. >> >> * Note we may want to rewrite imported images even if they are in PNG >> already: Some programs (notably Photoshop) are pretty bad at >> compressing PNG relative to libpng. >> >> * I opted to include a simple source-tracking tEXT chunk >> to aid import <-> export cycles >> >> * using PNG (and implicitly assuming 256-color sprite data) has the >> side effect that we can play nicer with eg Grafx2, which always writes >> a '256-color' image even though only the first 16 colors are used. > > I'm uneasy about using PNG: > * the biggest problem: paletted PNG files require a palette, which is > useless to us. We could instead encode our images as 4-bit/8-bit > greyscale pngs, but then the PNG is not even human-viewable; isn't > that half of the point of using PNGs? Would we put real palette > information (default palette) in a private chunk? I wasn't sure what to think about this. Because default palettes of the 256-color variety are NOT addressed with the current scheme. Although it looks like we can use negative values in defpal%d.bin to implement that. Anyway I changed to a reload-wrapped blob of pixels. > > * it would be a horrible idea to import whole PNG files. There could > be all kinds of other things in them that could change meaning later > (eg. requesting pixel data from libpng in a different way) > > * requires interfacing with an external library, all this
Re: [Ohrrpgce] better palette options
On Fri, Oct 8, 2010 at 11:42 PM, David Gowers (kampu) <00a...@gmail.com> wrote: > http://bpaste.net/show/10001/ > ^ > The format proposition > Note that I intentionally made it as unrestricted as possible. Imposing additional constraints could allow the removal of certain tuple types and runtime data fields. Whether additional constraints are appropriate is a matter for discussion. Comments on PNG: * I saw your suggestion, Mike, and I thought : though it isn't the only option, it has a lot going for it. * Note we may want to rewrite imported images even if they are in PNG already: Some programs (notably Photoshop) are pretty bad at compressing PNG relative to libpng. * I opted to include a simple source-tracking tEXT chunk to aid import <-> export cycles * using PNG (and implicitly assuming 256-color sprite data) has the side effect that we can play nicer with eg Grafx2, which always writes a '256-color' image even though only the first 16 colors are used. On another topic: * my proposition does not include backgrounds or tilesets. I believe backgrounds can fit into the scheme in a natural way. I'm not sure what to do with tilesets as I understood that some other upgrade relating to them is incipient, but not sure exactly what. * nor does my proposition address partial recoloring. With 256 color sprites, it would be far easier to standardize on recoloring, so, for example, you could fill a 256-color palette via sets of 16, and just swap in/out sets of 16 to recolor things easily. * related to the above: color cycling, while not *important*, might be something we want to consider (eg for showing elemental effects on a character) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
http://bpaste.net/show/10001/ ^ The format proposition ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Fri, Oct 8, 2010 at 2:53 PM, Ralph Versteegen wrote: > On 8 October 2010 14:59, David Gowers (kampu) <00a...@gmail.com> wrote: >> On Fri, Oct 8, 2010 at 9:44 AM, Jeremy Bursey wrote: >>> I'm only subscribed to the mailing digest, not the individual alerts, so I >>> have to cut and paste those comments related to the palette discussion I >>> introduced last night. Hopefully this doesn't come out all messy. >>> >>>>Did james modify my palette in that way? I know he changed it slightly >>>> before >>>>installing as the new default. That's a shame if so; I specifically >>>> designed >>>>it to not waste a single entry. >>> >>> The game I'm updating actually started 10 years ago on the old palette, so I >>> am using most of the original colors. I just had to move all my yellows to >>> the >>> top row to make room for all my "sand" tiles. So that's the specific issue >>> I'm >>> facing. At the moment, using my updated palette turns all the UI colors >>> yellow. >>> >>> I'd like to use yours at some point, actually, but the time it would take me >>> to convert 160 palettes, 300 backdrops, over 1,000 sprites, and so on to the >>> compatable colors would cause me unthinkable pain, and I'm not ready to take >>> that plunge yet. An update to how palettes are handled would certainly make >>> the effort more desirable. >> i wrote an app which does that automatically (CHGPAL). have you tried >> it on a copy of your rpg? >> http://rpg.hamsterrepublic.com/ohrrpgce/Graphics_Utilities > > CHGPAL doesn't handle box borders, portraits or UI colours, and it > uses the old master palette format instead of the new one, but it > shouldn't be much work to update it if desired. > >> It's a simple enough procedure that CUSTOM could conceivably >> implement it as an option on the master palette screen. > > Yes, it would be good. I don't know how to best pick the nearest match > in the new palette for a colour (and should you let the user override > this somehow, as they can when importing backdrops?), but in this case > Jeremy just wants to move colours around the palette. > >>> >>> That's the part that made me question whether this was worth discussing. >>> >>>>have you seen >>>>http://rpg.hamsterrepublic.com/ohrrpgce/Plan_for_256_color_sprites >>> >>> I have, actually. But it was my understanding that this was still light >>> years >>> away from becoming reality, if it was ever meant to become reality. >> >> I'm working on some formal YAML propositions regarding sprite >> upgrades. They have the property of being very succint, so hopefully >> that will help things to happen > > What is a "formal YAML proposition"?? A proposal for a new graphics > format? I'm keen to have that agreed upon. It looks like this O) ^^ 'smiling cyclops with a monocle' emoticon. animproposition1: description: (type)gfx.bin: type: sublumped subst: 'type': [hero, walk, enemy, portrait, ] gfxset: # a set of related frames # <=1 per usage class(eg hero, enemy) # with the possibility of >1 owners datadesc : header: SIGNATURE: METADATA_RECORD_SIZE: metadata_table: W : int16 H : int16 repeats_for : NFRAMES data: DATA : [byte, [H, W]] repeats_for : NFRAMES repeats_for : NSETS (type)anim.bin: subst: 'type': [hero, walk, enemy, portrait, ] anim: code : [[enum, [goto, gotorel, wait]]] param: goto: frame_index gotorel: frame_index_offset wait: nticks repeats_for : N_INSTRUCTIONS_IN_ANIM repeats_for : NSETS # animation patterns and goto+wait1, gotorel+wait1 should be implemented as a ui layer # over this IMO (incomplete so far) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Fri, Oct 8, 2010 at 2:38 AM, James Paige wrote: > On Fri, Oct 08, 2010 at 01:27:58AM +1030, David Gowers (kampu) wrote: >> Did james modify my palette in that way? I know he changed it slightly >> before installing as the new default. That's a shame if so; I >> specifically designed it to not waste a single entry. > > I don't remember making any changes to your palette, and I just checked > the master palette in ohrrpgce.new and I don't see any obvious repeats. True. I was probably thinking of SVN r1950 "Fix new default master palette so that box style 8 and 14 are consistent with the old defaults" Which actually touched OHRRPGCE.MAS, not PALETTES.BIN Though I note there is one duplication (black) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Fri, Oct 8, 2010 at 9:44 AM, Jeremy Bursey wrote: > I'm only subscribed to the mailing digest, not the individual alerts, so I > have to cut and paste those comments related to the palette discussion I > introduced last night. Hopefully this doesn't come out all messy. > >>Did james modify my palette in that way? I know he changed it slightly >> before >>installing as the new default. That's a shame if so; I specifically >> designed >>it to not waste a single entry. > > The game I'm updating actually started 10 years ago on the old palette, so I > am using most of the original colors. I just had to move all my yellows to > the > top row to make room for all my "sand" tiles. So that's the specific issue > I'm > facing. At the moment, using my updated palette turns all the UI colors > yellow. > > I'd like to use yours at some point, actually, but the time it would take me > to convert 160 palettes, 300 backdrops, over 1,000 sprites, and so on to the > compatable colors would cause me unthinkable pain, and I'm not ready to take > that plunge yet. An update to how palettes are handled would certainly make > the effort more desirable. i wrote an app which does that automatically (CHGPAL). have you tried it on a copy of your rpg? http://rpg.hamsterrepublic.com/ohrrpgce/Graphics_Utilities It's a simple enough procedure that CUSTOM could conceivably implement it as an option on the master palette screen. > > That's the part that made me question whether this was worth discussing. > >>have you seen >>http://rpg.hamsterrepublic.com/ohrrpgce/Plan_for_256_color_sprites > > I have, actually. But it was my understanding that this was still light > years > away from becoming reality, if it was ever meant to become reality. I'm working on some formal YAML propositions regarding sprite upgrades. They have the property of being very succint, so hopefully that will help things to happen ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] better palette options
On Thu, Oct 7, 2010 at 4:57 PM, Jeremy Bursey wrote: > Now that I'm upgrading my game's palette a little, I've noticed how painful > it > is to keep track of every graphical element in the game from tilesets to UI > color options. It seems like one little shift of the color placement can > wreck > months, if not years of work if one's not careful. > > I know that a full 16 million color palette is a long way off, if it's ever > destined to become a feature at all, but how reasonable would it be to split > the graphical components (tilesets, sprites, backdrops, etc.) into separate > planes and to allow each plane to use its own palette? > > For example: > > What if maptiles had a dedicated palette that sprites, battle backdrops, > etc. > did not interfere with? Personally I'd like one per tileset. Ditto for backdrops (in the absence of hi/truecolor support) > > What if sprites could dip into their own dedicated palette and promise the > player a consistency of appearance, despite the changes that may happen on > the > surface of the alien world? > > I think since we have the option to store a bunch of palettes in our game > files and since we can switch them on the fly using plotscript commands, why > not go an extra step and dedicate palettes to various graphical components > in > custom and even allow custom to manage them in-game? > > We've already been given the ability to split maps into multiple layers, so > would it be possible to extend this idea into the area where custom > handles palette displays? > > I figure, each component can stick to its current limitations (maptiles and > screens with 256 color palettes and sprites with 16 colors (though, 16 from > a > dedicated sprite palette that may canvas 256)), so it's not like we would > need > to display 257 colors or more from a single component at once. But I think > it > would be great if we could maintain the integrity of our character > walkabouts > from the chosen palette, even though the map itself may boast an all new > flashy palette that the sprite cannot adjust to without its skin turning red > and its hair blue. It would also be nice to maintain our UI display colors, > textbox designs, etc., should we choose to maximize our tileset palettes by > adding a new color to the top left row (colors 1-16) instead of sticking > ourselves with repeated colors, like the ones we have in the default > palette. Did james modify my palette in that way? I know he changed it slightly before installing as the new default. That's a shame if so; I specifically designed it to not waste a single entry. > With the ability to store thousands of custom palettes in each game, it > would > seem silly not to allow a position to use them effectively. > > And on the note of multiple palettes, would it be reasonable to add the > ability to load palettes automatically, instead of relying on a plotscript? > For example, when the player enters a new map or a battle screen, the music > can change. If we're being picky, we could also claim that the graphics > change. Would it be reasonable to add a bitset to all relevant gaming areas > that loads either the "default" palette or a custom palette? Again, this > would > only be helpful if sprites, slices, textboxes, UI colors, and so forth could > be assigned their own palettes so that graphics don't end up clashing with > one > another. But if any of this is possible, it would certainly upgrade the > OHR's > graphic ability to a stellar level. > > This may have already been discussed at some point, but I'd like to bring it > up again in case the ideas for it had hit a standstill. It's bothered me for > a > long time how uncooperative a single palette can be, especially when one > considers how many colors are represented in the real world. It also > frustrates me to know that all those extra palettes sitting in my game's > storage are there really only to tease me. "Come play with us," they say, > and > yet I can do no such thing because that would mean changing the lead hero's > ethnicity from Caucasion to Frog. > I think splitting the palette display into > sections and giving Custom the ability to assign specific palettes to > specific > graphic components would really improve a game's overall presentation > without > having to rewrite the entire color code into a massive 16 million color > display. Truecolor would actually be much easier to implement, and simpler, than the current system. Though some things would be unavoidably slower. In any case, we would need such code in order to display >256 colors total onscreen. > > Please consider this if it's at all possible to do such a thing. I'm posting > here instead > of bugzilla since discussions about improvements seem more active here. have you seen http://rpg.hamsterrepublic.com/ohrrpgce/Plan_for_256_color_sprites ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrp
Re: [Ohrrpgce] [Bug 847] New: Stop using Capslock
On Sun, Sep 26, 2010 at 1:50 AM, Ralph Versteegen wrote: > On 25 September 2010 18:44, David Gowers (kampu) <00a...@gmail.com> wrote: >> On Sat, Sep 25, 2010 at 2:49 PM, Ralph Versteegen wrote: >>> On 25 September 2010 16:56, David Gowers (kampu) <00a...@gmail.com> wrote: >>>> On Sat, Sep 25, 2010 at 1:52 PM, Ralph Versteegen >>>> wrote: >>>>> Ah! Good point. But that also argues against holding down space to >>>>> scroll, doesn't it? >>>> >>>> Yes, I don't like the idea of holding down space to scroll; rather >>>> just press it to toggle continuous scrolling (4 steps/second?), and >>>> use cursor keys to adjust scroll direction (not inertially, just to >>>> choose from the 8 possible scroll directions) >>> >>> So you're actually talking about the key repeat rate. >> insofar as there should be no key repeat at all in this context, yes. >> I actually meant semi-automatic scrolling, you just adjust the >> direction and stop/start it as need be. > > Oh, that is what I meant by inertial scrolling. I don't see why you > would want it - normally you would be scrolling < 10 pixels, carefully > lining up to the right pixel. Is there some kind of UI design > philosophy at play here? Not really, just thinking of my experience with fiddling around with sprite alignment to make things animate right. However, it probably isn't appropriate on it's own.. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] [Bug 847] New: Stop using Capslock
On Sat, Sep 25, 2010 at 2:49 PM, Ralph Versteegen wrote: > On 25 September 2010 16:56, David Gowers (kampu) <00a...@gmail.com> wrote: >> On Sat, Sep 25, 2010 at 1:52 PM, Ralph Versteegen wrote: >>> Ah! Good point. But that also argues against holding down space to >>> scroll, doesn't it? >> >> Yes, I don't like the idea of holding down space to scroll; rather >> just press it to toggle continuous scrolling (4 steps/second?), and >> use cursor keys to adjust scroll direction (not inertially, just to >> choose from the 8 possible scroll directions) > > So you're actually talking about the key repeat rate. insofar as there should be no key repeat at all in this context, yes. I actually meant semi-automatic scrolling, you just adjust the direction and stop/start it as need be. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] [Bug 847] New: Stop using Capslock
On Sat, Sep 25, 2010 at 1:52 PM, Ralph Versteegen wrote: >>> 1) whenever S is the active tool, arrow keys scroll the sprite >>> 2) when S is active, press space/mouse button to start scrolling, and press >>> again to stop >> This seems interesting. I'd like to combine it with : >>> 3) when S is active, press and hold space/mouse button and press arrow keys >>> to >>> scroll >> So that you press space/MB to start,arrows to alter direction, space/MB to >> stop. > > Isn't that exactly 2)? By "start scrolling", I meant "use arrow keys > to scroll". I think that you're talking about inertial scrolling (or > thought that I was), which I can't see as useful. I meant.. well, see below. > >>> 4) whatever tool is active, hold down S and press arrow keys to scroll. S is >>> temporarily made the active tool, and the previous tool becomes active again >>> when S is released. >>> >>> I don't see how 1) could work with the mouse (note that the mouse can't >>> currently be used to scroll at all). >> Drag the image to scroll it? > > In the sprite editor moving the mouse is mostly equivalent to pressing > arrow keys, so I was working under that assumption. So dragging the > image to scroll it was 3). But we could treat the two input methods > differently. Then my preference is your suggestion: 1) for arrow keys, > plus scrolling by dragging with the mouse. (Having to press both S and > space to start scrolling sounds annoying to me). > >>> My preference is either 1) or 4) >> >> IMO that resolves to 1. Because depending on simultaneous detection of >>>1 non-modifier keys is not portable between keyboards AFAIK > > Ah! Good point. But that also argues against holding down space to > scroll, doesn't it? Yes, I don't like the idea of holding down space to scroll; rather just press it to toggle continuous scrolling (4 steps/second?), and use cursor keys to adjust scroll direction (not inertially, just to choose from the 8 possible scroll directions) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] [Bug 847] New: Stop using Capslock
On Sat, Sep 25, 2010 at 11:49 AM, wrote: > http://rpg.hamsterrepublic.com/bugzilla/show_bug.cgi?id=847 > > It's about time we stopped using Capslock for scrolling graphics in the sprite > editors: > -It doesn't work properly with old versions of SDL that people might have > installed on Linux > -No key presses are reported at all in Mac OS X even with the latest version > of > SDL > -It's annoying even when it does work Some keyboard layouts don't even have one (eg colemak, my current layout, replaces it with another Backspace.) > > We could switch to Ctrl; I think that's safe. Shift isn't used either, but I'd > rather make that the fast movement key, like it is in several other editors. > > However, I feel that it's better to making scrolling a normal tool like the > others: that's more consistent and removes a magic key. Assume the shortcut is > S. Some possible control schemes: > 1) whenever S is the active tool, arrow keys scroll the sprite > 2) when S is active, press space/mouse button to start scrolling, and press > again to stop This seems interesting. I'd like to combine it with : > 3) when S is active, press and hold space/mouse button and press arrow keys to > scroll So that you press space/MB to start,arrows to alter direction, space/MB to stop. > 4) whatever tool is active, hold down S and press arrow keys to scroll. S is > temporarily made the active tool, and the previous tool becomes active again > when S is released. > > I don't see how 1) could work with the mouse (note that the mouse can't > currently be used to scroll at all). Drag the image to scroll it? > My preference is either 1) or 4) IMO that resolves to 1. Because depending on simultaneous detection of >1 non-modifier keys is not portable between keyboards AFAIK BTW, amazing work on the OSX port! I was certainly surprised :) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] scons+geany minor crazyness
On Thu, Sep 23, 2010 at 2:48 AM, James Paige wrote: > I use geany as my text editor for OHR programming, but today when I > happened to have SConscript and ohrbuild.py open in it while I was > working on the recursive include detection, my geany instance would > silently crash about 50% of the time when I would run scons. > > I am guessing that this is actually a geany problem, not a scons > problem. Geany detects external file modification so it can prompt you > to reload changes, and I am guessing it was interacting badly with > whatever method of "touching" files that scons is using. Very strange. I used geany to create and edit those two files and it never crashed for me. OTOH I'm using an svn build (r5237, to be precise.) and the bug may have already been fixed. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: kampu/3758 SConstruct based build system. Builds game, custom, and bam2mid currentl
On Tue, Sep 21, 2010 at 3:34 PM, Seth Hetu wrote: > Yes, Mike is right about that. And I checked; David didn't disable copying. > > I just want to say that I am now TOTALLY confused by this error. After > deleting "build" once (and then re-sconsing), the system now works > exactly as expected. If I change some code in wip/udts.bi, running > "scons" again will copy it to build/ and compile the relevant module. > > Ralph, if you delete build, does it build correctly? > > Maybe making that "clean" target delete build/ wasn't such a bad idea > after all... > > Willing to test any future changes, but otherwise I'm out of ideas, > -->Seth I thought that I'd already committed these changes, but apparently not: "SCons: revert separate build dir, as it seems to cause a few dep problems" Just committed ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: kampu/3758 SConstruct based build system. Builds game, custom, and bam2mid currentl
On Tue, Sep 21, 2010 at 9:59 AM, Ralph Versteegen wrote: >>> OK, now the killer problem: it doesn't rebuild anything when I edit a >>> file!! >>> Even if I run scons -c, it doesn't rebuild anything afterwards >>> (but maybe -c is completely unsupported?) >> -c works fine for me. >> Did you change the *content*, not just edit something in and out again? > > Yes, I changed something. To clarify : if I add a comment to game.bas, game.o will be rebuilt, because I altered the MD5 sum. then ohrrpgce-game will *not* be rebuilt, because game.o will have the same MD5 sum as before. If your result differs from this, I agree it's a problem. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: kampu/3766 Some Scons fixes suggested by TMC
On Tue, Sep 21, 2010 at 9:02 AM, Ralph Versteegen wrote: > On 20 September 2010 20:54, wrote: >> kampu >> 2010-09-20 01:54:38 -0700 (Mon, 20 Sep 2010) >> 33 >> Some Scons fixes suggested by TMC >> --- >> U wip/SConscript > > You seem to have missed my changes in r3764: it's not necessary to > import all of os.environ, just PATH. But the default ENV will contain > variables like HOME and TEMP, without which gcc crashes. I already > fixed this by adding "env['ENV']['PATH'] = os.environ['PATH']" > > But if you think it's cleaner to just import all of os.environ, that's > up to you. I saw that change, I just wasn't sure what to do with it. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: kampu/3758 SConstruct based build system. Builds game, custom, and bam2mid currentl
On Mon, Sep 20, 2010 at 5:58 PM, Seth Hetu wrote: > David, I get a weird error building with the latest (SVN) source & the > latest (SVN) sconscript file: > - > fbc -lang deprecated -c build\loading.bas -o build\game-loading.o -mt > -g -exx -s gui -d IS_GAME -m game > build\loading.bas(554) error 56: Illegal specification, at parameter 1 > (zmap) of CleanZoneMap() > SUB CleanZoneMap(zmap as ZoneMap, BYVAL wide as integer, BYVAL high as > integer) > ^ > build\loading.bas(555) error 25: Invalid data types > WITH zmap > ^ > (and more errors) > --- > Any ideas? Is this an error related to what FB version you're running? I cannot produce such an error either through 'make' or 'scons' (I suspect TMC might know, since he recently wrote the zonemap code) FreeBASIC 0.21.1 (08-11-2010) here. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: kampu/3758 SConstruct based build system. Builds game, custom, and bam2mid currentl
On Sun, Sep 19, 2010 at 9:08 PM, Ralph Versteegen wrote: > On 19 September 2010 11:37, David Gowers (kampu) <00a...@gmail.com> wrote: >> On Sun, Sep 19, 2010 at 8:49 AM, James Paige >> wrote: >>> On Sat, Sep 18, 2010 at 04:05:49PM -0700, subvers...@hamsterrepublic.com >>> wrote: >>>> kampu >>>> 2010-09-18 16:05:49 -0700 (Sat, 18 Sep 2010) >>>> 112 >>>> SConstruct based build system. Builds game, custom, and bam2mid currently. >>>> >>>> Still needs more testing on Windows. >>>> --- >>>> A wip/SConscript >>>> A wip/SConstruct >>>> A wip/ohrbuild.py >>> >>> Fails for me on compiling audwrap >>> >>> I don't know if audwrap has ever been compileable on my machine, as I >>> test with music_sdl almost 100% of the time. >>> >> Good point. It is a little overzealous. I've attached a fixed version >> which only adds it as a default target when it's actually going to be >> used. If it works for you, I'll commit. > > I see you forgot music_silence in the list of supported backends. > > When I try to compile, gcc crashes. gcc of course runs just fine if I > run it with the same options from the commandline. I added -v to the > gcc options and saw that it crashes right before it prints out its > commandline arguments and environmental information. I managed to fix > the problem by going back to ENV = os.environ. After a minute, and > thinking back to the man page, the problem is obvious: your fix > overwrites all the other environmental variables. > > Committed those two fixes. > > I can't figure out why at the moment, but the gfx commandline option > doesn't cause gver.txt to change. what happens if you quote it? > build/gver.txt appears to be a copy > of gver.txt: it has the same modified time. For this reason, building > anything other than the default backends either fails to link or > doesn't let me run them. Dependency bug. > > I see you added a rule to force gcc to be used for .c files. However, > audwrap is built completely differently, and you create an environment > for its build where you don't set CC and CXX, so it tries to run VC++. > I can't understand why you created another environment for it, so I > left this problem for you to fix. > > Switching from extra_env to env , I get: > g++ -c build\audwrap\audwrap.cpp -o build\audwrap\audwrap.o -c -g -O3 > build\audwrap\audwrap.cpp:9:21: error: audwrap.h: No such file or directory dependency bug. Easily fixed. > > OK, now the killer problem: it doesn't rebuild anything when I edit a > file!! > Even if I run scons -c, it doesn't rebuild anything afterwards > (but maybe -c is completely unsupported?) -c works fine for me. Did you change the *content*, not just edit something in and out again? >I still haven't tested on > Linux. Is rebuilding working for you? Certainly. > > Also, any reason to not place built executables in wip/ instead of > wip/build? It probably more common in other projects, but we already > have all the dlls there, and various distribution scripts which assume > that location. Nope, I just thought it was not suitable.but, changed locally. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: kampu/3758 SConstruct based build system. Builds game, custom, and bam2mid currentl
On Sun, Sep 19, 2010 at 8:49 AM, James Paige wrote: > On Sat, Sep 18, 2010 at 04:05:49PM -0700, subvers...@hamsterrepublic.com > wrote: >> kampu >> 2010-09-18 16:05:49 -0700 (Sat, 18 Sep 2010) >> 112 >> SConstruct based build system. Builds game, custom, and bam2mid currently. >> >> Still needs more testing on Windows. >> --- >> A wip/SConscript >> A wip/SConstruct >> A wip/ohrbuild.py > > Fails for me on compiling audwrap > > I don't know if audwrap has ever been compileable on my machine, as I > test with music_sdl almost 100% of the time. > Good point. It is a little overzealous. I've attached a fixed version which only adds it as a default target when it's actually going to be used. If it works for you, I'll commit. SConscript Description: Binary data ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewroteMakefile, for fast builds!)
All outstanding issues have been addressed, except the "'clean'
target" one mentioned by Seth. I haven't been able to commit since my
login no longer seems to work, so I'll just attach these, for now.
(I have emailed James about the login, but thought that might take
some time to work out.)
SConscript
Description: Binary data
SConstruct
Description: Binary data
#!/usr/bin/env python
import os
import platform
import re
import fnmatch
import sys
include_re = re.compile(r'^#include\s+"(\S+)"$', re.M)
standard_bi = ['crt.bi', 'fbgfx.bi', 'crt/limits.bi',
'file.bi', 'allegro.bi']
def basfile_scan(node, env, path):
contents = node.get_text_contents()
tmp = include_re.findall(contents)
if tmp:
for v in standard_bi:
while v in tmp:
tmp.remove (v)
for v2 in list (tmp):
if 'SDL' in v2:
tmp.remove (v2)
return tmp
def verprint (used_gfx, used_music, svn, git, fbc):
# generate cver.txt, gver.txt (gver is just a cver with removed bits)
# generate iver.txt (Install-info)
# generate distver.bat (?)
def openw (filename):
return open (os.path.join (filename), 'wb')
import datetime
results = []
supported_gfx = []
f = open ('../codename.txt','rb')
codename = f.readline().rstrip()
f.close()
# now automagically determine branch and svn
def missing (name, message):
tmp ="%r executable not found. It may not be in the PATH, or simply not installed." % name
tmp += '\n' + message
print tmp
def query_svn (*command):
from subprocess import Popen, PIPE
import re
date_rex = re.compile ('Last Changed Date: ([0-9]+)-([0-9]+)-([0-9]+)')
rev_rex = re.compile ('Last Changed Rev: ([0-9]+)')
date = datetime.date.today().strftime ('%Y%m%d')
rev = 0
output = None
try:
f = Popen (command, stdout = PIPE, stderr = PIPE)
output = f.stdout.read()
except WindowsError:
missing (command[0], 'version output may be wrong as a result.')
output = ''
except OSError:
missing (command[0], 'version output may be wrong as a result.')
output = ''
if date_rex.search (output):
date = date_rex.search (output).expand ('\\1\\2\\3')
if rev_rex.search (output):
rev = int (rev_rex.search (output).expand ('\\1'))
return date, rev
def query_fb ():
from subprocess import Popen, PIPE
import re
rex = re.compile ('FreeBASIC Compiler - Version (([0-9a-f.]+) ([0-9()-]+))')
try:
f = Popen ([fbc,'-version'], stdout = PIPE)
except WindowsError:
missing (fbc,'FBC is necessary to compile. Halting compilation.')
sys.exit (0)
except OSError:
missing (fbc,'FBC is necessary to compile. Halting compilation.')
sys.exit (0)
output = f.stdout.read()
if rex.search (output):
return rex.search (output).expand ('\\1')
return '??.??.? (-??-??)'
name = 'OHRRPGCE'
date, rev = query_svn (svn,'info')
if rev == 0:
date, rev = query_svn (git,'svn','info')
fbver = query_fb ()
for g in used_gfx:
if g.upper() in ('SDL','FB','ALLEG','DIRECTX','SDLPP'):
results.append ('#DEFINE GFX_%s_BACKEND' % g.upper())
supported_gfx.append (g)
for m in used_music:
if m.upper() in ('NATIVE','SDL','NATIVE2'):
results.append ('#DEFINE MUSIC_%s_BACKEND' % m.upper())
results.append ('#DEFINE MUSIC_BACKEND "%s"' % m)
results.append ('#DEFINE SUPPORTED_GFX "%s "' % ' '.join (supported_gfx))
tmp = ['gfx_choices(%d) = @%s_stuff' % (i, v) for i, v in enumerate (supported_gfx)]
results.append ("#define GFX_CHOICES_INIT " +\
" : ".join (['redim gfx_choices(%d)' % (len(supported_gfx) - 1)] + tmp))
gfx_code = 'gfx_' + "+".join (supported_gfx)
music_code = 'music_' + "+".join (used_music)
data = {'name' : name, 'codename': codename, 'date': date,
'rev' : rev, 'fbver': fbver, 'music': music_code,
'gfx' : gfx_code}
results.extend ([
'CONST version as string = "%(name)s %(codename)s %(date)s"' % data,
'CONST version_code as string = "%(name)s Editor version %(codename)s"' % data,
'CONST version_revision as integer = %(rev)d' % data,
'CONST version_branch as string = "%(codename)s"' % data,
'CONST version_build as string = "%(date)s %(gfx)s %(music)s"' % data,
('CONST long_version as string = "%(name)s '
'%(codename)s %(date)s.%(rev)s %(gfx)s/%(music)s FreeBASIC %(fbver)s"') % data])
#write to /tmp/ for now, ca le nu cipra
f = openw ('cver.txt')
f.write ('\n'.join (results))
f.write ('\n')
f.close()
for v in list (results):
if v.startswith('CONST
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewroteMakefile, for fast builds!)
On Fri, Sep 17, 2010 at 7:44 PM, Ralph Versteegen wrote:
> On 17 September 2010 19:56, David Gowers <00a...@gmail.com> wrote:
>>
>> (in case this appears in a double post.. this is the one with the
>> correct updated SConscript)
>
> I was really amazed when I ran scons... and the visual c++ commandline
> compiler spat back that it didn't recognise the gcc command line
> arguments. I'd completely forgotten I had VC++ installed (only used it
> once), and I certainly don't have it in my PATH. Looks like SCons'
> compiler autodetection is too clever.
>
> But no matter what I do, I can not get scons to use gcc instead! I've
> tried an awful lot of suggestions (like setting CC and CXX) I've found
> online and in the man page, and your tools = ['mingw'], but nothing
> will cause SCons to budge.
I can, don't worry.
I already did it for bas files. same thing.
>
> In the end, I can only get past this problem by commenting out the
> lines to build audwrap.
>
> The good news is that once I do so, game.exe and custom.exe build!
> They work, except I notice that you aren't specifying the correct
> FBFLAGS when linking. You need to pass whatever subset of -exx -g -s
> gui -mt, just like when compiling each source file. This also explains
> why you see a warning about inconsistent multithreading options.
ooh. So the inconsistency is from linking with wrong opts.
>
> I tried changing the backends by setting OHRGFX in my shell, and found
sorry, I broke that (somewhat intentionally) when I introduced the
'gfx' + 'music' options. I should support both. for now,
scons gfx=xyz music=abc is the way which works.
> that the script failed to rebuild modules as required (the
> 'semicommon' modules plus game.bas, custom.bas should be rebuilt). So
> I had to clean/delete build/ over and over while testing.
> After cleaning, I see that the order of backends in OHRGFX is ignored.
> The string of backends (passed to verprint) specifies not only which
> gfx backends to include, but also which order to try them in
> (specified by the order of appearance in gfx_choices).
ok.
sorry if I'm not very descriptive, just switched to colemak kb layout
and find my typing inept (+slllow).
>
> The following is unnecessary & broken (after fixing _ -> \, I get
> "TypeError: Command() takes exactly 4 arguments (2 given)"), so just
> remove it:
> if 'sdlpp' in used_gfx:
> sdlpp = Command ('cd gfx_sdl_source;make old;cd ..;cd ..')
> gfx_sdl.dll is provided; I can't remember, but it may not even be
> possible to build it with gcc.
>
> In music_map, you're using libpath instead of libpaths.
>
> allegro.bi needs to be added to standard_bi for gfx_alleg to build.
>
> music_native did not build without a little help, because
> audwrap/audwrap.bi includes "../lumpfile.bi", and build/lumpfile.bi
> was missing. However it was probably missing only because I had just
> deleted build/
>
> Other than the above problems, I've tested that all of gfx_fb,
> gfx_sdl, gfx_directx, gfx_alleg, gfx_sdlpp, music_sdl, music_native,
> music_native2 link and run. Pretty good progress!
Great :)
All your comments have been noted, some are fixed.
>
> Could you please check the build files into svn? Merging my changes
> with the new file you send every half hour is becoming pretty
> annoying; we have version control to handle such things. :)
k.
Working on it.
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewroteMakefile, for fast builds!)
On Fri, Sep 17, 2010 at 6:17 PM, Seth Hetu wrote: >> Yeah, I haven't disabled it or anything. >> It works fine. >> >> (in case this appears in a double post.. this is the one with the >> correct updated SConscript) > > Sorry, I'm continuing the crazy double-posting. > > I just tried "scons -c" ---it cleans all of the OBJECT files in > "build". The various ".bas" and ".bi" files that it copies into > "build" remain there. > > Just FYI. I have no problem deleting the build directory manually, but > since you've already isolated your build directory, why not just add a > clean target? After a bit of thought, I'd say that removing the object files is a better behaviour than removing build/. But I can add a 'clean' target, I suppose. Anyway, this is the latest version. It tries to enforce mingw on windows. If it doesn't succeed, please inform me, and try changing the line tools = ['mingw'] to tools = ['default','mingw'] to see if that produces the desired behaviour. SConscript Description: Binary data ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewroteMakefile, for fast builds!)
On Fri, Sep 17, 2010 at 5:53 PM, Ralph Versteegen wrote:
> On 17 September 2010 19:09, David Gowers <00a...@gmail.com> wrote:
>> On Fri, Sep 17, 2010 at 3:32 PM, Ralph Versteegen wrote:
>>> On 16 September 2010 23:42, David Gowers <00a...@gmail.com> wrote:
>>>> On Thu, Sep 16, 2010 at 8:35 PM, Mike wrote:
>>>>> IIRC, you need to include -m game while compiling game.bas itself.
>>>>> Possibly while compiling all modules.
>>>
>>> So far I've only tried it under windows. I got much further than Seth.
>>> I didn't have his problems with svn. "scons env="PATH:%PATH%"" doesn't
>>> work for me; I doubt it's correct. Instead I appended the Environment
>>> constructor call:
>>>
>>> env = Environment (OHRGFX = env.get('OHRGFX','sdl+fb'),
>>> ...
>>> ENV = os.environ)
>>
>> This is specifically noted as probably evil and probably not what you
>> want, in the SCons docs;
>> So I specifically imported only the PATH ('PATH = env[path]')
>> (In my fixed ver, which I started working on when I saw Seth's email)
>
> That's not right, it should be:
> ENV = {'PATH' : env['PATH']},
Indeed, thanks.
Fixed.
>
>> Well, you have done mainly good fixes, with a few weird happenings.
>>
>> What you did with blit.o appears to be one of the weird things.
>> My original line was definitely not correct, you remedied it to be
>> incorrect in a different way ^_^ Looking at makegame.bat, it's pretty
>> clear that on windows, we want to just link ``win32\blit.o`` and
>> ``win32\base64.o``, rather than compiling object files which we can
>> then link.
>
> I assumed you wanted to compile them.
Oh.
I assumed they could not be readily compiled, and thus were provided
as .o files.
Oops.
> Using prebuilt object files isn't necessarily better; they need to be
> maintained which can mean an annoying reboot into Windows. Eventually
> when the new script interpreter (in C and C++) is merged in, gcc will
> probably become required.
So you suggest just requiring them to be compiled with gcc?
okay, amended SConscript follows (in my next email in reply to Seth)
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewroteMakefile, for fast builds!)
On Fri, Sep 17, 2010 at 5:05 PM, Seth Hetu wrote: > David, if you're trying to fix my errors, I might point out that TMC's > patch fixed everything up to: > ---`build\blit.o' not found > ...which is the error you just described in this email. So we're both > on the same page. > > (Note that I still have my hacky SVN fix in place; TMC's patch only > fixed the ENV's path and a few minor points.) .. That would mean that svn is not in the path, AFAICS. but then, how would have VERPRINT.BAS found it? *confused* Well, I'll allow it to be explicitly specified, as in 'scons svn=C:\some\path\svn.exe' then you should be able to have it in whatever weird location you desire, named whatever. (similar options are also added for git and fbc) Amended SConscript attached :) SConscript Description: Binary data ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewroteMakefile, for fast builds!)
On Fri, Sep 17, 2010 at 1:18 PM, Seth Hetu wrote: > Currently having some trouble building this on Windows. If I just run > scons from the root source directory, I get: > > --- > ...it seems to be unable to load the current revision information from > SVN. (I am using this on a freshly checked-out source, using SVN). Any > ideas? What is happening is that the 'svn' command is not found. I need to catch WindowsError as well too, though.. so it needs a fix anyway (Windows behaviour WRT popen is sadly uninformative and misleading, but I got some experience with it last week debugging jbofacki, so I know what this kind of printout means.) > > > If I tweak the code to return a random date/revision (hooray, > standards!) then I get: > --- > 'fbc' is not recognized as an internal or external command, > operable program or batch file. > yup, looks like path problems. As TMC pointed out, PATH needs to be explicitly imported to make Windows happy; I didn't realize this. --- > This is VERY odd, because if I type "fbc" in the command line, I > immediately get Free Basic's help text. > > > I tried to hack around this by manually passing the path to scons: > scons env="PATH:%PATH%" > but then I got: > --- > scons: *** Do not know how to make File target `Shared\Windows' > (D:\visual_studio_projects\OHRRPGCE\Shared\Windows). Stop. what. I never specify any target that looks remotely like the above, nor is it referenced in the sourcecode. <_|. <_< >_> > --- > ...which might either be the first real error with the script, or some > artifact from me forcing through the path/changing the SVN revision, > etc. Better to figure out the first two first, I think. Thanks a lot for the feedback Seth. I've attached an updated SConscript which hopefully fixes these problems (and some others that TMC noticed, thanks TMC) Tell me if you get the same Shared\Windows error. (the PATH issue should definitely be fixed) > > Question: can you still run the default 'clean' target? > E.g.: > http://www.scons.org/doc/production/HTML/scons-user/x335.html Yeah, I haven't disabled it or anything. It works fine. (in case this appears in a double post.. this is the one with the correct updated SConscript) SConscript Description: Binary data ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewroteMakefile, for fast builds!)
On Fri, Sep 17, 2010 at 3:32 PM, Ralph Versteegen wrote:
> On 16 September 2010 23:42, David Gowers <00a...@gmail.com> wrote:
>> On Thu, Sep 16, 2010 at 8:35 PM, Mike wrote:
>>> IIRC, you need to include -m game while compiling game.bas itself. Possibly
>>> while compiling all modules.
>
> So far I've only tried it under windows. I got much further than Seth.
> I didn't have his problems with svn. "scons env="PATH:%PATH%"" doesn't
> work for me; I doubt it's correct. Instead I appended the Environment
> constructor call:
>
> env = Environment (OHRGFX = env.get('OHRGFX','sdl+fb'),
> ...
> ENV = os.environ)
This is specifically noted as probably evil and probably not what you
want, in the SCons docs;
So I specifically imported only the PATH ('PATH = env[path]')
(In my fixed ver, which I started working on when I saw Seth's email)
>
> It's interesting that SCons ignores the environment's PATH by default.
> While this is mostly fine on Unix, it means you are pretty much always
> going to have to add the external PATH to env under Windows if you're
> calling any external programs.
Good point, thanks.
>
> I found and fixed quite a few other problems (see patch), however I
> get stuck at this point:
>
> scons: *** [build\game.exe] Source `build\blit.o' not found, needed by
> target `build\game.exe'.
> scons: building terminated because of errors.
>
> Looks like the rules for .c files are default, so I have no idea
> what's going on here.
Well, you have done mainly good fixes, with a few weird happenings.
What you did with blit.o appears to be one of the weird things.
My original line was definitely not correct, you remedied it to be
incorrect in a different way ^_^ Looking at makegame.bat, it's pretty
clear that on windows, we want to just link ``win32\blit.o`` and
``win32\base64.o``, rather than compiling object files which we can
then link.
>
>> -- ie build game and custom.
>> It needs testing on Windows.. in particular, I don't know whether
>> --std=c99 will work for some of the Windows music .c source files
>> (it's currently used for all C compilation)
>
> There are no music .c source files (there was a typo: audwrap.c should
> be audwrap.o).
.. or audwrap/audwrap.o?
that's what it looks like to me.
> Note that audwrap and music_native work under linux
> (IIRC, you should be able to compile audwrap even if you don't have
> audiere installed, but you can't link the whole program).
Okay, thanks for the hint. (I've now linked that into the build process)
> music_native2 is windows-only.
>
> It looks like you're using a mix of compiling .c files, and using the
> precompiled audwrap.o files. Currently we don't require gcc to compile
> the OHR under windows. The Makefile doesn't assume gcc is available
> either. Whether or not the scons build system depends on gcc is a
> completely independent question;
It doesn't; it will autodetect C and C++ compilers.
> but I suggest sticking to one style
> or the other.
>
yeah, point taken.
Updated SConscript attached in my reply to Seth following this.
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewroteMakefile, for fast builds!)
On Thu, Sep 16, 2010 at 8:35 PM, Mike wrote: > IIRC, you need to include -m game while compiling game.bas itself. Possibly > while compiling all modules. Thanks muchly Mike, that fixed it :D The attached SConscript should have functionality roughly equivalent to TMC's Makefile (plus dependency auto-discovering, and not needing verprint.bas) -- ie build game and custom. It needs testing on Windows.. in particular, I don't know whether --std=c99 will work for some of the Windows music .c source files (it's currently used for all C compilation) Currently building is done in a build/ subdir. This obsoletes eg 'clean' targets (just 'rm -r ./build') and keeps .o files from cluttering up the source directory. I'll look at the reload stuff etc tomorrow. Also the utils, and the distrib commands. That should just about cover the functionality and I can start making it look more organized :) SConscript Description: Binary data ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewrote Makefile, for fast builds!)
On Thu, Sep 16, 2010 at 5:52 PM, Ralph Versteegen wrote: > On 16 September 2010 20:01, David Gowers <00a...@gmail.com> wrote: >> This is going a little slower than I thought. > > Should have figured; the Makefile took me a whole evening :) I've got it in a reasonable state now. Stick the above two files in the OHRRPGCE dir and run "scons" (assuming you have it installed) For some reason I haven't been able to decipher, it doesn't link,though. It will build all the .o files and then claims that it can't find the main() function (despite the -m game option, and game being included in the linking list) . Ditto for custom. >> fbc -lang deprecated -c reloadext.bas -mt -g -exx >> fbc -lang deprecated -c slices.bas -mt -g -exx >> ar rc libohrcommon.a blit.o base64.o gfx_sdl.o gfx_fb.o gfx_sdl.o >> allmodex.o backends.o lumpfile.o compat.o bam2mid.o common.o browse.o >> util.o loading.o reload.o reloadext.o slices.o >> ranlib libohrcommon.a > > Read the Makefile very careful: files needs to be compiled with either > -d IS_GAME or -d IS_CUSTOM, although of all the common modules, only > $(semicommon_sources) actually depend on these defines and need > rebuilding. Yeah, I noticed that. Fixed. SConscript Description: Binary data SConstruct Description: Binary data ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewrote Makefile, for fast builds!)
This is going a little slower than I thought.
I've got bam2mid building (easy);
Just now, I've made it collect all the common object files
into a library ('libohrcommon.a')
scons: done reading SConscript files.
scons: Building targets ...
scons: `all' is up to date.
fbc -lang deprecated -c gfx_sdl.bas -mt -g -exx
fbc -lang deprecated -c gfx_fb.bas -mt -g -exx
fbc -lang deprecated -c allmodex.bas -mt -g -exx
fbc -lang deprecated -c backends.bas -mt -g -exx
fbc -lang deprecated -c lumpfile.bas -mt -g -exx
fbc -lang deprecated -c compat.bas -mt -g -exx
fbc -lang deprecated -c bam2mid.bas -mt -g -exx
fbc -lang deprecated -c common.bas -mt -g -exx
fbc -lang deprecated -c browse.bas -mt -g -exx
fbc -lang deprecated -c util.bas -mt -g -exx
fbc -lang deprecated -c loading.bas -mt -g -exx
fbc -lang deprecated -c reload.bas -mt -g -exx
fbc -lang deprecated -c reloadext.bas -mt -g -exx
fbc -lang deprecated -c slices.bas -mt -g -exx
ar rc libohrcommon.a blit.o base64.o gfx_sdl.o gfx_fb.o gfx_sdl.o
allmodex.o backends.o lumpfile.o compat.o bam2mid.o common.o browse.o
util.o loading.o reload.o reloadext.o slices.o
ranlib libohrcommon.a
.. currently, dependencies are working well -- dependencies are
automagically detected from the source files, and modifying the
respective .bi dependencies or main .bas file triggers a rebuild.
One thing that I found slightly misleading was that, in some
circumstances, SCons will rebuild .o files but not relink the library!
This happens when the .o files are all identical to the last build
(for example, it happens if you simply add a comment somewhere in the
file.. the compiled result is the same.)
This is a bit smarter than I'm used to :) (It occurs because SCons
compares MD5 hashes to determine whether there is actually anything
new created, or just a duplicate of the old result.)
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] New build system (Was: SVN: teeemcee/3714 I rewrote Makefile, for fast builds!)
On Thu, Sep 16, 2010 at 9:30 AM, Ralph Versteegen wrote: >> >> We should use SCons >> >> --- >> James > > Scons has a lot of prestige, so I'm not surprised that it's the first > suggestion, but there are other options: any other suggestions? > > Has anyone used Scons extensively who can confirm that it's a good idea? I haven't used Scons *extensively* (I use it to build all the docs for my various software.). I know enough about it that I'd be willing and able to create an SCons build for OHRRPGCE. I find it quite capable (and I'll say no more, in order to not bias happenings) ... I currently have most of an SConstruct file for OHRRPGCE made, including an implementation of verprint that is readable :) I'll post it later for perusal. Re: seth's mail: IMO, since scons is extensible, built-in FreeBasic support is not necessary. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sun, Sep 12, 2010 at 9:35 PM, Ralph Versteegen wrote:
> On 12 September 2010 19:18, David Gowers <00a...@gmail.com> wrote:
>
> I always suspected I was exploiting a bug; I guess that it was fixed.
> It was too good to be be true anyway.
>>>>> c = memmap ('/tmp/kamplita',dtype = 'B', mode = 'r', shape = (4,3), order
>>>>> = 'F')
>>>>> c
>> memmap([[120, 121, 122],
>> [120, 121, 122],
>> [120, 121, 122],
>> [120, 121, 122]], dtype=uint8)
>>
>> ^^^ xyz xyz xyz xyz :)
>>
>> This loses the fields. In the case of .L, this is not a great loss at
>> all IMO. (we could probably do some automagic that would slow down
>> reading but allow us to use fields. Whether that's worthwhile is
>> another question :)
>
> I wouldn't consider efficient access to .L data important; and it's
> the only lump mangled in this way.
I was more considering whether implementing the wrapper was worth the
time. But I just realized, it can be done readily and without any
appreciable slowdown (I <3 the automagic of python attributes + class
inheritance)
..
**later**
It is done! (actually, there were a few planar formats that needed
these special wrapper classes)
(and now, map access works to some extent. oo.rst now tests
doorlink and door def access -- mainly because I was
anticipating them to mess up more than any simple linear format.)
>
> I'm planning to replace the .L lump in the near future anyway.
Yeah, I was looking at that.
(and your .Z lump def. When vikings gets to using that, I'll
write a reader)
>> Not fixed. Anyway, that API is obsolete now with the OO RPGDir,
>> RPGFile, OhrData etc API.
>>
>> (in which eg `general.passcode` form (attribute style access to
>> fields) is standard. It's lovely, so far :)
>
> Wow! I missed this - I put off trying out my RELOAD node comparison
> code while actually sitting in front of a Python interpreter, and
> further playing around with nohrio.
>
> I'm going to have to have a proper look later.
>
> I think I made some changes to colouruse.py since you last saw it.
> I'll send a patch for it, so perhaps it's better to hold off on
> porting it to the new API. Which I'd be pleased to do myself,
> actually.
I look forward to it. (I actually finally committed colouruse.py after
fixing the copious endline issues.)
I tried to explain the preferred way of array indexing in oo.rst,
hopefully this will help you simplify colouruse.py.
I'm considering writing a GTK+ app for say.. simultaneous alignment of
hero and weapon handles. Dunno how that will work, but I'm more able
to casually write GUI apps these days.
I suppose a master palette tuner could be another possibility.
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
[Ohrrpgce] A little script to find all your strings
Hello all, I was investigating menus.bas and came up with this script which lists all the unique strings,in order, in BAS/BI files (whichever files you specify). I thought that could have applications for developing, so I'm posting it here. (currently requires grep, but that could easily be changed.) It produces output like this: scrcommands.bi :"noop" scrcommands.bi :"wait" scrcommands.bi :"waitforall" scrcommands.bi :"waitforhero" scrcommands.bi :"waitfornpc" scrcommands.bi :"suspendnpcs" scrcommands.bi :"suspendplayer" scrcommands.bi :"resumenpcs" scrcommands.bi :"resumeplayer" scrcommands.bi :"waitforkey" scrcommands.bi :"walkhero" scrcommands.bi :"showtextbox" scrcommands.bi :"checktag" scrcommands.bi :"settag" scrcommands.bi :"," customsubs.bas :"THE INN COSTS (# gold)" customsubs.bas :"THE INN COSTS" customsubs.bas :"You have (# gold)" customsubs.bas :"You have" customsubs.bas :"CANNOT RUN!" menus.bas :"Pass through walls" menus.bas :"Pass through NPCs" menus.bas :"Enable NPC activation" menus.bas :"Enable door use" menus.bas :"Do not hide leader" menus.bas :"Do not hide party" menus.bas :"Dismount one space ahead" menus.bas :"Pass walls while dismounting" menus.bas :"Disable flying shadow" menus.bas :"default" menus.bas :"A" menus.bas :"B" menus.bas :"A and B" menus.bas :"A or B" menus.bas :"not A" menus.bas :"not B" menus.bas :"neither A nor B" IMO such output could eg. be used to automatically produce descriptions of lists of bitsets for wiki fileformat docs. [attached] #!/usr/bin/env python """ Extract strings from Freebasic BAS or BI files, omitting duplicates (but preserving order.) Intended for application to OHRRPGCE sourcecode. """ from subprocess import Popen, PIPE import sys command = ['grep','-oEe','"[^"]+"'] + sys.argv[1:] def extract (args): command = ['grep','-oTHEe','"[^"]+"'] + args f = Popen (command, stdout = PIPE) lines = f.stdout.readlines() del f for l in list(lines): if lines.count(l) > 1: lines.pop (lines.index (l,lines.index(l)+1)) for l in list(lines): print l[:-1] if __name__ == "__main__": extract (sys.argv[1:])___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
*resurrects thread* {doi se mrilu ko na morsi u'i}
(sorry, I'm a little addicted to speaking in Lojban)
On Wed, Sep 2, 2009 at 6:08 PM, David Gowers <00a...@gmail.com> wrote:
> l = np.memmap('viking.l00', dtype = dtypes['l.linear'], mode = 'r+',
> offset = BLOAD_SIZE, order='F')
This doesn't work, BTW (perhaps it did when you wrote that, but now,
it only 'appears' to work -- numpy.memmap docs explicitly say re:
order "This only has an effect if the shape is greater than 1-D." --
ie it doesn't effect ordering WITHIN dtypes.) The following doctest
demonstrates the problem and a solution.
(yeah, it's slightly Lojban-ish. should still be perfectly understandable.)
>>> datni = ''
>>> f = open ('/tmp/kamplita', "wb")
>>> f.write (datni)
>>> f.close ()
>>> dtype = ([('x','B'),('y','B'),('z','B')])
>>> from numpy import memmap
>>> a = memmap ('/tmp/kamplita',dtype = dtype, mode = 'r')
>>> a
>>> b = memmap ('/tmp/kamplita',dtype = dtype, mode = 'r', order = 'F')
>>> b
>>> a == b
memmap([ True, True, True, True], dtype=bool)
# the above arrays also print identically :| xxx xyy yyz zzz, instead
of xyz xyz xyz xyz
Here is my solution for now:
>>> c = memmap ('/tmp/kamplita',dtype = 'B', mode = 'r', shape = (4,3), order =
>>> 'F')
>>> c
memmap([[120, 121, 122],
[120, 121, 122],
[120, 121, 122],
[120, 121, 122]], dtype=uint8)
^^^ xyz xyz xyz xyz :)
This loses the fields. In the case of .L, this is not a great loss at
all IMO. (we could probably do some automagic that would slow down
reading but allow us to use fields. Whether that's worthwhile is
another question :)
>
> James must be the reincarnation of an old Fortran hacker!
>
> :resolution: FIXED. Added the ability to tweak order via the mmap wrapper.
> Will be further addressed later in the higher level
> wrapping code (automatically
> choosing 'F' order in these situations)
> :resolvedin: eadfb9a
Not fixed. Anyway, that API is obsolete now with the OO RPGDir,
RPGFile, OhrData etc API.
(in which eg `general.passcode` form (attribute style access to
fields) is standard. It's lovely, so far :)
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Script structures/objects (Was: NPC instance script commands)
On Sat, Sep 11, 2010 at 2:11 AM, Mike Caron wrote: > It depends on the situation. It boils down to has-a vs is-a, really. Or, on > other words, if it looks like a duck, quacks like a duck, and has a feathers > property, but was created by CreateChimera(), can we still treat it as a > duck? > > (Lua, Python and Javascript say yes. C/++, FreeBasic and others say no) Lua, Python, and Javascript are all far more like current HamsterSpeak than any of the statically typed languages you mention. IMO, Lua has a good deal in common philosophically with HS, while doing things in a cleaner way; hence I'd put it forward as a good model for what you want to do. >> A whimsical thought: it occurred to me that using : instead of . would >> be more idiomatic for HS. > > Just a minor problem: All 12 hojillion constants that already use : Not to mention that if attribute access is meant to become common, you don't want to place additional obstacles (Shift+; for every attribute access, vs .) in the way of the script writer. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
[Ohrrpgce] FB 0.21 compatibility? (or something else?)
Thanks to TMC and James' help, I managed to get a working FB install
(of 0.21.1).
Everything compiles okay, *except* for base64.[ch] (when I run makereload.sh).
They have these mysterious "restrict" keywords before 'in' and 'out'
parameters, that GCC doesn't like.
(deleting the keywords fixes the compilation issues, and reloadtest
then passes all it's tests ok except for Pedantic Compare W/ XML..
caveat below...)
(GCC 4.5.0 20100610, FYI)
Oh, and this:
/usr/share/freebasic/bin/linux/ld: cannot find utf8toisolat1.o: No
such file or directory
That doesn't surprise me; line 5 of makereload.sh is the kind of
command that I would say 'causes undefined behaviour'. <_>
(you want to compile an .o file into an .o file ?!?! that's like
"Tell me, Is this question true?")
Indeed, removing line 5 and the reference to utf8toisolat1.o in line 8
allows xml2reload to compile okay.
I've attached a patch which fixes those things
(I think James gave me commit permissions when I started the git-svn
mirror, but I'm not 100% sure that these are the correct changes to
do. TMC should be able to apply this patch easily;
it's pretty trivial to apply manually, even.)
urg. the reload stuff has other issues, too -- reloadtest assumes
xml2reload is in the path (rather than the current directory). Now
that I think about it, that could be easily worked around by making a
reloadtest.sh wrapper script that looks like:
export PATH=$PATH:./
./reloadtest
From 023698f1c0ba0a59ba52551f36a5e7adb6f06f20 Mon Sep 17 00:00:00 2001
From: David Gowers <00a...@gmail.com>
Date: Fri, 3 Sep 2010 12:16:52 +0930
Subject: [PATCH] Fix makereload.sh and related files so RELOAD utils all
successfully compile under Linux.
---
base64.c |8
base64.h |8
makereload.sh |5 ++---
3 files changed, 10 insertions(+), 11 deletions(-)
diff --git a/base64.c b/base64.c
index 468c4c2..b4add42 100644
--- a/base64.c
+++ b/base64.c
@@ -64,8 +64,8 @@ to_uchar (char ch)
possible. If OUTLEN is larger than BASE64_LENGTH(INLEN), also zero
terminate the output buffer. */
void
-base64_encode (const char *restrict in, size_t inlen,
- char *restrict out, size_t outlen)
+base64_encode (const char *in, size_t inlen,
+ char *out, size_t outlen)
{
static const char b64str[64] =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
@@ -310,8 +310,8 @@ isbase64 (char ch)
that, when applicable, you must remove any line terminators that is
part of the data stream before calling this function. */
bool
-base64_decode (const char *restrict in, size_t inlen,
- char *restrict out, size_t *outlen)
+base64_decode (const char *in, size_t inlen,
+ char *out, size_t *outlen)
{
size_t outleft = *outlen;
diff --git a/base64.h b/base64.h
index 6bb9a97..0e1f5c5 100644
--- a/base64.h
+++ b/base64.h
@@ -31,13 +31,13 @@
extern bool isbase64 (char ch);
-extern void base64_encode (const char *restrict in, size_t inlen,
- char *restrict out, size_t outlen);
+extern void base64_encode (const char *in, size_t inlen,
+ char *out, size_t outlen);
extern size_t base64_encode_alloc (const char *in, size_t inlen, char **out);
-extern bool base64_decode (const char *restrict in, size_t inlen,
- char *restrict out, size_t *outlen);
+extern bool base64_decode (const char *in, size_t inlen,
+ char *out, size_t *outlen);
extern bool base64_decode_alloc (const char *in, size_t inlen,
char **out, size_t *outlen);
diff --git a/makereload.sh b/makereload.sh
index dd19367..60f5e9a 100755
--- a/makereload.sh
+++ b/makereload.sh
@@ -2,11 +2,10 @@
fbc -v -c -g -lang deprecated reload.bas reloadext.bas lumpfile.bas util.bas || exit 1
gcc -c -g -O3 base64.c || exit 1
-gcc -c -g -O3 utf8toisolat1.o
fbc -v -g -profile -lang deprecated reloadtest.bas reload.o reloadext.o lumpfile.o util.o base64.o
-fbc -v -g -profile -lang deprecated xml2reload.bas reload.o reloadext.o lumpfile.o util.o base64.o utf8toisolat1.o -p . -l xml2
+fbc -v -g -profile -lang deprecated xml2reload.bas reload.o reloadext.o lumpfile.o util.o base64.o -p . -l xml2
fbc -v -g -profile -lang deprecated reload2xml.bas reload.o lumpfile.o util.o base64.o
fbc -v -g -profile -lang deprecated reloadutil.bas reload.o reloadext.o lumpfile.o util.o base64.o
-rm reload.o reloadext.o lumpfile.o util.o base64.o utf8toisolat1.o
+rm reload.o reloadext.o lumpfile.o util.o base64.o
--
1.7.2
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Progress?
On Thu, Sep 2, 2010 at 1:53 AM, Ralph Versteegen wrote:
> On 1 September 2010 15:20, David Gowers <00a...@gmail.com> wrote:
>> On Tue, Aug 31, 2010 at 11:43 PM, Ralph Versteegen
>> wrote:
>>> On 31 August 2010 02:34, David Gowers <00a...@gmail.com> wrote:
>>>> On Mon, Aug 30, 2010 at 9:52 PM, Ralph Versteegen
>>>> wrote:
>>>>> I tried it out, discovered it was broken, and eventually discovered
>>>>> that this was due (only) to errors in the RELOAD documentation. Nice
>>>>> work! Patch attached.
>>>>
>>>> Thanks! Pushed to git master + fixed the doctests that had broken due
>>>> to the stringtable changes.
>>>
>>> Ah, since I don't have nose I forgot about the doctests.
>>
>> Hehe, you don't need nose to do doctesting. but okay.
>
> Well, I did manually copy the lines into a python shell to try them
> out, if that's what you mean.
Ohh. No, doctesting is built into Python.
http://docs.python.org/library/doctest.html
-> see section 25.2.2
"python -m doctest -v doc/reload.rst" would be appropriate in this context.
(it's rather more verbose, but essentially the same)
>
>>
>>
>> import yaml
>> f = open (myfile,'rb')
>> y = yaml.safe_load(f)
>> f.close()
>> from nohrio.reload import reload_from_dict
>> r = reload_from_dict (y, 'root')
>> f = open (myoutfile,'wb')
>> r.write_root(f)
>
> Very nice.
Of course, given your example, I now realize there is no way to do >1
of a same-named node.
I'll fix that as well (reload_from_tuples())
>>>>> It now correctly reads and writes all RELOAD
>>>>> documents that I threw at it, including the unittest.rld that
>>>>> reloadtest produces. Also, I reimplemented reload2xml in a couple
>>>>> lines (MUCH nicer than the 'real' thing :) ),
>>>> lxml2 is pretty nice :) I had it's model in mind vaguely when
>>>> implementing my system.
>>>
>>> I might take a look at lxml2. The official reload interface still
>>> feels unfinished to me.
>>
>> Oh, k.. just had assumed you used that, since it's available in many
>> python installations.
>
> What, used XML? Of course not!
(that = lxml2. I suppose you could have just manually written XML
using sys.stdout.write () etc..)
>
> Actually, to be honest I don't use python for more than 20-liners,
> although I really wish I did. The problem is that all the projects I'm
> involved in are written in nasty languages like FB!
>
> I meant that you compared {'a':0, 'a':1} and {'a':1, 'a':0} as inequal.
oh. I didn't consider repeated nodes.
In fact, my module explicitly fails on those >_< ('a child already
exists with the name "%s"')
Fixed.
>
> But, Mike was quite right that order does matter, I had confused
> myself. However in practice, order doesn't matter for most of of our
> RELOAD-based file formats. For example, the zone file format right now
> contains the list of zones ordered as they appear in a hash table -
> essentially randomly. And all these zones are stored in nodes named
> "zone" (with null values) indistinguishable to a first degree.
>
> So, could you please add the previous elements_equal function back?
> Even better, this function, which further allows identically named
> children to be out of order:
>
> def fuzzy_equal (x, y):
> if x.name != y.name:
> return False
> if x.data != y.data:
> return False
> if len(x.children) != len (y.children):
> return False
> xh = [hash(v) for v in x.children]
> yd = dict((hash(v),v) for v in y.children)
> if set(xh) != set(yd.keys()):
> return False
> return all(fuzzy_equal(c1, yd[c1]))
hm.
I can probably implement this by modifying the hashing function slightly.
So I'll do that and then write some doctests for it.
> I don't know whether those are correct, because python isn't installed
> on this laptop :(. I also wonder whether there is a more efficient
> method that doesn't require temporary sets, and whether it can be made
> to not assume equally hashing nodes are equal.
that's acceptable for now. (only 1/2^(32|64) comparisons of that kind
will give a false positive)
See the current fuzzy_eq implementation :)
David
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Progress?
On Wed, Sep 1, 2010 at 2:28 AM, James Paige wrote: >> I just looked at the archlinux script for that package, and can see >> that someone really screwed it up. If you don't want to move installed >> files around, you could download a standalone build instead, and throw >> it in a folder in ~. > > On my Ubuntu box I always use the standalone package, and I install it > systemwide as root. it has never given me any problems. > > I have never once gotten the non-standalone build to work for me. O_O It works (I just built GAME)! Thanks James and TMC. 'standalone' was the keyword. I'd tried the non-standalone build before, and indeed it just honked aimlessly at me. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Progress?
On Wed, Sep 1, 2010 at 12:50 PM, David Gowers <00a...@gmail.com> wrote: >> efficiently? Or does defining a __eq__ method override any use of >> hashes? > > I should discard the __eq__ method and just define a __hash__ method. > Thanks, TMC :) Oh, btw, __eq__ and __hash__ are for different things, if that is what you meant. If you look at the revision history, you can see I used self.__hash__ inside the __eq__ method. (I've just swapped that implementation out with a more optimized 'perfect' comparison -- see latest commit on nohrio git master. quick_eq can be patched in if the speed difference effects you (probably if your document contains lots of large strings/ binary blobs, or if you have ludicrous numbers of nodes.)) > >> I'm not sure it's needed, since AFAIK the only way that a node would >> actually change type to something textually equivalent is through >> being converted to XML and back. mm.. I think I'll leave this out for now. If it turns out someone actually definitely needs it, I'll implement it. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Progress?
On Tue, Aug 31, 2010 at 11:43 PM, Ralph Versteegen wrote: > On 31 August 2010 02:34, David Gowers <00a...@gmail.com> wrote: >> On Mon, Aug 30, 2010 at 9:52 PM, Ralph Versteegen wrote: >>> I tried it out, discovered it was broken, and eventually discovered >>> that this was due (only) to errors in the RELOAD documentation. Nice >>> work! Patch attached. >> >> Thanks! Pushed to git master + fixed the doctests that had broken due >> to the stringtable changes. > > Ah, since I don't have nose I forgot about the doctests. Hehe, you don't need nose to do doctesting. but okay. > >> (also I had added a reload_from_dict function since then. That's >> available now too FWIW) > > Looks useful. But I'm not sure just what is or is not useful until I > actually sit down and build a real program using nohrio.reload. No > ideas for one yet. Well I was mainly thinking if you wanted to write documents in YAML and convert them to RELOAD, this would make it only slightly harder than falling off a log. (providing you didn't care about node order -- unless you are using Py3, the YAML module doesn't provide ordered mappings.) import yaml f = open (myfile,'rb') y = yaml.safe_load(f) f.close() from nohrio.reload import reload_from_dict r = reload_from_dict (y, 'root') f = open (myoutfile,'wb') r.write_root(f) > >>> It now correctly reads and writes all RELOAD >>> documents that I threw at it, including the unittest.rld that >>> reloadtest produces. Also, I reimplemented reload2xml in a couple >>> lines (MUCH nicer than the 'real' thing :) ), >> lxml2 is pretty nice :) I had it's model in mind vaguely when >> implementing my system. > > I might take a look at lxml2. The official reload interface still > feels unfinished to me. Oh, k.. just had assumed you used that, since it's available in many python installations. > > However, I notice you're still assuming children with the same name > are in the same order. hm? I compared the sets of their names. If those don't differ, then I can safely assume that I can sort only one of the lists of names and use it to reorder both lists of children. (I'm assuming you have the latest changes; everything has been pushed) Their order may or may not matter depending on > the document. It's not very pleasant to fix, but I assume that Python > uses hashes when comparing objects, so it could be done fairly Using hashes sounds like a MUCH better idea. (It has been done. And pushed. The hashing solution is QUITE elegant.) > efficiently? Or does defining a __eq__ method override any use of > hashes? I should discard the __eq__ method and just define a __hash__ method. Thanks, TMC :) (it turned out I do need a __eq__ method but it can be trivial. I also defined a 'perfect' __eq__ just in case.) >>> I suggest comparing the textual >>> representation of data, so that None == "", and maybe an option for >>> exact comparison like reloadutil's --pedantic option. Not quite how I >>> would have designed things, but that's how things worked out: the >>> official implementation's interface doesn't distinguish between null >>> and zero-length string nodes. >> That seems slightly evil. I'll have to consider it. > > I'm not sure it's needed, since AFAIK the only way that a node would > actually change type to something textually equivalent is through > being converted to XML and back. > > I just looked at the archlinux script for that package, and can see > that someone really screwed it up. If you don't want to move installed > files around, you could download a standalone build instead, and throw > it in a folder in ~. > Okay, I'll try that. thanks TMC. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Progress?
On Tue, Aug 31, 2010 at 11:57 PM, Mike wrote: > I can't reply to this properly, since I'm mobile at the moment, but I would > like to make a note about a few things. > > First, the child nodes in a RELOAD document are not sorted in any way. The > nodes will be present in the order they were added, unless they are added in > a different order (AddBefore, etc). This is because node order is important, > and context-sensitive. > > A serialized document will always have the same order as the memory document > that created it, and vice-versa (unless it is modified, of course) > > reloadtest uses these facts to validate that the serialization and > deserialization routines work correctly. > > It will not, however, compare two trees that have the same nodes, but in a > different order (rightly so, since the trees are not the same) That means my original implementation was almost right. Heh, thanks for the clarification Mike. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Progress?
On Mon, Aug 30, 2010 at 9:52 PM, Ralph Versteegen wrote: > I tried it out, discovered it was broken, and eventually discovered > that this was due (only) to errors in the RELOAD documentation. Nice > work! Patch attached. Thanks! Pushed to git master + fixed the doctests that had broken due to the stringtable changes. (also I had added a reload_from_dict function since then. That's available now too FWIW) > It now correctly reads and writes all RELOAD > documents that I threw at it, including the unittest.rld that > reloadtest produces. Also, I reimplemented reload2xml in a couple > lines (MUCH nicer than the 'real' thing :) ), lxml2 is pretty nice :) I had it's model in mind vaguely when implementing my system. > to check that it really > was reading things correctly, and not just that writing was undoing > some kind of error in the read code. > > And the string table optimisation really works, it trimmed 1kB off a 15kB > file! Great - I figured the canonical code didn't do that, and I thought it would be worthwhile for repetitious documents :) > > I noticed that when testing nodes/trees for equality, you don't > actually look at their values. kinda slow, and creates more temporary data (get the names, sort them, copy the list of children, sort according to name order) Anyway, implementation is in my local git checkout; not tested/pushed yet. (probably do that tomorrow.. my clock says 12.04) This means it's no longer assumed that children are in the same order (eg. sorted). This will probably avoid considerably weird breakage in some circumstances. > I suggest comparing the textual > representation of data, so that None == "", and maybe an option for > exact comparison like reloadutil's --pedantic option. Not quite how I > would have designed things, but that's how things worked out: the > official implementation's interface doesn't distinguish between null > and zero-length string nodes. That seems slightly evil. I'll have to consider it. (for the purposes of conditionals in python, '' is equivalent to None; that is, 'not 0', 'not None' "not '' " all evaluate to True (meaning that if you remove the Not, they all evaluate to False.). That may or may not help, though.) > > BTW, why don't the FB packages for your system work? What system is it? Arch Linux. It can't find its include files (crt.bi, for example). It seems to have part of itself installed in /usr and the other part in /usr/local (it claims to be compiled with prefix=/usr/local; the includes are in there IIRC but it doesn't find them) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Progress?
On Mon, Aug 23, 2010 at 9:42 PM, Ralph Versteegen wrote: > internet connection. Luckily due to git this is no hindrance (I'm > completely in love with git now - thanks David!). But I've Heh, that makes two of us. (I even use it for versioning art these days :) BTW, nohrio's RELOAD code actually works now.. 9_9 As I'm sure you discovered, it was hilariously broken in several places (arr.. must write tests before code..!) doc/reload.rst does the testing (and demoing) -- 'nose --with-doctest doc/reload.rst' runs all the current tests. I still can't test it against, say, xml2reload output, because of the absurd brokenness of the freebasic package on this system. But the tests verify that node nesting, testing RELOAD trees for equality, and all node types work. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Zone maps; file format
I'm thinking about this. Currently I favor a scanline-based approach: * zonemaps * lines * * nzones_on_this_line * data : unsigned-short array, (serialized with zlib compression?) ZONE NENTRIES START END [START END.. ] (repeat for each zone which has some part within this line of the map) * zone * id * name simple UI (it's easy to translate a bitarray into a list of scanline spans), possibly faster checking, less memory usage. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] RELOAD Delay Loading
I'd just like to mention that, as this seems to be a variant of lazy loading (http://en.wikipedia.org/wiki/Lazy_loading), it might be more appropriate to use that term - not that it matters much here, but moreso in the code. (IMO Delay loading suggests a *time-based* loading (ie nodes being fully loaded after some time interval), whereas what you describe is deferred partial loading *on demand*.) On Thu, Apr 8, 2010 at 9:33 PM, Mike Caron wrote: > This is my plan for how Delay Loading will work. This is both for my > reference, and so you guys can comment on it. > > I believe this is necessary for a few reasons. In the future, more lumps will > be in the RELOAD format. Heroes, enemies and text boxes are prime candidates, > right off the bat. However, loading and unloading the entire text box tree > every time you show one would take a forever. And, keeping the tree in memory > might not be a good idea either (I actually have no idea how efficient the > in-memory tree is, to be honest). > > So, instead, you would load the tree once, with delay loading enabled (which > it will be by default, but can be overridden). > > RELOAD will open the document, parse the header and root node, and then stop. > The root node will be marked with a flag saying "this node is only partially > loaded", and the child node will be null. Further, RELOAD will cache the file > handle in the Doc UDT, and the file position in the Node UDT. > > When the child node is dereferenced (by GetFirstChild), RELOAD will notice > that 1. It's null, and 2. It's only partially loaded. It will then partially > load the children of the node, and clear the flag on it. Repeat for all the > children, etc. > > The important thing to keep in mind is that whenever one child is loaded, all > its immediate brethren are loaded too. This ensures that the > Next/PrevSibling() commands are always valid. > > It will also be possible to tell GetFirstChild to disable Delay Loading, > where upon it will load the entire branch of the tree. Again, this will be > useful in the situation of loading a text box, for example. Once you find the > text box in question, you know you're going to read all the children, so it > will be more efficient to load it all at once. > > One last thing. Obviously, adding a new child to a partially loaded node will > trigger loading. I think I will also add a command to go through the entire > tree and load everything, so the file can be closed. (We can't close the file > while not using it, because if it happens to change somehow, then any time we > load a partially loaded node, we risk corrupting something!) > > Anything I'm forgetting here? > > -- > Mike Caron > ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: james/3379 Update text files for ypsiliform+2
coi,
On Fri, Feb 12, 2010 at 7:51 PM, Ralph Versteegen wrote:
>> Basically, anything that could have a space in it needs quotes. The only
>> potential edge case might be if it contains a caret (^), which is the shell
>> escape character (and, is not allowed in file names, I believe).
>>
>> --
>> Mike Caron
>
> Doesn't work. The correct way is
>
> system("\"\"path to exe\" flags -o \"path to output file\" \"path to
> input file\"\"")
You'll have to IFDEF that.. on Linux it interprets that as :
run the binary named "/some/path/to/binary flags -o outputpath inputpath"
with no parameters.
Which unsurprisingly does not exist.
Also, on Linux, quotes are legal in file/directory names.
try
$ echo foo > /tmp/\"foo\"
$ mkdir /tmp/\"oof\"
$ ls /tmp/*"*
/tmp/"foo" /tmp/"oof"
quoting's never as easy as it looks. You may be able to get away with
ignoring quotes in filenames though (because after all, 75% of
software handles shell-significant characters like []&*<>\; in
filenames badly. : seems to be handled okay. At least / and (NULL ie
character code #0) are not possible to have in filenames :)
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: teeemcee/3398 Remove 8.3 lump name limit. New limit is: any string of up to 50 charact
On Thu, Feb 11, 2010 at 12:06 AM, wrote: > teeemcee > 2010-02-10 05:36:07 -0800 (Wed, 10 Feb 2010) > 92 > Remove 8.3 lump name limit. New limit is: any string of up to 50 characters > from [a-z0-9._-] , 'case insensitive'. Lump name checking is now implemented in nohrio per this spec; It also checks that in a case insensitive filesystem, there are no collisions (eg FOO vs Foo). Wow, I'm reorganizing everything these days, seems like. nohrio certainly looks like it's due a rationalization of what goes where :) ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] A module format to add
On Thu, Jan 28, 2010 at 9:16 PM, Ralph Versteegen wrote: > On 28 January 2010 19:26, Mike West wrote: >> I don't know if you all know about it but there's a new module format for >> music that is much better than .mod... it's for a tracker called Protrekker, >> you can find it on Google code (there is a replayer routine). It would be >> really nice to be able to use those modules in games. I don't know how hard >> it would be to add support for it I'm not a programmer but if you can that >> would be great. >> > > We depend on libraries to play music and sound effects. music_sdl uses > SDL_Mixer which uses MikMod, and music_native uses Audiere which uses > DUMB. It might be possible to extend Audiere, but SDL_Mixer is > absolutely horrible when it comes to this sort of thing. You'd have to > convince these libraries to add Protrekker support. > > I wasn't able to find the Google Code project, or almost any mention > of Protrekker whatsoever. And there's a camera accessory with the same > name. I'm almost certain that Mike is talking about ProTracker XMs, which should already be supported. I was tempted to ask 'Are you going to ask for more than 20 maps next?' ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
I thought I'd just write a bit about what's happening with nohrio, before I go to the dentist today. There is no direct work on nohrio currently; That's because it's waiting on correct 'pixel packing' code, which I mean to take from my other project, pel.core: http://gitorious.org/pel-core Which is currently being refactored to be more functional (in the computer-programming sense, not in the 'works at all' sense) -- nearly done, indexed conversion support just needs improvement. Nohrio is not going to require pel.core, just use its packing support. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] [Bug 788] New: colour swap tool
On Wed, Nov 18, 2009 at 1:49 PM, wrote: > > http://rpg.hamsterrepublic.com/bugzilla/show_bug.cgi?id=788 > > Summary: colour swap tool > Product: OHRRPGCE > Version: unknown > Platform: PC > OS/Version: Windows XP > Status: NEW > Severity: feature request > Priority: P3 > Component: Graphics > AssignedTo: ohrrpgce@lists.motherhamster.org > ReportedBy: robin.geode...@gmail.com > > > It would be super great to have the ability to search a maptile and say "for > every pixel of colour X, swap it with a pixel of colour Y" > > I have found myself wishing for this almost every time I'm editing maptiles. > In fact, 1/2 of my tile editing time is spent swapping colours. I'm always > asking: "what would this tree look like if it was pink?" > If your tiles aren't using solid blocks of colour, it's not easy to change the > colours without a few minutes of fill-tooling. > > I've tried exporting maptiles to photoshop and tweaking the hue, but it > doesn't > let me do it in 8 bit colour (as far as I can tell). Have you tried: 1. Open the image in Grafx2, make sure background color = 0. 2. Using 'B' to grab a rectangular brush from the tile or area you want 3. Hit TAB to switch to the spare page, F11 to grab brush colors 4. Draw the brush once (D will select freehand drawing tool if needed) 5. Rightclick on the fill button to choose color-replace tool 6. Choose colors, click once to replace all pixels of that color 7. Press 'B' again to grab the brush, TAB to switch back to main page, 'D' to select freehand drawing tool 8. Draw the brush once (over the previous instance of the tile.) If you want to do a color replace on the entire tileset, it's much simpler: 1. Rightclick on the fill button to choose color-replace tool 2. Choose colors, click once to replace all pixels of that color ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] Plan for bigger walkabouts
I just posted a scheme over at http://gilgamesh.hamsterrepublic.com/wiki/ohrrpgce/index.php/Talk:Plan_for_raising_sprite_frame_limits Which might be pertinent ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] XML size bloat experiment (Was: SVN: pkmnfrk/2428 Breaking the ground on my XML idea. for the time being, I'm not hacking)
Claimer: the following is more or less rambling, without necessarily a specific point. On Sat, Oct 17, 2009 at 7:27 PM, Ralph Versteegen wrote: > (ED: I originally wrote the following in November 2008, but never sent > it. Finding it in my draft emails folder, I've dusted it off and am > sending it for amusement.) > > FINE THEN. Look what you made me do! I wrote a utility to export > attack data to xml just to end up something I could test! I took the > dt6 page off the wiki, wrote a crude script to process it, added lots > of hand editing, and wrote a program around it. Near the end I > realised that xml doesn't allow spaces in tag names, so I renamed the > output format to NOTXML. > > I also realised that I could probably have used the code for the > actual flexmenu attack menu code which specifies the data in the lump > in the nearly same way, for a really-good time saving. > > Here's an example attack: > > > 39<\Animation picture> > 52<\Animation palette> The usual solution here is like 52<\AnimationPalette> (CamelCase?) Which seems to work quite well, in conjunction with readable XML indentation. Please note, however, that I am not in favor of XML for data storage; as I've observed, it's a Markup Language, not a Data Language. JSON is a fairly nice fit; YAML is near ideal but I realize that it's much more involved to wrap or implement. > Ally excluding self<\Target Class> > Focused<\Target Setting> > No Damage<\Damage Equation> > Never misses<\Aim Math> > Atk<\Base Attack Stat> > 30<\MP Cost> > Projectile<\Attack Animation> > 1<\Number of hits> > HP<\Target Stat> > Element 1 Damage<\bit> > Cannot target enemy slot 1<\bit> I would expect these to be like element-1-damage cant-target-eslot1 for memorability (no care needed for letter case) > Cannot target enemy slot 2<\bit> > Cannot target enemy slot 3<\bit> > Cannot target hero slot 1<\bit> > Cannot target hero slot 2<\bit> > Cannot target hero slot 3<\bit> > Cannot target hero slot 4<\bit> > Store Target<\bit> > Automaticaly choose target<\bit> > Show attack name<\bit> > FireBurst<\Name> > 30<\Caption Display Time> > Cloak Combustion: Now for judgment...<\Caption> This is another reason why this is not XML: OHR Strings are not ascii. Of course you can escape them (I took this approach with the YAML support in NOHRIO). > Mutable<\bit> > <\attack> > > I could have grouped related items like tag id/condition/check and > damage options, but didn't care to go further. > > > So here's the results for Adventures of Powerstick Man (158 attacks): > DT6 + ATTACK.BIN: 31916 bytes > DT6 + ATTACK.BIN compressed (.zipped using Windows XP which isn't very > good, but that's what most users would probably use): 4254 bytes You may have shot yourself in the foot here somewhat because using zlib you don't have to depend on the user to do anything with subpar utilities -- you can get something pretty close to gzip -9 compression level with no particular effort. With zlib and an XML format, you can do streaming decompression (which can help when searching). With a 'collection of records' paradigm like OHR, it may be smart to compress each record individually (then you'd be able to load it fast.. and with a bit of thought, it even helps you save it fast..) At the moment I think SQLite is a fairly good fit for OHRRPGCE, including for a save format, since it matches the record-based, fairly optimized approach, while being self-describing (and particularly, being relational -- with the amount of cross-referencing of resources that goes on in OHRRPGCE data files, the automatic verification that they actually are pointing somewhere valid is helpful. this is planned for the next significant version of SQLite, though. Had I always the absolute choice of data format/framework, I would always use either HDF5 or SQLite for 'binary' (large volume, predefined fields and no others), preferring HDF5 for its 'attach-arbitrary-metadata-which-can-itself-be-structured' capability; and YAML for 'textual' (structurally varying; large amount of typically-omitted data (ie. fields whose value is implied by their absence)) data. > powerxe_attacks.notxml: 103127 bytes > powerxe_attacks.notxml compressed: 6727 bytes > > Hey, what! That wasn't fair! I just noticed that most of the attack > lumps are actually taken up by 3 string fields: name, description and > caption. Most attacks in most games (unlike powerxe) have 2 of these > blank, and most data fields are left to defaults. > > > Wandering Hamster (151 attacks): > DT6 + ATTACK.BIN: 30,502 bytes > DT6 + ATTACK.BIN compressed: 5,397 bytes > wander_attacks.notxml: 97,922 bytes > wander_attacks.notxml compressed 7,353 bytes > > Gerania (22 attacks): > DT6 + ATTACK.BIN: 4,444 bytes > DT6 + ATTACK.BIN compressed: 1,183 bytes > ohrrpgce_attacks.notxml: 16,068 bytes > ohrrpgce_attacks.notxml compressed 1,638 bytes > > Vikings (213 attacks): > DT6 + ATTACK.BI
Re: [Ohrrpgce] SVN: james/2887 Apply a modification of David's patch to warn of possible corruption whe
On Sun, Sep 27, 2009 at 1:45 AM, Ralph Versteegen wrote:
> 2009/9/25 :
>> james
>> 2009-09-24 09:11:32 -0700 (Thu, 24 Sep 2009)
>> 242
>> Apply a modification of David's patch to warn of possible corruption when
>> binsize shrinks
>> All I changed was to move the check to updaterecordlength() instead of
>> setbinsize()
>> because setbinsize doesn't get called unless the change is positive
>> ---
>> U wip/common.bas
>
>
> I think that Neo's original patch made sense, but this doesn't. This
> is basically equivalent to preventing some arbitrarily chosen majority
> (since some binsize is bound to get incremented fairly regularly) of
> games newer than the version of Game/Custom from being loaded at all.
>
> Binsize was meant to be both forward and backward compatible: if the
> data record contains stuff you don't understand you can still load it,
> and with 90% likelyhood the meaning of the older data will still be
> the same.
I had originally considered a threshold, and perhaps that might be a
better option.
say .. 100 (for game), and the current behaviour (for custom)
What I mean is something like:
Warn if you load a game with larger than expected binsize values (like
the current warning for old RPG files, really)
Warn really obnoxiously ('the behaviour of the game may be impossible
or gibberish') if the difference is more than 100
Refuse to load newer-than-we-understand games in custom (or do a backupy thing)
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: james/2885 updaterecordlength() now screams bloody murder if curbinsize() returns a
On Thu, Sep 24, 2009 at 5:24 PM, David Gowers <00a...@gmail.com> wrote:
> On Thu, Sep 24, 2009 at 9:59 AM, wrote:
>> james
>> 2009-09-23 17:29:42 -0700 (Wed, 23 Sep 2009)
>> 234
>> updaterecordlength() now screams bloody murder if curbinsize() returns an
>> odd number.
>> This would have prevented my data-corrupting bug from this morning, but it
>> is certainly
>> no substitute for me paying attention to what I am doing :P
>> ---
>> U wip/common.bas
>
> Do you think it's reasonable to also complain if the new size is
> smaller than the old?
> It seems to me that old fields are far more likely to be repurposed
> than record size ever shrinking.
Now that I think about it again, they are probably not likely to be
repurposed for backward-compatibility reasons, but still to never
shrink. I still maintain this is a good check to do, since it detects
another kind of mistake, and could also detect binsize.bin corruption
before it causes harm.
patch attached implementing checking in setbinsize (although I can't
compile OHRRPGCE to test it); appliable with
'patch -p1 --- b/common.bas 2009-09-24 18:05:44.385771210 +0930
+++ a/common.bas 2009-09-24 18:05:22.615776457 +0930
@@ -896,6 +896,10 @@ FUNCTION dimbinsize (id) as integer
END FUNCTION
SUB setbinsize (id, size)
+oldsize = getbinsize (id)
+IF oldsize > size THEN
+ fatalerror "Oh noes! getbinsize(" & id & ") =" & oldsize & ", but new value " & size &" is less than that! This means your RPG file's BINSIZE.BIN may be corrupted!"
+END IF
fh = FREEFILE
OPEN workingdir + SLASH + "binsize.bin" FOR BINARY AS #fh
PUT #fh, 1 + id * 2, CAST(short, size)
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] SVN: james/2885 updaterecordlength() now screams bloody murder if curbinsize() returns a
On Thu, Sep 24, 2009 at 9:59 AM, wrote: > james > 2009-09-23 17:29:42 -0700 (Wed, 23 Sep 2009) > 234 > updaterecordlength() now screams bloody murder if curbinsize() returns an odd > number. > This would have prevented my data-corrupting bug from this morning, but it is > certainly > no substitute for me paying attention to what I am doing :P > --- > U wip/common.bas Do you think it's reasonable to also complain if the new size is smaller than the old? It seems to me that old fields are far more likely to be repurposed than record size ever shrinking. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
My internet connectivity is back to normal now A partial implementation of the RPG interface (lump spec matching the previous email) is in git HEAD (locally and on gitorious). YMMV as a) it's partial b) I haven't done any significant amount of testing c) API will definitely change somewhat before it's done. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
I've just begun work on the final of those 7 issues (to TMC: IMO,
unless you are planning on major revision, having nice wrapper classes
won't require any additional changes, so hopefully this does not block
updating your code.)
I'm currently looking at abstracting away the need to know lump names
for normal operations (ie. simple i/o, regardless of how large scale
it is).
For example, here is an example bit of code:
### I'm assuming rpg is an already-created RPGFile or RPGDir instance
>>> gen = rpg.load('GENERAL') # ID is case insensitive -- i just capitalized it
>>> here for clarity.
>>> gen.shape
()
>>> gen.maxmap
22
At this point I need to mention: I realized something about getting a
view of a memmap as a recarray -- it means there is no 'flush' method
available on it.
This means the only way to ensure immediate flushing to disk is by 'del gen'
followed by manually triggering the GC (to collect the now unreferenced memmap).
This does not appear to be an issue in OHRRPGCE's case: the lumps
you'd want in recarray format are either singleton types (GENERAL,
BINSIZE) or scarcely touched (BROWSEINFO,UICOLORS). Feel free to point
out errors here, this is just from a quick inspection.
If the ID is not recognized (and you've specified 'dtypeid' kwarg,
which is normally not needed), we try to open a lump matching it's
name directly, interpreting the dtypeid
Slicing:
Simple 1d slicing is supported in an ID,
as in 'PALETTES16[16]' -- the 16th 16color palette
or 'PALETTES16[:32]' -- the first 32 palettes.
not specifying a slice is the same as specifying a slice '[:]' -- ie all.
Appending:
I'd like to have an API that makes it easy to append records.
It doesn't necessarily have to be built into the load() method (it
could be achieved by
explicit specification of shape there); perhaps a grow (LUMPID,
absolute_nrecords, template = None) method?
Record counting:
I'd also like a method that returns how many records there are for a
particular lumpid. something like a len(lumpid) method.
Maps:
npcdefs etc are associated with a specific map.
I think I should address this within lumpids like this:
11:NPCDEF (that is, precede the actual lump type with a map number
followed by ':')
so the fullest specifiable lumpid might be something like this:
11:NPCDEF[16:24] (records #16..#23 of map 11 NPC definitions).
Comments would be greatly appreciated :)
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
I've just pushed the latest changes to nohrio to gitorious, fixing 6
of the outstanding issues.
Details follow ('missing' number three is 'do more highlevel wrapping
of rpg format', which is a somewhat more long term aim)
TMC, note that ptshapes was not normalized in the way I previously
said, but in a different way! Sorry.
Resolved issues
1.
> Currently nohrio only accepts linearized data (in cases where it was
> planar) and nohrio accepts only spliced attack data.
> This is mainly to do with issues relating to representing a single
> record as a single record (rather than 200 x, 200 y, etc). In future
> it will probably happen transparently
Planar lumps other than tilesets/backdrops should be nearly trivial
thanks to numpy's strides. But how do you hope to transparently
rewrite attacks and til/mxs? With a wrapper object that writes back to
the memmap as it is modified? I'd like to hear about this, because
there are other things you could wrap, such as opening a lumped RPG.
->
> It's actually very easy, but extremely ugly.. well, think about door
> locations..
> does it make any sense to have 200 x coordinates then 200 y coords
> then 200 target door ids?
> the rearrangement of this is so we can serialize it in a way that is
> easy to export and lacks strange constraints (only exactly 200
> records? why?), with each x,y, target triple as a single record.
I think you misunderstood. I meant something like this:
l = mmap('viking.l00', (INT, (5,300)),
BLOAD_SIZE).transpose(0,2,1).ravel().view(dtypes['l.linear'])
This actually works, except it seems to do a copy as you can't use it
to write back to the file. However, there's a much easier way that
works in every way, letting you write to the file!
l = np.memmap('viking.l00', dtype = dtypes['l.linear'], mode = 'r+',
offset = BLOAD_SIZE, order='F')
James must be the reincarnation of an old Fortran hacker!
:resolution: FIXED. Added the ability to tweak order via the mmap wrapper.
Will be further addressed later in the higher level
wrapping code (automatically
choosing 'F' order in these situations)
:resolvedin: eadfb9a
.. note::
Documentation was also added -- planar_dtype was marked as obsolete.
2. Spliceattack bug
-It seems that you have to explicitly import sys in spliceattack under windows.
:resolution: FIXED. Really import sys :)
:resolvedin: a787ea7
4. linear TIL/MXS dtypes not defined:
til.linear and mxs.linear aren't there, though I guess those are only
temporary hacks anyway?
(later clarified -- TMC meant they are not defined dtypes)
:resolution: FIXED.
:resolvedin: d18d6e1
5. TMC gets scalar arrays, not integers (is this a problem? they
calculate fine):
gen.shape = ()
gen = gen.view(np.recarray)
Though as I mentioned, it still only does 98% of what I expect,
because I get scalar arrays instead of integers, and I wonder whether
that will cause problems.
:resolution: NOTABUG. (absent any actual evidence that this causes any
awkwardness when actually
coding to a purpose, rather than experimentally poking things)
.. note::
We should still take care to give the programmer a scalar array
rather than (1,) shaped array
when that is the only reasonable outcome (GEN, BROWSE.TXT, ..)
6. TMC says the following doesn't work:
for fieldname in uicols.dtype.names:
if fieldname == 'textboxframe':
continue
coluse[uicols[fieldname]] += 1
The problem lies in the dtype, which is nested ('textbox')
Solution: write a flatten_dtype() function to flatten
x/y/z/foo,bar,bif into xyzfoo, xyzbar, xyzbif.
:resolution: FIXED. (AFAIK -- I specifically tested it on uicolors)
if you create the memmap using the dtype
``flatten_dtype(o.dtypes['uicolors.bin'])``
you can then happily do the above type of looping.
:resolvedin: 1591655
7. npcdef 'npc' field is redundant:
>>> I noticed that the NPC definitions are each inside an 'npc' field, eg.
>>> mmap[npcno]['npc']['picture']. Why?
>>
>> see above -- nice record serialization.
>
> Don't think I follow: I haven't seen this extra layer anywhere else.
hmm, maybe it's just a brainfart then. I'll look into it.
:resolution: FIXED. (removed needless nesting for n and mn dtypes)
:resolvedin: 75a8bd2
David
___
Ohrrpgce mailing list
ohrrpgce@lists.motherhamster.org
http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
> Your editos the othe wiki on gilgamesh all failed because they were urls > and you were not logged in. Doh! Thanks James, I've fixed it now :) David ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
I've just added two links to introductions to git-svn to the ohrrpgce-svn wiki (http://gitorious.org/ohrrpgce-svn/pages/Home) http://orestis.gr/blog/2008/08/23/git-svn-tutorial/ http://utsl.gen.nz/talks/git-svn/intro.html which give a more comprehensive coverage of git<->svn interoperation ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sat, Aug 29, 2009 at 3:16 AM, Ralph Versteegen wrote: > 2009/8/27 David Gowers <00a...@gmail.com>: > A problem: both of those stories are from before the release of SVN > 1.5, which introduced merge tracking. So I'm not sure that anything > you say is true. Having to manually create and apply patches is > definitely false in most cases. SVN doesn't seem to lose the histories > of changesets that are merged either, see > http://svnbook.red-bean.com/nightly/en/svn.branchmerge.advanced.html#svn.branchmerge.advanced.logblame SVN merge tracking seems pretty effective, actually, from what I read here: http://subversion.tigris.org/merge-tracking/func-spec.html > > Since I've never worked actually with branches in SVN before, I just > tried it out in a test repository, branching, making compatible and > conflicting changes in the trunk and branch, cherry picking revisions > to copy from one to the other out-of-order, and syncing back and forth > between the branch and trunk. I even made a grievous error and > reverted a commit. I clearly got too complicated for svn to handle, > the conflicts and reversion seems to have confused it, as once when > presenting some conflicts to me it compared the wrong two versions. It > also downloads a relatively enormous amount of data from the server to > merge a one-line text file. > > Never did I have to resort to patches or do anything other than > invoking svn and manually cleaning up conflicts. Also, svn merge has > many options and variations that could be tried if the first attempt > doesn't produce the expected result, but I never did this - I probably > should have done when it broke down. > > In the end, there were a couple mistakes in the blames, but the logs > were complete. Over complete in fact, as some revisions were counted > multiple times for which indirect route by which they were merged in. Okay, I apologize: clearly merging is not anywhere near the pain it used to be. The 'squashing into one commit' method is, AFAICS, a method to minimize that pain (ie. quick merge, but all your revision for the branch got squashed together.) > > Anyway, if the OHR was branched in git, how would it be merged back > into the trunk? By suddenly making a thousand commits to to SVN when > it was finished? How else could the revision history be preserved? Or > would it have to be done as a single monolithic commit? Git is really good with commits. There is a standard mechanism for getting a set of patches between two specified revisions, and they include the necessary data that means they are correctly attributed (ie. someone else can commit them, the actual author attribute will remain accurate) You have the option to rewrite history -- for example, squashing two commits together or splitting a commit into several (though you shouldn't do this if you've already published the commits you want to rewrite -- it's rather confusing to do so) If you wanted to mirror git progress back to SVN 'as it was happening', you could do that IIRC http://www.kernel.org/pub/software/scm/git/docs/git-svn.html is a handy reference """ DESIGN PHILOSOPHY Merge tracking in Subversion is lacking and doing branched development with Subversion can be cumbersome as a result. While git svn can track copy history (including branches and tags) for repositories adopting a standard layout, it cannot yet represent merge history that happened inside git back upstream to SVN users. Therefore it is advised that users keep history as linear as possible inside git to ease compatibility with SVN (see the CAVEATS section below). CAVEATS For the sake of simplicity and interoperating with a less-capable system (SVN), it is recommended that all git svn users clone, fetch and dcommit directly from the SVN server, and avoid all git clone/pull/merge/push operations between git repositories and branches. The recommended method of exchanging code between git branches and users is git format-patch and git am, or just 'dcommit'ing to the SVN repository. Running git merge or git pull is NOT recommended on a branch you plan to dcommit from. Subversion does not represent merges in any reasonable or useful fashion; so users using Subversion cannot see any merges you've made. Furthermore, if you merge or pull from a git branch that is a mirror of an SVN branch, dcommit may commit to the wrong branch. git clone does not clone branches under the refs/remotes/ hierarchy or any git svn metadata, or config. So repositories created and managed with using git svn should use rsync for cloning, if cloning is to be done at all. Since dcommit uses rebase internally, any git branches you git push to before dcommit on will require forcing an overwrite of the existing ref on the remote repository. This is generally considered bad practice,
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Thu, Aug 27, 2009 at 8:14 PM, Ralph Versteegen wrote: > 2009/8/27 James Paige : >> I like that. merging both files into attacks.bin and removing the olds >> files seems clean to me. > > But could we name it something other than attacks.bin, as that's too > easy to confuse. Believe me, I thought of that.. But what else are we going to name it? I chose attacks.bin because of its obviousness value. I guess there is atk.bin; I just regard that as more ugly and less obvious. BTW, after reviewing our conversation I found 7 outstanding issues. one of them (spliceattack inexplicably working for me despite never importing sys) has been fixed locally (and will be in next git push) David ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Thu, Aug 27, 2009 at 12:44 AM, James Paige wrote: > On Wed, Aug 26, 2009 at 09:54:15PM +0930, David Gowers wrote: >> On Tue, Aug 25, 2009 at 1:41 AM, James Paige wrote: > I think you are being overly hard on svn. I have never had a problem > with using svn's history in a treelike manner. I know I don't do it much > in the OHR repository, but I do it fairly frequently in my svn > repositories at work. I do agree with your assessment of snv's > branchs as being "heavy", but since I am normally working in an > enviroment unconstricted by either disk space or bandwidth, i ahve never > really been bothered. Well, I've only had limited experience with SVN branching, but I've read in several places that merging is extremely painful* (100% manual -- make a patch for each commit you want to merge, then apply it and commit with an appropriate message, lather rinse and repeat. Alternately, do a little messing around ending with 'svn merge'**, and basically lose the revision history belonging to the branch by squashing it into a single commit). AFAICS eventual merging is the whole point of branching. With git or any other DVCS, Merging is automatic and gives an accurate view of version history (ie. you get all the same commits, with correct timestamps +authors etc., followed by a 'merge XXX branch of FOO' commit, after you perform the merge). * just now for example http://37signals.com/svn/posts/1071-git-smart-how-were-using-git-to-track-our-source-code (although it does seem there are external tools to make svn merging much easier) ** as in http://www.sepcot.com/blog/2007/04/SVN-Merge-Branch-Trunk Perhaps I should mention this: say someone clones from my gitorious ohrrpgce-svn repo -- they could then add their own changes and push that repo to gitorious, where it would be detected as a clone and added to the list of clones, then they could develop some more on it, maybe push the branch they are developing on, or ask me to pull the changes and commit back to SVN... In this way, we can see what multiple people are doing without the need for a single centralized repo. > > Also, one oft-lauded feature of git feels like a liability to me. To me > local branches in your working copy seem like a bad thing. For me, the > second-most-important feature of svn (after versioning) is > centralization. You handle this with a policy about what branches to push to the 'main' repository[*], IMO. Say 'if a branch has been around for more than a week, push it'. One of the main valuable things about local branches is the ability to play around without necessarily coming up to the standards of the centralized repo, so particularly young branches may not be appropriate for pushing. Or you just have all the people on your team pull the branches from others that are relevant to their work. see below where I talk about gitorious and github. [*] there is of course not actually a 'main' repository -- anyone whose repository you can access There are people who use git just like subversion. I think that's excessively restrictive, however it's perfectly possible and not painful at all.. It's just that using the additional features and ridiculous speed of git can improve that workflow significantly. > I don't really like that git is decentralized. I like > the deveopment model where everything you do gets commited (whether it > gets commited to the trunk, or to a branch, or to a branch of a > branch) That is important to me because with svn branches, I can look at > all of your branches, and you can look at all of my branches. If you do > a month worth of work on a git local branch, *I* can't see it. But if I Please note that there is no difference between the 'main' branch (usually called 'master') and any other branches -- if you have read access to my repository you can check out any or all of my branches. > do aa month worth of work on an svn branch, you can check out my branch > and look at it at any time. I think what you would want here is something like gitorious or github where you would push all your changes periodically after committing them locally. see this post, for example: http://drnicwilliams.com/2008/02/03/using-git-within-a-team/ "With Git, you can stop caring about the central SCM. You make your own local changes, you push them to your own remote fork of the repository, and if they get absorbed into the common/default/central bundle so be it." (above article gives quite some good info about common ways of doing team work using git, btw -- although I need to highlight "I do believe that your fork/clone distinction is something that’s specific to github, not applicable across git in general. Within git, every clone is technically a fork, but github calls clones that happen within its own domain “f
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Tue, Aug 25, 2009 at 1:41 AM, James Paige wrote: > On Tue, Aug 25, 2009 at 02:57:09AM +1200, Ralph Versteegen wrote: >> So, load() would usually return a memmap of the whole lump then? How >> would you handle lumps like attack.full and .til; by not allowing you >> to ask for writeable memmaps of these lumps? I suppose that you will >> also have an option to return a single record for the same reason. > > Now that we have unified RPG format updating, and now that the attack > related code is (comparatively) cleaned up, it would probably be quite > easy to merge .dt6 and attack.bin permanently, and do away with the > split file thing. It would just require and RPG version number bump, > which is no big deal. > > Thoughts? > The logical place to put the merged data, to me, is in attack.bin. However I don't know if this is strictly compatible with the way binsize.bin is supposed to work (is it okay for records to grow backwards?). maybe swapping those two for 'attackS.bin' is more appropriate (S capitalized just for clarity) . > I presume that is to prevent the accadental creation of unwanted branches in the SVN? Making SVN branches is something I fortunately cannot do simply by making git branches and pushing. It's simply because mapping git's almost infinitely-flexible and lightweight branch model onto SVN's monolithic, super-heavyweight branch model is fairly painful. Not only do I want to spare myself this pain, but also users of the repository -- if there is just one branch, workflow is simple and obvious, they don't have to deal with the pain of SVN branching, and in particular the conflict between SVN's linear model of revision history and git's tree model. The gitorious repos is intended to illustrate the benefits of transitioning to git by providing a) the ability to interact with SVN servers when you must, and the ability to use git to do the actual development in a more playful and sandboxable way. So introducing git branches (how do these map into a linear revision history such as SVN? they don't, basically. SVN sez "there can only be ONE!" :)) into this sandbox really gets in the way of portability between VCSs (if OHRRPGCE development does end up transitioning to git, *then* IMO would be the time at which full git branching on the master repository might come to be useful. ) Making SVN branches is something I fortunately cannot do simply by making git branches and pushing <- I was going to delete that duplicate, but I've decided to leave it here to emphasize how VERY MUCH OF A GOOD THING that is. David ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Mon, Aug 24, 2009 at 12:16 AM, Ralph Versteegen wrote:
> Yes, that's quite sensible.
> But I'm not sure that you're still talking about
> textserialize_dtype_tweaks. Anyway, I haven't looked at serialisation
> at all, so don't know how it happens. I'll get around to it.
oh, that. It doesn't do anything right now. I'm not sure if it's needed.
> Well, it's not a true memmap because I can't write to the file with
> it,
sorry, that's a gotcha: mmap is written to open with 'r' mode, not 'r+'
def mmap (fname, dtype, offset = 0, shape = None):
return np.memmap (fname, dtype = dtype, mode = 'r',
offset = offset, shape = shape)
http://gitorious.org/nohrio/nohrio/blobs/master/nohrio/ohrrpgce.py
>> of course writeable memmaps.
>> It's just that you could get a writeable memmap, or a recarray,
>> depending on your requirements.
>
> What are/how will these load and save 'methods' work? Are we talking
> about methods on classes/objects for each lump? Or maybe on an
> RPG-file class?
rpgfile class method sounds more like what I was thinking. Load() is
what you'd mainly use -- but you might want to use save() if you say,
had some records that you wanted to write to a segment of the lump
without bothering with opening a memmap for the purpose (mainly this
could happen when you serialize to yaml and later load in that yaml
and convert it to an array.)
>> They behave better than you might be thinking, though:
> game.lump('.gen', shape=())['maxmap'] + 10
>> 58
>
> Something odd is going on here:
game.lump('.gen', shape=())['maxmap'] + 10
> memmap(58)
oh, whoops. Maybe there is a difference in behaviour between IPython
and normal Python console.
Anyway, repr(game.lump('.gen', shape=())['maxmap'] + 10)
produces the value you are expecting for me. str(..) produces '58'
IME you can indeed treat such memmaps as if they were integers except
in special circumstances (they don't like being pickled or serialized
to YAML -- basically anything that really truly wants to ask them 'are
you an integer?')
I think the only case you might run into is like foo += 10 (which of
course writes to the memmap). OTOH, I think it's actually pretty hard
to do that unintentionally.
Oh btw, I had to fix RPGdir : it assumes that the lump name is case-insensitive
(when I run it on the test data in Linux, it complains 'no such file
../tests/data/Viking.gen'). Imo the correct choice here is to
lowercase what we get from archinym (I'm considering modifying
archiNym to ensure that output is always lowercase and input always
gets converted to lowercase; but if you implement the change in
RPGdir, it will still work after I fix archiNym)
>
> game.lump('.gen', shape=())['maxmap'] == 48
>> True
>
game.lump('.gen', shape=())['maxmap'] == 48
> memmap(True, dtype=bool)
>
> "%d" % game.lump('.gen', shape=())['maxmap']
>> '48'
>>
>> the hash of X is X for any integer X, btw. If you treat the integer as
>> just a series of bytes, then hash generates something more like what
>> you might want -- in which case,
>
> Wow, are these the same hashes used internally in dicts? That can be a
> bad idea for hashmaps!
Technically it can, but I think you will find practically, there is
only a tiny chance of collision (because the set of integers that are
likely to be used are actually fairly small (reflected in Python's
choice to preallocate integers -1..100 and allocate at runtime all
others as needed). Python's hasher (and I think all decent hashers)
tend to produce about equal entropy for all bits, versus typical
integers (which have 0 entropy for higher bits)
Anyway any decent implementation of hashmaps includes collision
handling, so if this happens, it is only a minor slowdown, not
disasterous.
>> true true; I want to say that your discovery of that way to get planar
>> stuff working nicely is AWESOME and SOOO helpful!
>>
>> btw, numpy also provides a facility that automatically converts
>> between fortran and C layout. it might be a method on arrays, not sure
>> currently. that should kill linearize dead, heh.
I looked this up but have forgotten. I'll check when I reply to the remainder
> I'll look into it and see if I can improve unpack_4bpp_array further:
> at the moment it's actually returning the data with the last two axes
> transposed in memory, which isn't ideal. I couldn't manage to write
> unpackpt to write the data straight into that layout without doing a
> copy at the end, but Fortran layout might be the trick needed.
I've currently locally got unpack_4bpp_array basically as you wrote
it, but with a parameter (defaulting to True) controlling whether the
array is transposed before returning it.
After I've checked over this thread and tried to address any issues I
missed in nohrio re:that, I'll reply to the rest of your email
AFAIK fortran reverses striding. That is, in a C array [frames][y][x],
the first N bytes will be variations of x, then y, then frames,
whereas Fortran arrays, in a [frames][y][x]
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sun, Aug 23, 2009 at 8:43 AM, David Gowers<00a...@gmail.com> wrote: > On Sun, Aug 23, 2009 at 2:17 AM, Ralph Versteegen wrote: >> Well, I fixed unpack_4bpp_array (it was also much less efficient than >> unpackpt). No kidding! I just opened it up and realized I was doing this totally redundant messing around with array shape. doh. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sun, Aug 23, 2009 at 10:59 AM, Denny Abraham wrote: > Hi > > Is this git mirror going to remain current with the SVN head? Certainly. When I get my internet connection working reliably, I'll set it up to 'git svn rebase; git push' once a day. It is currently current with SVN head > Also, > will this be a canonical repository, ie can we submit patches/ pull > requests (this may be github specific terminology) directly to the > repo? I would have to discuss this with James (among other things it would require me to have commit authorization for SVN). Currently this seems feasible - with the caveat that I insist on the main repo having only one branch at a time. (clones, obviously, can have as many as they want) Without any special arrangements required, it is already possible to build patch sets for SVN and commit them (assuming you have appropriate authorization), using the git svn layer. David ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sun, Aug 23, 2009 at 2:17 AM, Ralph Versteegen wrote:
> 2009/8/22 David Gowers <00a...@gmail.com>:
>> just pushed. 'git shortlog' says:
>>
>> ohrrpgce: make palette/picture pairs more consistent in naming
>> (req. by TMC)
>> ohrrpgce: fixBits, archiNym: support inline RPG lump I/O;
>> fixbits: support slicing
>> README: add mission statement, improve and augment examples,
>> expand credits
>> ohrrpgce: reorder ptshapes to match on-disk format, doh. and document it
>
> Opps, still wrong: the on-disk order is [frames][width][height]. I'm a
> bit confused about what you want dtypes['pt?'] to be.
oh damn. I dislike developing with spotty internet access.
So the output does need transposition before it can be in a conventional format.
Will be really fixed in next git push.
> [frames][width][height/2], or [pixels/2]? I assumed the former, and
> that ptshapes should give the unpack dimensions.
I intended [frames][width][height/2],
but [frames][width][height] is probably wiser.
Oh! BTW, I understand why you believe pygame uses [x][y] memory format
-- it doesn't, it only (confusingly) requires that you index it in
that order. SDL Surface is the underlying object, which is indeed
[y][x] order just like everything else.
"The PixelArray wraps up a Surface and provides a direct 2D array
access to its pixels using the surface its rows as first and its
columns as second axis."
I have no idea why they have done things like this -- aargh. I mean it
might match the other places where [x, y] is used (eg Rects) but it
blatantly conceals the memory format and sacrifices sanity for
consistency.
I verified this using the following:
s = pygame.surface.Surface ((4,4), depth = 8)
for i in range (4):
s.set_at((i, 0), i)
b = s.getbuffer()
arr = numpy.frombuffer (b, dtype = 'B')
print arr
# the first 4 items printed are 0,1,2,3, therefore the memory format is y,x
# if it were x,y, the items printed would be 0, 0, 0, 0, 1, 0, 0, 0,
2, 0, 0, 0, 3, 0, 0, 0
An easier way would have been to create a buffer with different width/height and
check which the pitch matches.
Not that this knowledge seems to help yet -- I still seem obligated to
provide disordered [x][y] data when blit_array-ing.
>
> def unpack_4bpp_array (array):
> """convert a 4bpp array [.., h, w/2] -> [.., h, w] ..."""
>
> So this should be [.., w, h/2] -> [.., h, w]
>
> Also, you forgot to reverse the .pt4 shape, (20, 20, 8)
>
>> nohrio.egg-info/: removed, it bit James and clearly is not fit
>> for civilized VCSs
>>
>> Notably, the new example in README shows how to read inline from an
>> RPG file (not dir)
>
> Neat, that's pretty simple!
>
> "
> You should also note that we don't currently deal directly with dt0
> and attack.bin, since they are in
> a quite bizarre 'needs-stapling-together' format. So you cannot
> entirely avoid unlumping if you want
> to read attack data -- we need to do the joining, then memmap the
> resulting file.
> "
> Unlumping just to splice together dt6 (not dt0) and attack.bin would
> be way too much! It would be easy to do it inline. And I don't see why
> memmapping something is crucial. I guess retrieving (splicing) data
> only as it's requested won't work? You could perhaps live with SoJ's
> 436kB of attack data in memory.
.. Ok, I clearly need to clarify.
I meant just reading the file headers and unlumping just those two
files. You could reasonably splice them as you are doing so (and then
of course you can memmap the file). I'm assuming you aren't going to
be adding records, since you're reading inline; specifying shape
explicitly will prevent bugs from writing past the end of the file
>
>>
>> fixes needed for colouruse.py:
>>
>> * either use the unpack_4bpp_array function in nohrio.gfx, or adjust
>> (remove) the transposition in unpackpt to match change of ptshapes.
>
> Well, I fixed unpack_4bpp_array (it was also much less efficient than
> unpackpt).
> First, I had to fix ptshapes, and make an assumption about what was intended:
>
> ptshapes = ((8, 40, 32), (1, 34, 34), (1, 50, 50), (1, 80, 80), (8, 20, 20),
> (2, 24, 24), (3, 50, 50), (16, 16, 16), (1, 50, 50))
>
> Then I updated 'pt?' in dtypes, again, had to decide what was intended:
>
> for i, data in enumerate (ptshapes):
> frames, h, w = data
> dtypes['pt%d' % i] = [('pixels', (np.uint8, (frames, w, h/2)))]
yup
>
> Then:
>
> def unpack_4bpp_array(array):
> """convert a 4bpp array [.., w, h/2] -> [.., h, w]"""
> newshape = shape_bpp_adjust
Re: [Ohrrpgce] svn <-> git bridge, nohrio
just pushed. 'git shortlog' says: ohrrpgce: make palette/picture pairs more consistent in naming (req. by TMC) ohrrpgce: fixBits, archiNym: support inline RPG lump I/O; fixbits: support slicing README: add mission statement, improve and augment examples, expand credits ohrrpgce: reorder ptshapes to match on-disk format, doh. and document it nohrio.egg-info/: removed, it bit James and clearly is not fit for civilized VCSs Notably, the new example in README shows how to read inline from an RPG file (not dir) fixes needed for colouruse.py: * either use the unpack_4bpp_array function in nohrio.gfx, or adjust (remove) the transposition in unpackpt to match change of ptshapes. * match the naming changes oh, did I mention, nohrio.ohrrpgce already imports * from ohrstring itself, so there's not a need to import it manually most often. BTW, the gitorious interface allows you to comment on individual commits. I just noticed this (eg: http://gitorious.org/ohrrpgce-svn/ohrrpgce-svn/commit/cd8b27b153e890e83956d59fa030826a7c142a28 -- you may only get the option of commenting if logged into gitorious) and thought that might be useful. David ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sat, Aug 22, 2009 at 5:32 PM, David Gowers<00a...@gmail.com> wrote: >> Which is for example unhashable. > Disagree here -- I cannot generate any memmap which is unhashable. > In fact, I was being annoyed by numpy.ndarray being unhashable. > So I did 'hash(arr.view(np.memmap))' and hashed it fine! > crazy. actually, for any array of any kind, hash(array.data) is always valid. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sat, Aug 22, 2009 at 5:33 PM, David Gowers<00a...@gmail.com> wrote: > If not, for now, running 'git reset --hard master' before 'git pull > --rebase' should work around this effectively. aargh! (I had plenty of time to correct that too) s/master/HEAD/g, you savvy? David ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sat, Aug 22, 2009 at 3:55 AM, James Paige wrote: > nohrio.egg-info/PKG-INFO: needs update > nohrio.egg-info/SOURCES.txt: needs update > nohrio.egg-info/top_level.txt: needs update > refusing to pull with rebase: your working tree is not up-to-date > ahahah. well, that says to me quite clearly that those files should not be in git, as they are automatically being updated by setup.py and causing conflicts. Removed. Next git push will remove that directory from versioning, which should fix your problem. That might even happen before you read this. If not, for now, running 'git reset --hard master' before 'git pull --rebase' should work around this effectively. > I don't remember changing any of those files, and I don't know how to > update them. git pull without the --rebase says: > > Updating 9d6cfa6..1463af0 > error: Entry 'nohrio.egg-info/PKG-INFO' not uptodate. Cannot merge. > Sorry about that James. David. ___ Ohrrpgce mailing list ohrrpgce@lists.motherhamster.org http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
Re: [Ohrrpgce] svn <-> git bridge, nohrio
On Sat, Aug 22, 2009 at 1:56 AM, Ralph Versteegen wrote:
>> IIRC I'm planning to use that mainly to get graphics to serialize in a
>> neat, human-editable format rather than as a lump (haven't checked how
>> they currently serialize; I would guess it is as a single escaped
>> string, which probably looks pretty ugly / irregular) .
>
> Oh, I think I see, assuming you mean reading/writing data fields such
> as bitsets instead of graphics.
er, no. bitsets are currently fairly reasonable in YAML serialization
format (I would like to eventually have them made up of a list of eg
ABSORBS4 MP_IDIOT BOSS flags rather than numbers, as detailed in
milestone1.yaml). To me, this format (with autocompletion) is the
quickest way to flag things when the number of possible flags are
large.
I'm talking about nicely formatted images:
1000
1100
1110
(4x4 image in ideal serialization format)
Anyway, as I said later, that class is probably going to be removed
because of my better understanding of YAML serialization -- almost
certainly not needed.
>
>>>
Currently nohrio only accepts linearized data (in cases where it was
planar) and nohrio accepts only spliced attack data.
This is mainly to do with issues relating to representing a single
record as a single record (rather than 200 x, 200 y, etc). In future
it will probably happen transparently
>>>
>>> Planar lumps other than tilesets/backdrops should be nearly trivial
>>> thanks to numpy's strides.
>> It's actually very easy, but extremely ugly.. well, think about door
>> locations..
>> does it make any sense to have 200 x coordinates then 200 y coords
>> then 200 target door ids?
>> the rearrangement of this is so we can serialize it in a way that is
>> easy to export and lacks strange constraints (only exactly 200
>> records? why?), with each x,y, target triple as a single record.
>
> I think you misunderstood. I meant something like this:
>
> l = mmap('viking.l00', (INT, (5,300)),
> BLOAD_SIZE).transpose(0,2,1).ravel().view(dtypes['l.linear'])
What you have now is not a memmap, is it? it's an ordinary array, right?
(printing a memmap shows 'memmap (' as the first few characters,
whereas what you have after doing the above is almost certainly an
array (because transpose must make a copy rather than reconfiguring
the array strides.))
That's fine, I was focusing on memmaps rather strongly for their nice
easy I/O, but I'm happy to accommodate this usage too.
>
> This actually works, except it seems to do a copy as you can't use it
> to write back to the file. However, there's a much easier way that
> works in every way, letting you write to the file!
>
> l = np.memmap('viking.l00', dtype = dtypes['l.linear'], mode = 'r+',
> offset = BLOAD_SIZE, order='F')
Oh hah! Nice one! That's perfectly true, I never thought about using
fortran style striding :)
EXCELLENT!
>
> James must be the reincarnation of an old Fortran hacker!
>
>> For example, it's almost trivial to export tilemaps as grayscale PGM
>> graphics once you have them linearized. I have a really strong focus
>> on text serialization here, because once a thing is in a consistent
>> textual format, it can be parsed easily (even dumbly), allowing you
>> to, eg, find all attacks with "slash" in the name, find all 16-color
>> palettes which refer to color index #128, using simple tool like
>> 'grep'
>> (which lowers the learning requirement for data debugging)
>
> Which is going to be pretty neat, I'll have a look at the YAML some other
> time.
>
>>> But how do you hope to transparently
>>> rewrite attacks and til/mxs? With a wrapper object that writes back to
>>> the memmap as it is modified? I'd like to hear about this, because
>>> there are other things you could wrap, such as opening a lumped RPG.
>> Sorry, you just blew my mind (although, what I was planning was sort
>> of similar) . I'll get back to you on this subject.
>>
>>>
>>> In particular, how would Gfx4bpp work?
>> That's yet another 'scratch' class. I think I was intending it to
>> allow you to access 4bpp pixels as if they were 8bpp (ie. not 2 pixels
>> packed into one byte :)
>> No guarantee that will stay around (although some 16color graphics
>> nicety will be around)
>>
>>>
>>> Also, I'd like to embed python in that wxwidgets wrapper around Custom
>>> that I am writing, so that plugins or portions of editor could be
>>> (re)written in it, in which it makes sense to use the nohrio api to
>>> access game data. There might be differing requirements there.
>>
>> Yes, you would definitely want planar access there, I see that.
>> Hmm, I must meditate on this.. :)
>
> Planar access? Actually, I was thinking along the lines of possibly
> being handed a buffer containing a single record of some type, perhaps
> with INTs replaced with LONGs, instead of memmapping a lump.
I'm a little unclear here: when you say buffer you probably mean
(conventional, non-memmapped array); do you want automatic expansion
of 16-bit ints into 32-
