On Fri, Jan 14, 2005 at 20:45 +0100, Christian Tismer wrote:
> Samuele Pedroni wrote:
> >After discussing with Armin and Holger, I have created a new branch to
> >explore typed unwrapping.
>
> Oh, did I miss an IRC discussion?
i guess so. We are actually discussing things on #pypy quite
a bit these days. It would be great if you keep an eye on it :-)
> Does somebody happen to have a log file of the session?
> I'm just curious to follow the reasoning, and to see
> if I can help something.
Sure, below you'll find the chat session. I just cut the log at the
start and the end but not in between. Also there was an earlier
(shorter) discussion which i didn't recover. But almost all
of the arguments are recapped in the discussion below ...
cheers,
holger
Jan 14 11:05:32 <pedronis> should we also really rename is_true to true_w
Jan 14 11:05:43 <hpk> IMO: yes
Jan 14 11:05:55 <hpk> we may do that in a branch as well
Jan 14 11:06:05 <hpk> and warn on pypy-dev
Jan 14 11:06:10 <arigo> good points
Jan 14 11:06:12 <pedronis> I can create a branch
Jan 14 11:06:18 <hpk> i don't know e.g. if Christian's translator breaks
etc.pp.
Jan 14 11:06:29 <hpk> although global replace works fairly well i guess
Jan 14 11:06:52 <hpk> (but christian's translator is special because it
targets interp-level python)
Jan 14 11:07:14 <pedronis> well, it also depends what we do with the
generic unwrap
Jan 14 11:07:24 <pedronis> if we remove it, rename it
Jan 14 11:07:54 <pedronis> I think some other translator stuff may break
and need changes
Jan 14 11:08:02 * hpk is off for an hour to do tax declaration work
(argl!)
Jan 14 11:08:48 <arigo> pedronis: I rediscovered unwrap_builtin()
Jan 14 11:08:56 <pedronis> I saw that too
Jan 14 11:09:14 <arigo> I'd believe that we only need that and the int_w&co
Jan 14 11:09:32 <pedronis> probably yes
Jan 14 11:09:50 <pedronis> I don't think we want to keep lists and tuple
unwrappable
Jan 14 11:09:59 <arigo> no it doesn't really make sense
Jan 14 11:09:59 <pedronis> I mean directly unwrappable
Jan 14 11:10:17 <pedronis> because of the mixed type cases
Jan 14 11:10:19 <arigo> it's only used in some tests
Jan 14 11:10:36 <pedronis> I think code objects use them
Jan 14 11:10:43 <pedronis> to unwrap tuples of strings
Jan 14 11:10:50 <pedronis> but it can be done differently
Jan 14 11:10:55 <arigo> yes
Jan 14 11:11:30 <pedronis> should str_w, int_w be multimethods in the std
objspace
Jan 14 11:11:39 <pedronis> if we consider multiple impl of types
Jan 14 11:11:44 <pedronis> maybe yes
Jan 14 11:13:07 <arigo> is_true() is no longer a MM
Jan 14 11:13:45 <pedronis> yes, it's a bit of special case
Jan 14 11:13:56 <arigo> right
Jan 14 11:14:03 <pedronis> because it corresponds also to __nonzero__
Jan 14 11:14:11 <arigo> well we have __int__, __str__ etc
Jan 14 11:14:12 <pedronis> so it goes through descrops
Jan 14 11:14:30 <pedronis> yes but I don't think we want int_w to
correspond to __int__
Jan 14 11:14:38 <pedronis> or maybe yes
Jan 14 11:14:41 <arigo> I thought so
Jan 14 11:15:21 --> lypanov ([EMAIL PROTECTED]) has joined #pypy
Jan 14 11:15:47 <pedronis> well but for str_w in often does not make
sense, some function takes a string not everything that defines __str__
Jan 14 11:16:00 <pedronis> s/takes/takes just
Jan 14 11:16:13 <arigo> hum, right
Jan 14 11:16:35 <pedronis> we already have space.int
Jan 14 11:16:37 <arigo> should it be readbuf_w()? (argh)
Jan 14 11:16:37 <pedronis> btw
Jan 14 11:17:12 <pedronis> that means that in parse_w we could have
specifier both behaviors
Jan 14 11:17:25 <pedronis> that menas something that implements the buffer
intf
Jan 14 11:17:32 <pedronis> s/menas/means
Jan 14 11:18:16 <arigo> raah, trying to figure out what CPython really does is
messy
Jan 14 11:18:29 <arigo> suppose x.__class__ has an __int__
Jan 14 11:18:36 <arigo> then you can do range(x) but not float(x)
Jan 14 11:18:55 <pedronis> I think it is easier to have both kind of
primitive op
Jan 14 11:18:58 <pedronis> ops
Jan 14 11:19:17 <pedronis> both unwrap only if of the "right" type or
invoke the __xxx__ method
Jan 14 11:19:28 <pedronis> and then be clever at parse_w level
Jan 14 11:19:31 <arigo> (you can't do 1+x either, btw)
Jan 14 11:20:05 <pedronis> yup
Jan 14 11:20:18 <arigo> ouack
Jan 14 11:20:21 <pedronis> so we need int_w which is different from int
Jan 14 11:20:29 <arigo> "abcdef"[1:x] -> TypeError
Jan 14 11:20:33 <arigo> "abcdef".__getslice__(1,x)
Jan 14 11:20:34 <arigo> 'bcdef'
Jan 14 11:21:17 <pedronis> I think right now it is more worth to concetrate
Jan 14 11:21:31 <pedronis> moving away from the generic unwrap to typed
versions thereof
Jan 14 11:21:55 <pedronis> sorting the mess what CPython accepts here and
there
Jan 14 11:22:03 <pedronis> is a slightly different issue
Jan 14 11:22:14 <arigo> so we define int_w and str_w as only accepting the real
type?
Jan 14 11:22:23 <pedronis> yes
Jan 14 11:22:37 <pedronis> you have space.int and space.str
Jan 14 11:22:43 <pedronis> for the other behavior
Jan 14 11:23:18 <pedronis> that's why I'm not sure that space.is_true
should become true_w
Jan 14 11:23:24 <arigo> I see
Jan 14 11:23:26 <pedronis> true_w would just work with boolean
Jan 14 11:23:53 <pedronis> although I think there's not much in CPython
that behaves that way
Jan 14 11:24:04 <pedronis> with bool args
Jan 14 11:24:53 <arigo> it starts to look messy again, with a lot of xxx_w()
MMs for each way we'd like to unwrap objects
Jan 14 11:25:04 <arigo> readbuf_w(), writebuf_w(), ...?
Jan 14 11:25:36 <pedronis> well, the othe option is to pass a 2nd type arg
to unwrap
Jan 14 11:26:41 <pedronis> but it gets tricky to express that as true
RPython
Jan 14 11:26:53 --> Eventh ([EMAIL PROTECTED]) has joined #pypy
Jan 14 11:27:05 <arigo> float_w should accept an int, I guess
Jan 14 11:28:04 <pedronis> well, if they are multimethods at least in the
std objspace
Jan 14 11:28:10 <pedronis> it easy to tweak that
Jan 14 11:28:49 <arigo> I'm more trying to understand what we really want...
Jan 14 11:30:06 <pedronis> right now a big parts of our unwraps
Jan 14 11:30:12 <pedronis> are about ints and strings
Jan 14 11:31:01 <pedronis> so at least int_w and str_w seem to make sense
Jan 14 11:31:35 <arigo> ok
Jan 14 11:31:54 <pedronis> parse_w will also make sense
Jan 14 11:32:10 <pedronis> so I think creating a branch
Jan 14 11:32:26 <pedronis> and introducing int_w and str_w as first step
Jan 14 11:32:46 <arigo> looks good
Jan 14 11:32:58 <pedronis> then I think it is easier to see what we need
beyond that
Jan 14 11:33:04 <pedronis> from parse_w point of view
Jan 14 11:33:12 <arigo> ok.
Jan 14 11:36:38 <pedronis> I will create a src-typedunwrap branch
Jan 14 11:36:43 <pedronis> and mail pypy-dev
Jan 14 11:36:57 <pedronis> now I'm going to have lunch
Jan 14 11:37:03 --- pedronis is now known as pedronis_lunch
Jan 14 11:55:42 <-- arigo has quit ("Leaving")
Jan 14 12:15:09 <-- thingie24 has quit (leguin.freenode.net
irc.freenode.net)
Jan 14 12:15:09 <-- dlk has quit (leguin.freenode.net irc.freenode.net)
Jan 14 12:15:09 <-- etrepum has quit (leguin.freenode.net irc.freenode.net)
Jan 14 12:20:20 --> thingie24 ([EMAIL PROTECTED]) has joined #pypy
Jan 14 12:20:20 --> dlk ([EMAIL PROTECTED]) has joined #pypy
Jan 14 12:20:20 --> etrepum ([EMAIL PROTECTED]) has joined #pypy
Jan 14 12:22:56 --> arigo ([EMAIL PROTECTED]) has joined #pypy
Jan 14 12:26:30 --- pedronis_lunch is now known as pedronis
Jan 14 12:47:44 <-- adim ([EMAIL PROTECTED]) has left #pypy
Jan 14 12:57:18 <mwh> afternoon
Jan 14 12:59:42 <pedronis> hi
Jan 14 12:59:57 <arigo> afternoon
Jan 14 13:08:07 <hpk> afternoon
Jan 14 13:22:27 <lac-sleeps> afternoon
Jan 14 13:22:32 --- lac-sleeps is now known as lac
Jan 14 13:27:54 <mwh> nothing else to say, i'm afraid :)
Jan 14 13:31:46 <lac> :-)
Jan 14 14:05:57 <mwh> arigo: i hope people read and take notice of your
python-dev interface post
Jan 14 14:07:54 <arigo> i hesitated
Jan 14 14:08:08 <arigo> it's a bit lost in the thread but i don't want to
insist more, specially to avoid flooding python-dev again
Jan 14 14:10:38 <mwh> well, i think the idea that strings represent >1
concept is illuminating
Jan 14 14:14:32 --> adim ([EMAIL PROTECTED]) has joined #pypy
Jan 14 14:59:59 <lac> I am going to work now. With a bottle of champaigne.
Jan 14 15:00:04 <mwh> hehe
Jan 14 15:00:05 --- lac is now known as lac-work
Jan 14 15:12:19 <hpk> arigo, mwh: indeed i find "global adapter registries"
scary
Jan 14 15:14:35 <hpk> another serious concern which i haven't seen mentioned
is debugging, btw
Jan 14 15:15:16 <hpk> for example, how will a traceback involving adapations
on arguments look like?
Jan 14 15:21:29 <arigo> if there is an exception while adapting?
Jan 14 15:22:17 <arigo> you'll probably see the "def f(x: y)" line in the
traceback, which is a good hint that it's during adaptation
Jan 14 15:34:57 <-- idnar has quit ("kbye")
Jan 14 16:07:41 <hpk> i am pretty sure that it will make things harder to
debug, anyway
Jan 14 16:15:37 <-- dlk has quit (Read error: 104 (Connection reset by
peer))
Jan 14 16:23:31 <arigo> the global registry is dangerous
Jan 14 16:23:49 <arigo> __conform__() and __adapt__() should be reasonable,
though
Jan 14 16:24:21 <hpk> as methods on objects? (i didn't follow python-dev
closely on all this)
Jan 14 16:24:55 <arigo> I think the idea is that the built-in adapt() function
just calls __conform__() and __adapt__()
Jan 14 16:25:13 <arigo> a bit like "a+b" calls __add__() and __radd__(), btw
Jan 14 16:25:57 <pedronis> arigo: unrelated, I'm thinking what to do about
co_consts
Jan 14 16:26:33 <pedronis> I rediscovered the fact that our code objects
are supposed to be space indipendent
Jan 14 16:26:48 <pedronis> but that means they must store co_consts
unwrapped
Jan 14 16:27:00 <arigo> yes
Jan 14 16:27:31 <pedronis> but that exactly a mixed type tuple
Jan 14 16:27:49 <arigo> oups
Jan 14 16:28:11 <pedronis> so we would like co_consts_w
Jan 14 16:28:34 <pedronis> but the the code objects are really designed to
be space indipendent
Jan 14 16:29:15 <hpk> although i am not sure this design decision is
neccessary
Jan 14 16:30:15 <hpk> what does it buy us?
Jan 14 16:30:16 <pedronis> probably not, but is is quite pervasive
Jan 14 16:30:28 <hpk> yes, and Armin had good arguments
Jan 14 16:31:24 <pedronis> because the gateways are also space independent
Jan 14 16:31:47 <arigo> it's all a bit messy
Jan 14 16:32:26 <pedronis> I have converted most of interpreter/* to use
just int_w,str_w and unwrap_builtin
Jan 14 16:32:54 <arigo> there are really two approaches that exist in practice:
Jan 14 16:32:59 <pedronis> althoug unwrap_builtin could probably have a
better name
Jan 14 16:33:05 <arigo> in CPython we have real objects in co_consts
Jan 14 16:33:30 <arigo> in many other VMs the literals are stored in some
bytecodeish compact format
Jan 14 16:33:46 <arigo> and expanded into real objects by the equivalent of
LOAD_CONST
Jan 14 16:34:05 <arigo> e.g. LOAD_STRING n, where n somehow points to raw
string data
Jan 14 16:34:32 <arigo> so far PyPy does the latter, with LOAD_CONST n meaning
wrap the "raw" co_consts[n]
Jan 14 16:35:22 <arigo> a the nice property is that we don't have to preprocess
code objects: the interpreter could theoretically just run off mmaped .pyc files
Jan 14 16:36:20 <pedronis> ah but then our co_const would be in marshal
format
Jan 14 16:37:11 <arigo> yes, note that the marhsal format is similar to
whatever format we'd have to invent to represent a mixed type tuple anyway
Jan 14 16:38:09 <pedronis> well, for now I keep the unwrap and unwrap
Jan 14 16:38:33 <arigo> in theory, we could tell the translator that the Code
class and its attributes are implemented as a "view" over read-only packed data
in marshal format
Jan 14 16:38:45 <pedronis> and *if* we remove direct tuple list
wrapping/unwrapping
Jan 14 16:38:52 <arigo> and get a version of the interpreter that really
handles only marshal data, never "real" Code instances
Jan 14 16:39:18 <pedronis> yes
Jan 14 16:39:58 <hpk> probably
Jan 14 16:40:12 <arigo> well it's messy anyway, because of long or complex
literals that are maybe not R-Python types
Jan 14 16:40:38 <arigo> I mean, literals of type long or complex
Jan 14 16:41:33 <pedronis> also true
Jan 14 16:41:50 <pedronis> well with marshal approach
Jan 14 16:42:20 <pedronis> that's not an issue
Jan 14 16:42:37 <pedronis> but those wrap and unwrap need to be spelled in
a special way
Jan 14 16:43:13 <arigo> hum, it looks like a marshaled object is an object is
the MarshalObjSpace :-)
Jan 14 16:43:35 <pedronis> well that's still a special way
Jan 14 16:43:40 <arigo> then those wrap and unwrap become sending objects
across spaces
Jan 14 16:43:42 <pedronis> because it's a different space
Jan 14 16:43:49 <arigo> precisely.
Jan 14 16:43:55 <mwh> arigo: i suspect you might be getting ahead of yourself
here?
Jan 14 16:44:02 <pedronis> :)
Jan 14 16:44:13 <hpk> oh, he is probably thinking of making use of adapt()
:-)
Jan 14 16:44:26 <hpk> (not really)
Jan 14 16:44:33 <pedronis> for now I will leave the wrap and unwrap
Jan 14 16:44:36 <arigo> hpk: not that I am fully aware of :-)
Jan 14 16:44:39 <mwh> adapt(1, ApplicationLevel) ?
Jan 14 16:44:40 <pedronis> anyway I have not disabled
Jan 14 16:44:52 <mwh> adapt(1, IApplicationLevelThingie), rather
Jan 14 16:44:56 <pedronis> the wrap/unwrap functionality for tuples or
lists
Jan 14 16:45:00 <pedronis> if I do that
Jan 14 16:45:01 <mwh> let's use a decorator too!
Jan 14 16:45:08 <arigo> pedronis: ok
Jan 14 16:45:15 <pedronis> I will use some special names for those cases
Jan 14 16:45:18 <arigo> mwh: :-)
Jan 14 16:47:08 <arigo> well it's all fuzzy, e.g. what should a built-in
compiler produce? Code objects with what in the co_consts? It looks like it
shouldn't depend on a space...
Jan 14 16:48:01 <mwh> well, indeed
Jan 14 16:49:28 <pedronis> yes, but I don't see a problem in finding a
suitable space independent format for co_consts
Jan 14 16:49:51 <hpk> the fact that we want to have a space-independent repr
of a code object does not imply that PyPy's internal code objects could not be
space-dependent
Jan 14 16:50:15 <hpk> (sorry for the negation-complexity here :^)
Jan 14 16:52:54 <arigo> ok
Jan 14 16:53:25 <arigo> that's quite possible after all...
Jan 14 16:54:15 <hpk> also it seems that apart from LOAD_CONST we hardly have
"generic" wraps in interpreter/
Jan 14 16:54:43 <pedronis> yup
Jan 14 16:54:44 <mwh> most of them are wraps of obvious strings or integers,
i think
Jan 14 16:55:01 <hpk> yes, mostly wraps of messages, filenames, varnames
Jan 14 16:55:35 <pedronis> as I said the generic unwrap is also the one of
co_consts
Jan 14 16:55:42 <pedronis> s/the/the only
Jan 14 16:56:43 <arigo> there must be a generic wrap for default args of
gateways
Jan 14 16:57:53 <pedronis> yes, I said unwrap
Jan 14 16:58:22 <hpk> app2interp.getdefaults() does the generic wrap, right
Jan 14 16:58:28 <pedronis> it's also marked as NOT_RPYTHON
Jan 14 16:58:33 <hpk> yes :-)
Jan 14 16:58:36 <pedronis> and happens at bootstrap
Jan 14 16:58:47 <arigo> ok ok :-)
Jan 14 17:00:56 <arigo> if we want to allow things like new.code(....,
co_consts=(AnyThing(),), ...)
Jan 14 17:01:03 <arigo> to be done by the user
Jan 14 17:01:13 <arigo> then we have no choice but keep the object wrapped
Jan 14 17:01:37 <pedronis> true
Jan 14 17:03:16 <hpk> the same regarding func_defaults
Jan 14 17:03:25 <hpk> it works in CPython (to override func_defaults)
Jan 14 17:03:28 <hpk> but doesn't in PyPy apparently
Jan 14 17:04:15 <arigo> no?
Jan 14 17:04:39 <arigo> you can say def f(x=AnyThing())
Jan 14 17:04:48 <arigo> so you can create func_defaults with anything at all
Jan 14 17:05:01 <hpk> yes, maybe the current limitation is just shallow
Jan 14 17:05:02 <arigo> (even though we might not support assigning to
.func_defaults)
Jan 14 17:05:19 <pedronis> Functions are space depedent
Jan 14 17:05:21 <arigo> that's quite different, func_defaults are not a
syntactic property
Jan 14 17:05:31 <pedronis> in fact they store defaults as defs_w
Jan 14 17:06:07 <pedronis> it's code objects and gateways that are space
independent
Jan 14 17:10:54 <pedronis> looking further maybe it's not too hard to make
code objects space depedent if we want to
Jan 14 17:11:42 <pedronis> it seems gateways just need to become even more
lazy
Jan 14 17:12:26 <arigo> could BuiltinCode remain space-independent, and PyCode
be space-dependent?
Jan 14 17:13:53 <pedronis> I think so
Jan 14 17:14:08 <hpk> it should be possible
Jan 14 17:15:00 <pedronis> BuiltinCode is constructing a PyCode just to be
able to use .signature
Jan 14 17:15:15 <pedronis> but can be done in some other way
Jan 14 17:15:35 <hpk> i guess we should untangle the cases more
Jan 14 17:15:55 <hpk> maybe we tried too hard to reuse code for
BuiltinCode/PyCode cases
Jan 14 17:17:37 <pedronis> the question is, do we want PyCode to be space
depedent, it probably has to because of what we found
Jan 14 17:18:19 <arigo> I guess so
Jan 14 17:18:43 <pedronis> I can do the change
Jan 14 17:19:16 <pedronis> on the branch?
Jan 14 17:19:31 <hpk> makes sense i think
Jan 14 17:20:27 <pedronis> I'm going first to check in what I did so far
Jan 14 17:23:48 <arigo> ok :-)
_______________________________________________
[email protected]
http://codespeak.net/mailman/listinfo/pypy-dev
