[Python-ideas] Re: Add a "partial with placeholders" function to the functools module
Indeed it is important to not only consider the potential use cases for such a placeholder function, but to consider the cases that go beyond of what a lambda can do. A lambda is always a good option if it is used as a "single-serving" function, i.e. one that is only relevant locally where it is coded (and not stored somewhere). If on the other hand the function is to be reused or users are expected to interact with it, more clarity is needed (e.g. a descriptive `repr`). Of course in such cases a developer could always go with a proper function definition and even supply it with a doc string; it's just that for simple cases this feels a bit heavy. In the end it would provide a concise solution for cases where (a) partializing via keyword is not possible (positional-only parameters now being official might give rise to new use cases as well) and (b) the function is not "single-service", i.e. it is retained in some way (where PEP 8 says "do not assign a lambda"). As an example say there's a 2D array (semantics like numpy) and we want to convert a flat index to 2D-index (numpy contains such functionality but let's say we're doing something more lightweight): a = array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) convert_flat_index = placeholder(divmod, ..., a.shape[1]) assert a[convert_flat_index(4)] == 5 assert a[convert_flat_index(8)] == 9 The other option would be a full-fledged function definition (which seems a bit heavy): def generate_index_converter(n): def convert_flat_index(x): return divmod(x, n) return convert_flat_index ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XMHDFQGN5KLUFP7KKO3CDMXEBAXS542L/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add a "partial with placeholders" function to the functools module
An extension of `partial` itself would indeed be a clean solution (+ backwards compatibility), my concern was just that when most people are still using it in the "traditional" way (for example partializing via keyword), that the additional checks for `SKIP` introduce an unnecessary overhead for those cases. However by checking if `args` contain any placeholder in `__new__` and storing that information in a separate variable it should be possible to keep it to a minimum. Regarding third-party implementations one aspect to consider is compatibility with `partial` itself, i.e. what should happen when further `partial`'izing a previously defined placeholder function (it would need to be a placeholder function again, otherwise the `Ellipsis` or `SKIP` doesn't make much sense). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VNMZY5XBVR6UTOP7VMIF4XTRMNZPWPSH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Utilities for easier debugging
Minimal strawman proposal. New keyword debug. debug EXPRESSION Executes EXPRESSION when in debug mode. debug context Prints all the variables of the enclosing closure and all the variable names accessed within that block. For example, if in foo you access the global variable spam, spam would be printed. The format would be: variableName: value variableTwo: value where "value" is the repr() of the variable. Separated by new lines. The exact output format would not be part of the spec. ?identifier would print "identifier: value." Repr as before. Using this in non-debug mode emits a warning. ?identifier.property.property is also valid. A new property descriptor on the global variable, “debugger.” This is an alias for importing PDB and causing the debugger to pause there. The behavior of this descriptor in non-debug mode is TBD. Debug mode may be specified per-module at interpreter launch. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/B2FLIPTEFG72HTEA2DHAYBXMDKLNTRVP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: for ... except, with ... except
[redirecting back to the list] On 07/27/2019 09:38 PM, James Lu wrote: On Jul 27, 2019, at 12:44 PM, Ethan Furman wrote: Sure, folks /know/ what it means, but it's a common bug because it doesn't read as "if some_var is assigned 7" but as "if some_var is equal to 7". That’s a straw man- a modern linter or compiler would catch that. Many editors have linters integrated inside. The way to disable it is to put parentheses around the assignment expesssion. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TKH234SZCXCOCK2OTJPRZTWROY672QFQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Utilities for easier debugging
James Lu wrote:
> Minimal strawman proposal. New keyword debug.
> debug EXPRESSION
> Executes EXPRESSION when in debug mode.
> debug context
> Prints all the variables of the enclosing closure and all the variable names
> accessed
> within that block. For example, if in foo you access the global variable
> spam, spam would
> be printed. The format would be:
Wouldn't be dropping into PDB more efficient here? Anyway you could still use
something like `inspect.stack()` and inspect the closure of the function.
Together with a dedicated variable (from OS or via argv) you could distinguish
between debug and non-debug mode.
> variableName: value
> variableTwo: value
> where "value" is the repr() of the variable.
> Separated by new lines. The exact output format would not be part of the spec.
> ?identifier
> would print "identifier: value." Repr as before. Using this in non-debug mode
> emits a
> warning.
This you can do in Python 3.8 via f-strings:
[`f'{indentifier=}`](https://docs.python.org/3.8/whatsnew/3.8.html#f-strings-now-support-for-quick-and-easy-debugging)
> ?identifier.property.property
> is also valid.
> A new property descriptor on the global variable, “debugger.” This is an
> alias for
> importing PDB and causing the debugger to pause there.
This is possible via
[`breakpoint`](https://docs.python.org/3/library/functions.html#breakpoint).
> The behavior of this descriptor in non-debug mode is TBD.
> Debug mode may be specified per-module at interpreter launch.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/XLWLMQ4WIOKF7QQDL4U5F6W4ETEPUXRH/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Utilities for easier debugging
James Lu wrote:
> Minimal strawman proposal. New keyword debug.
> debug EXPRESSION
> Executes EXPRESSION when in debug mode.
> debug context
> Prints all the variables of the enclosing closure and all the variable names
> accessed
> within that block. For example, if in foo you access the global variable
> spam, spam would
> be printed. The format would be:
Wouldn't be dropping into PDB more efficient here? Anyway you could still use
something
like `inspect.stack()` and inspect the closure of the function. Together with a
dedicated variable (from OS or via argv) you could distinguish between debug
and non-debug
mode.
> variableName: value
> variableTwo: value
> where "value" is the repr() of the variable.
> Separated by new lines. The exact output format would not be part of the spec.
> ?identifier
> would print "identifier: value." Repr as before. Using this in non-debug mode
> emits a
> warning.
This you can do in Python 3.8 via f-strings:
[`f'{indentifier=}`](https://docs.python.org/3.8/whatsnew/3.8.html#f-strings-now-support-for-quick-and-easy-debugging)
> ?identifier.property.property
> is also valid.
> A new property descriptor on the global variable, “debugger.” This is an
> alias for
> importing PDB and causing the debugger to pause there.
This is possible via
[`breakpoint`](https://docs.python.org/3/library/functions.html#breakpoint).
> The behavior of this descriptor in non-debug mode is TBD.
> Debug mode may be specified per-module at interpreter launch.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/WZMNBLC3WDN2J7GV7PQTU7PTZWQZB3SM/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Utilities for easier debugging
On Jul 27, 2019, at 22:19, James Lu wrote: > > Minimal strawman proposal. New keyword debug. > > debug EXPRESSION > > Executes EXPRESSION when in debug mode. You really mean expression, not statement here? That doesn’t seem very useful. Sure, some expressions are used for side effects, but most are used for values, while most things you do want to execute for side effects are statements (and not just expression statements). Also, why is it not sufficient to use if DEBUG: expr (which already works, and can take a statement, even a compound statement)? > debug context > > Prints all the variables of the enclosing closure and all the variable names > accessed within that block. For example, if in foo you access the global > variable spam, spam would be printed. The format would be: What do you want this for? The previous statement seems like it would only be useful for non-interactive use, but this one seems like it would be useless there. If I’m trying to debug a daemon or a webservice or a GUI app or something, I probably want the debug output to go to my logs, or at least stderr, rather than stdout. An inspect function that gave a string I could print (or a stack of functions similar to the traceback module ones, where the lowest one gives a dict, but there are convenience helpers above that to give a pre-formatted string instead, or to just dump that string directly) seems like it would be more useful (although still, the top one seems like it should dump to stderr rather than stdout). How does the interpreter know which globals are accessed from the current frame? Looking at the frame’s code object’s names gets you about 70% of the way there, but that’s the names of all identifiers that aren’t local/cell/free vars, which includes more than global vars, and it doesn’t include globals looked up indirectly, and it includes globals that are only referenced in dead code, and so on. Plus, if you wanted that, you could write it today as a 3-line function using inspect; no need for a new statement. If you want the interpreter to keep track of things at runtime instead of relying on the statically available information, it seems like the only way to do that is to have debug mode do the equivalent of a per-opcode trace function or an all-variables watchpoint so it can maintain the information in case you ask for it. This would probably be pretty slow. Also, most of the globals used in most functions are the top-level functions and classes used in the function. Since these don’t usually change at runtime, and their reprs don’t tell you anything useful beyond their name, what’s the point in dumping them? Also, what if you have a variable named context? That’s hardly rare as a name, and your two statements seem to be ambiguous in that case. Also, except for this being a statement that needs a new keyword and a new context-sensitive keyword instead of a plain old function you stick in builtins named debug_context(), it seems like any feasible version of this could already be written and used in Python today. (The first one, using static frame info, should be about 3 lines of inspect calls.) > ?identifier > > would print "identifier: value." Repr as before. Using this in non-debug mode > emits a warning. This would break iPython’s improved interactive console, which already uses this syntax to provide a similar feature. And in non-interactive use, I can’t see any use for this. Especially since it would have to be wrapped in #if DEBUG, so the one-liner brevity doesn’t seem to buy you anything. > ?identifier.property.property > > is also valid. Why not just allow anything that’s valid as a target, like identifier[expr]? Or even just any expression at all? Is there an advantage to defining and using a similar but more limited syntax here? > A new property descriptor on the global variable, “debugger.” This is an > alias for importing PDB and causing the debugger to pause there. You want this added to every module’s globals? Why not put it in builtins instead? And why isn’t the existing builtin breakpoint sufficient? I can only see two differences. First, you have to call breakpoint as a function, not just reference it, but I’m having trouble imagining when that’s a problem. Second, your magic property would always use pdb, even if the breakpoint hook has been set to another debugger, but that seems like it’s just always worse when it’s different. > Debug mode may be specified per-module at interpreter launch. Wait, this new debug mode is separate from the existing debug mode? Then how do they interact? It might help to show us how all this stuff would actually be used in debugging some realistic code in some realistic scenario, maybe by giving us some code with functions whose bodies are like `#80 lines of complicated stuff to get the value for spam` followed by `?spam`, and then a fake console dump or interactive session log showing the program being
[Python-ideas] Fwd: Utilities for easier debugging
Sent from my iPhone Begin forwarded message: > From: James Lu > Date: July 28, 2019 at 6:23:06 PM EDT > To: Andrew Barnert > Subject: Re: [Python-ideas] Utilities for easier debugging > > >> On Jul 28, 2019, at 4:26 PM, Andrew Barnert wrote: >> >> You want this added to every module’s globals? Why not put it in builtins >> instead? > It’s not a function call, it’s an expression that when evaluated triggers a > breakpoint, so you have to add it as a property descriptor. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EQY2RQMRVQF2IERDNIB6IEAT2KI4RLPI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Fwd: Utilities for easier debugging
Sent from my iPhone Begin forwarded message: > From: James Lu > Date: July 28, 2019 at 6:21:04 PM EDT > To: Andrew Barnert > Subject: Re: [Python-ideas] Utilities for easier debugging > > >> On Jul 28, 2019, at 4:26 PM, Andrew Barnert wrote: >> >> Why not just allow anything that’s valid as a target, like identifier[expr]? >> Or even just any expression at all? Is there an advantage to defining and >> using a similar but more limited syntax here? > Hmm, it should support any expression. I limited it at first because it was a > minimal strawman. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W2M4CX6EVLBHKORB5T6YKA3XQGZFCT26/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Fwd: Utilities for easier debugging
Sent from my iPhone Begin forwarded message: > From: James Lu > Date: July 28, 2019 at 6:22:11 PM EDT > To: Andrew Barnert > Subject: Re: [Python-ideas] Utilities for easier debugging > > >> On Jul 28, 2019, at 4:26 PM, Andrew Barnert wrote: >> >> This would break iPython’s improved interactive console, which already uses >> this syntax to provide a similar feature. > If it’s so similar, I doubt it would break anything. This is intended to make > it easier to log variables for a complex application (multiple files). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ED7Z2YEZHQQLT5FHSHUN27KNITI6HNXO/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Fwd: Utilities for easier debugging
Sent from my iPhone Begin forwarded message: > From: James Lu > Date: July 28, 2019 at 6:21:04 PM EDT > To: Andrew Barnert > Subject: Re: [Python-ideas] Utilities for easier debugging > > >> On Jul 28, 2019, at 4:26 PM, Andrew Barnert wrote: >> >> Why not just allow anything that’s valid as a target, like identifier[expr]? >> Or even just any expression at all? Is there an advantage to defining and >> using a similar but more limited syntax here? > Hmm, it should support any expression. I limited it at first because it was a > minimal strawman. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/4LTFTK5ZPN52CQDPG72DG7BBP43677XM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Fwd: Utilities for easier debugging
Sent from my iPhone Begin forwarded message: > From: James Lu > Date: July 28, 2019 at 6:22:11 PM EDT > To: Andrew Barnert > Subject: Re: [Python-ideas] Utilities for easier debugging > > >> On Jul 28, 2019, at 4:26 PM, Andrew Barnert wrote: >> >> This would break iPython’s improved interactive console, which already uses >> this syntax to provide a similar feature. > If it’s so similar, I doubt it would break anything. This is intended to make > it easier to log variables for a complex application (multiple files). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NZMKXYJ2YCM53KVDPEN57ZMGAGCB5YQH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Utilities for easier debugging
On Sun, Jul 28, 2019 at 01:19:38AM -0400, James Lu wrote: > Minimal strawman proposal. New keyword debug. > > debug EXPRESSION > > Executes EXPRESSION when in debug mode. That's what assert does, in part. Since print is now a function, not a statement, you can do this: assert print(expression) or True to get the same effect. > debug context So "context" is also going to be keyword? That's two new keywords, breaking twice as much existing code: anything that uses "debug", anything that uses "context". By the way, you know that Python has a read-only global variable that tells you whether you are in debug mode? You can write a function to display anything you like, and wrap it in a test like this: if __debug__: display(locals()) -- Steven ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GP3EAWBC6QCXONPLP53RTYRRI2HNVKTJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Utilities for easier debugging
Steven D'Aprano wrote: > By the way, you know that Python has a read-only global variable that > tells you whether you are in debug mode? You can write a function to > display anything you like, and wrap it in a test like this: > if __debug__: > display(locals()) Oh interesting, I wasn't aware of this global. For anyone else just finding out about it, here's the [documentation](https://docs.python.org/3.9/library/constants.html#__debug__). Thanks for mentioning it, I feel like I end up learning something new in almost every python-ideas and python-dev thread, even if the proposal from the author isn't accepted. I should've subscribed a while ago (: Is there any other way to toggle/specify debug status other than the use of `-O`? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/UNT7ZXGMZJ773KHYO4GVSZCAB3GRZGRM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Fwd: Re: Universal parsing library in the stdlib to alleviate security issues
Forward to the list because Abusix had blocked google.com initially. Nam -- Forwarded message - From: Nam Nguyen Date: Sun, Jul 28, 2019 at 10:18 AM Subject: Re: [Python-ideas] Re: Universal parsing library in the stdlib to alleviate security issues To: Sebastian Kreft Cc: Paul Moore , python-ideas Let's circle back to the beginning one last time ;). On Thu, Jul 25, 2019 at 8:15 AM Sebastian Kreft wrote: > Nam, I think it'd be better to frame the proposal as a security > enhancement. Stating some of the common bugs/gotchas found when manually > implementing parsers, and the impact this has had on python over the years. > Seeing a full list of security issues (CVEs) by module would give us a > sense of how widespread the problem is. > Since my final exam was done this weekend, I gathered some more info into this spreadsheet. https://docs.google.com/spreadsheets/d/1TlWSf8iM7eIzEPXanJAP8Ztyzt4ZD28xFvUKeBuQtdA/ I think a strict parser can help with the majority of those problems. They are in HTTP headers, emails, cookies, URLs, and even low level socket code (inet_atoi). > Then survey the stdlib for what kind of grammars are currently being > parsed, what ad-hoc parsing strategy are implemented and provide examples > of whether having a general purpose parser would have prevented the > security issues you have previously cited. > Most grammars I have seen here come straight from RFCs, which are in ABNF and thus context-free. Current implementations are based on regexes or string splitting. My previous example showed that at least 30500, 36216, 36742 were non-issues if we started out with a strict parser. > > Right now, it is not clear what the impact of such refactor would be, nor > the worth of such attempt. > Exactly the kind of response I'm looking for. It is okay to suggest that the benefits aren't clear or that there are requirements X and Y that a general parser won't be able to meet, but it's not convincing to brush aside this because there is "existing, working code." Many of the bugs in that sheet are still open. It's not comfortable to say the code is working with a straight face as I have experienced with my own fix for 30500. I just couldn't tell if it was doing the right thing. > > What others have said earlier is that you are the one that needs to > provide some of the requirements for the proposed private parsing library. > And from what I read from your emails you do have some ideas. For example, > you want it to be easy to write and review (I guess here you would > eventually like it to be a close translation from whatever is specified in > the RFC or grammar specification). > Yes, that's the most important point because "readability counts." It's hard to reason about correctness when there are many transformations between the authoritative spec and the implementation. I definitely don't want to touch the regexes, string splits, and custom logic that I don't understand "why" they are that way in the beginning. How do I, for example, know what this regex is about ^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))? (It's from RFC 3986.) But you also need to take into consideration some of the list's concerns, > the parser library has to be performant, as a performance regression is > likely not to be tolerable. > Absolutely. That's where I need inputs from the list. I have provided my own set of requirements for such a parser library. I'm sure most of us have different needs too. So if a parser library can help you, let's hear what you want from it. If you think it can't, please let me understand why. Thanks, Nam > > > On Thu, Jul 25, 2019 at 10:54 AM Nam Nguyen wrote: > >> On Thu, Jul 25, 2019 at 2:32 AM Paul Moore wrote: >> >>> On Thu, 25 Jul 2019 at 02:16, Nam Nguyen wrote: >>> > Back to my original requests to the list: 1) Whether we want to have a >>> (possibly private) parsing library in the stdlib >>> >>> In the abstract, no. Propose a specific library, and that answer would >>> change to "maybe". >>> >> >> I have no specific library to propose. I'm looking for a list of features >> such a library should have. >> >> >>> >>> > and 2) What features it should have. >>> >>> That question only makes sense if you get agreement to the abstract >>> proposal that "we should add a parsing library. And as I said, I don't >>> agree to that so I can't answer the second question. >>> >> >> As Chris summarized it correctly, I am advocating for a general solution >> to individual problems (which have the same nature). We can certainly solve >> the problems when they are reported, or we can take a proactive approach to >> make them less likely to occur. I am talking about a class of input >> validation issues here and I thought parsing would be a very natural >> solution to that. This is quite similar to a context-sensitive templating >> library that prevents cross-site-scripting on the output side. So I don't >> know why (or what it takes) to convince
[Python-ideas] Re: Namespace context managers
On Fri, Jul 26, 2019 at 01:37:29PM -0700, Andrew Barnert wrote:
> It’s not clear to me whether people want a module-like, class-like, or
> function-like namespace here, but I do think it’s probably exactly one
> of those three. And it’ll be a lot easier to define and implement—and
> for everyone to understand future Python code—if it is.
I'm not going to speak for "people", but for me, I think that the answer
should be obvious: it is module-like.
I'm not even sure what you mean by "function-like" or "class-like"
namespaces. Functions aren't namespaces (except in the sense that they
are objects with a __dict__ and so support attributes): you can't access
a function locals from the outside. Classes are namespaces, but if you
want a namespace that behaves like a class, use a class.
For me, the idea is to (optionally) decouple module objects from .py
files. Of course most of the time when you want a module object, it is
most useful to split the code out into a seperate file, and that won't
change.
But there are times when you have a relatively small amount of code that
cries to be split off into a seperate namespace, but shifting it into a
seperate physical file is inconvenient. Going from one physical file to
two physical files is a significant jump in project complexity that's
not always worth the cost.
So a namespace object that behaves like a module but doesn't need to
split off into a seperate .py file would be helpful.
The code in a namespace object ought to behave precisely the same as
if it were copied into a seperate file. (*Almost* -- see below.)
That is, given some module file containing code:
# module.py
code
I should be able to copy the content of the file ("code"), and paste it
into another file like this:
# another.py
with Namespace("module") as module:
# paste and indent here
code
and the resulting namespace object bound to the name "module" should be
(almost) identical to what you would have got from calling "import
module" in the first place.
A few possible differences:
1. It isn't clear what the namespace __file__ and __package__ attributes
ought to contain, or if it should have them at all.
2. For introspection purposes, and to allow for possible future
changes, it might be wise to use a subclass of ModuleType for
namespace objects.
3. The repr will probably be different.
One more difference needs a bit of explanation:
The standard scoping rule used by Python is LEGB, which goes:
- Local
- Enclosing functions (non-local/closures)
- Global (module)
- Builtins
with a slight variation on that for code inside classes. So when you run
code in module.py, the global namespace it sees is module.py i.e.
itself.
But consider what happens when you copy the code from module.py and
paste it into another file, into a namespace. It would be surprising if
the code inside the namespace with block couldn't see module level
globals. So we need a slight variation on the scoping rule:
- Local
- Enclosing functions (non-local/closures)
- Namespace
- Global (module)
- Builtins
(and an analogous change for when classes are involved). To make it more
clear with a concrete example:
a = 1
b = 2
c = 3
with Namespace("ns") as ns:
a = 100
b = 200
def function():
a = 999
return a, b, c
ns.function() ought to return (999, 200, 3).
This implies a small difference in behaviour for code in a namespace
versus a seperate physical file. If I copied that code block out of ns
above, and pasted it into a physical file, then running function() would
raise:
NameError: name 'c' is not defined
I think that this slight difference will be considered desirable,
unsurprising (it would be surprising if namespaces didn't see their
surrounding global scope, and even more surprising if external modules
could pry into another module!) and hopefully uncontroversial.
> I don’t think anyone cares about fast locals here, and I hope nobody
> cares about things like super magic.
A function shouldn't become slower just because you move it into a
namespace. And classes ought to behave the same, including super,
whether they were defined inside or outside a namespace.
> But the differences in how
> variables in the namespace interact with functions and classes (and
> new-kind-of-namespace-things) defined within the namespace do probably
> matter. And you’d probably want to know what to expect from eval/exec,
> locals/globals/vars, etc.,
Absolutely! There are a couple of tricky corner cases involving eval
(and exec?) already, related to comprehensions. They will need to be
ironed out.
I've spent some time (unsuccessfully) trying to get this to work using a
class statement:
@namespace
class ns:
a = 100
def function():
print(a)
but I couldn't get the function inside the class to see the surrounding
"a". If Batuhan Taskaya has solved that problem using a with statement,
it sounds like there's no techni
[Python-ideas] Re: Namespace context managers
On Fri, Jul 26, 2019 at 10:52:46AM -0400, Calvin Spealman wrote:
> Let's say you do this or any of the variants suggested... What does this do?
>
> a = {"foo": 1}
> b = {}
> with a:
> with b:
> foo = 0
Re-writing that to my suggested version:
with namespace("a") as a:
with namespace("b") as b:
foo = 0
will create a namespace object (a subclass of ModuleType) and bind it to
"a", containing a namespace object "b", containing a variable "foo":
assert isinstance(a, NamespaceType)
assert isinstance(a.b, NamespaceType)
assert a.b.foo == 0
That would be analogous to this existing code:
class a:
class b:
foo = 0
except classes have different scoping rules to modules.
--
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/W3XEBOHQS3AWJ3Z26GA3H24GBGECM6FL/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Namespace context managers
I feel like this could have some interesting side uses (typing on mobile so
ignore the crappy indentation):
with Namespace('a'):
x = 1
y = 2
print(a.__dict__) # we now have a dict
The GN build system flat-out has no dictionary / mapping type in favor of
scopes like this.
On Sun, Jul 28, 2019, 9:36 PM Steven D'Aprano wrote:
> On Fri, Jul 26, 2019 at 10:52:46AM -0400, Calvin Spealman wrote:
>
> > Let's say you do this or any of the variants suggested... What does this
> do?
> >
> > a = {"foo": 1}
> > b = {}
> > with a:
> > with b:
> > foo = 0
>
> Re-writing that to my suggested version:
>
> with namespace("a") as a:
> with namespace("b") as b:
> foo = 0
>
> will create a namespace object (a subclass of ModuleType) and bind it to
> "a", containing a namespace object "b", containing a variable "foo":
>
> assert isinstance(a, NamespaceType)
> assert isinstance(a.b, NamespaceType)
> assert a.b.foo == 0
>
>
> That would be analogous to this existing code:
>
> class a:
> class b:
> foo = 0
>
>
> except classes have different scoping rules to modules.
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/W3XEBOHQS3AWJ3Z26GA3H24GBGECM6FL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/REXZHAZ2SW3CJMQHB7IETICF3XNVMRS4/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Namespace context managers
> > > It’s not clear to me whether people want a module-like, class-like, or > > function-like namespace here > > I'm not going to speak for "people", but for me, I think that the answer > should be obvious: it is module-like. > I was also talking 100% about module-like namespaces, and I also am not clear what definition or use the other two kinds would have. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/6PG7YFT2J2OTWU56M4G7LHOEJFPLW227/ Code of Conduct: http://python.org/psf/codeofconduct/
