[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jim Baker]

On Thu, Jul 8, 2021 at 6:25 PM Nick Coghlan  wrote:

> On Tue, 6 Jul 2021, 7:56 am Jim Baker,  wrote:
>
>>
>>
>> On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum  wrote:
>>
>>> FWIW, we could make f-strings properly nest  too, like you are proposing
>>> for backticks. It's just that we'd have to change the lexer. But it would
>>> not be any harder than would be for backticks (since it would be the same
>>> algorithm), nor would it be backward incompatible. So this is not an
>>> argument for backticks.
>>>
>>
>> Good point. At some point, I was probably thinking of backticks without a
>> tag, since JS supports this for their f-string like scenario. but if we
>> always require a tag - so long as it's not a prefix already in use (b, f,
>> r, fr, hopefully not forgetting as I type this email in a parking lot...) -
>> then it can be disambiguated using standard quotes.
>>
>
> There's a deferred PEP proposing a resolution to the f-string nesting
> limitations:
> https://www.python.org/dev/peps/pep-0536/#motivation
>
>
Thanks for pointing that PEP out - this looks quite reasonable for both
f-strings and for the tagged templates proposed here. I believe some
limitations on nesting expressions in f-strings with quote changing was
introduced in a relatively recent fix in 3.8 or 3.9, but I need to find the
specifics.


>>
>>> Separately, should there be a way to *delay* evaluation of the templated
>>> expressions (like we explored in our private little prototype last year)?
>>>
>>
>> I think so, but probably with an explicit marker on *each* deferred
>> expression. I'm in favor of Julia's expression quote, which generally needs
>> to be enclosed in parentheses, but possibly not needed in  expression
>> braces (at the risk of looking like a standalone format spec).
>> https://docs.julialang.org/en/v1/manual/metaprogramming/
>>
>> So this would like
>> x = 42
>> d = deferred_tag"Some expr: {:(x*2)}"
>>
>> All that is happening here is that this being wrapped in a lambda, which
>> captures any scope lexically as usual. Then per that experiment you
>> mentioned, it's possible to use that scope using fairly standard - or at
>> least portable to other Python implementations - metaprogramming,
>> including the deferred evaluation of the lambda.
>> (No frame walking required!)
>>
>> Other syntax could work for deferring.
>>
>
> It reminds me of the old simple implicit lambda proposal: https://www.
> python.org/dev/peps/pep-0312/
>
> The old proposal has more ambiguities to avoid now due to type hinting
> syntax but a parentheses-required version could work: "(:call_result)"
>
>
PEP 312 - suitably modified - could be quite useful for both deferring
evaluation in tagged templates and function calls (or other places where we
might want to put in a lambda). Ideally we can use minimum parens as well,
so it's

tag"{:x+1}"
f(:x+1, :y+2)

(but I haven't looked at ambiguities here).

A further idea that might make the approach in PEP 312 more powerful than
simply being an implicit lambda is if this was like Julia's quoted
expressions https://docs.julialang.org/en/v1/manual/metaprogramming/#Quoting
and
we recorded the *text body of the expression*. (Such recording would
presumably be done in any template scheme, but let's make this explicit
now.)

So let's call this implicit lambda with recorded text body a *quoted
expression*.

What's nice about doing such quoted expressions is that the lambda also
captures any lexical scope. So if I have an expression tag"{:x*y}", the
variables x and y are referenced appropriately. This means it would be
possible to do such things as rewrite complex expressions - eg an index
query on a Pandas data frame - while avoiding the use of dynamic scope.

I wrote a small piece of code to show how this can be exercised. The text
of the expression is simply an attribute on the lambda named "body", but we
might have a new type defined (eg types.QuotedExpression).

```
from functools import update_wrapper
from textwrap import dedent
from types import FunctionType


def rewrite(f, new_body):
# Create a temporary outer function with arguments for the free
variables of
# the original lambda wrapping an expression. This approach allows the
new
# inner function, which has a new body, to compile with its dependency
on
# free vars.
#
# When called, this new inner function code object can continue to
access
# these variables from the cell vars in the closure - even without this
# outer function, which we discard.
#
# This rewriting is generalizable to arbitrary functions, although the
# syntax we are explorin

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jim Baker]

On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum  wrote:

> FWIW, we could make f-strings properly nest  too, like you are proposing
> for backticks. It's just that we'd have to change the lexer. But it would
> not be any harder than would be for backticks (since it would be the same
> algorithm), nor would it be backward incompatible. So this is not an
> argument for backticks.
>

Good point. At some point, I was probably thinking of backticks without a
tag, since JS supports this for their f-string like scenario. but if we
always require a tag - so long as it's not a prefix already in use (b, f,
r, fr, hopefully not forgetting as I type this email in a parking lot...) -
then it can be disambiguated using standard quotes.


> Separately, should there be a way to *delay* evaluation of the templated
> expressions (like we explored in our private little prototype last year)?
>

I think so, but probably with an explicit marker on *each* deferred
expression. I'm in favor of Julia's expression quote, which generally needs
to be enclosed in parentheses, but possibly not needed in  expression
braces (at the risk of looking like a standalone format spec).
https://docs.julialang.org/en/v1/manual/metaprogramming/

So this would like
x = 42
d = deferred_tag"Some expr: {:(x*2)}"

All that is happening here is that this being wrapped in a lambda, which
captures any scope lexically as usual. Then per that experiment you
mentioned, it's possible to use that scope using fairly standard - or at
least portable to other Python implementations - metaprogramming, including
the deferred evaluation of the lambda.
(No frame walking required!)

Other syntax could work for deferring. Maybe backticks, they could be
available? 

- Jim



> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2ECKVDW7A5PHJWJHI6QUS6CPW46X2IHA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jim Baker]

On Mon, Jul 5, 2021, 12:56 PM Barry Scott  wrote:

>
>
> On 5 Jul 2021, at 08:07, Thomas Güttler  wrote:
>
> This means backticks, but without the dollar sign.
>
>
> In bash the backtick was so often a problem that $(cmd) was added.
>
> Having removes the grit-on-Tim's-screen backtick in python 3 I would
> not like to see it return with its issue of being confused with
> single-quote.
>

One mitigation is that the backtick should always require a tag as a
prefix. So seeing something like

elem = html`Some item: {value}`

is hopefully fairly obvious what's going on - it's not just going to be
mixed up with a single quote. Uses like log(f`foo`) should be hopefully
discouraged, in the same way that we don't use l (that's the lower-case
letter L if you're not reading this email with the numeric codepoints) as a
variable, but we are happy enough to write something like limit = 42 - it's
clear in the context.

However, I think the Bash example is exactly illustrative of the opposite.
So in reviewing at least this one FAQ on  the topic
http://mywiki.wooledge.org/BashFAQ/082, it reminds me of why I don't use
backticks in Bash - it's the lack of nesting support when compared to
$(...).

Ironically, this nesting is exactly what backticks can help here on - an
unused character as of Python 3, commonly used for building some type of
string/identifier in a variety of languages, that we can give some nice
semantics that allows for simple nesting when used in conjunction with
braces delimiting expressions. Such braces of course always give us a new
nesting, similar to the statement in the wiki above that "$() forces an
entirely new context for quoting, so that everything within the command
substitution is protected and can be treated as though it were on its own,
with no special concern over quoting and escaping." That this usage of
backticks has worked quite well for JavaScript provides some useful
confirmation.

I also expect that no one will confuse this with Bash usage, given that's
going to be in Python code - except perhaps in a readily written sh
function (so something like sh`...`, which returns a list of stdout or
something like that and makes use of shlex, etc). I will leave what horrors
that could actually look like to the reader :) although in limited form, it
could be quite useful.

- Jim
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/I7K57YBDLNM5ZAF4SQORFXPG7OIRPE3G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jim Baker]

JavaScript's tagged template literals do provide nice ergonomics. These
should work well with some variation on PEP
501's types.InterpolationTemplate as the backend implementation. Some
thoughts on what that should look like:

* I also increasingly prefer the idea of using backticks to define such
templates, as opposed to further overloading standard quoting. More below.
* We should separate the static elements of the template from evaluated
expressions. This implies that a tagged template function would have a
signature like def html(template, *exprs). To provide the same user
interface as PEP 501, one could write a tag template function "i" that is
an identity function - thus demonstrating a basic equivalence of these
proposals.
* Literal static strings from the templates are available in both raw and
cooked forms (Unicode escapes applied). So there's no need to add a "r"
prefix for raw.
* Expressions are evaluated immediately using normal scoping and rendered
later (taking into account escaping rules, et). Such rendering can be into
any Python object. So using some "html" tag function would return a DOM
(let's call it HtmlElement in this example) which can be further rendered
eventually into returned text in the scenario of serving HTML.
* Greater likelihood of syntax support. Being able to specify html`Some
expr: {x * y}` probably simplifies being able to provide an IDE
extension that is aware that the html tag function was imported from some
popular library, and can therefore provide assistance with HTML tags or any
other aspects of the templating language used.
* Backticks are multiline and they nest in the context of expressions. This
means there's no need to invent new templating syntax, such as Jinja's {%
for ... %} ... {% endfor %} to provide iteration over a sequence. Just use
Python in the evaluated expressions. So with reference to the example in
the Jinja doc https://jinja.palletsprojects.com/en/3.0.x/templates/#synopsis,
we could have functions like the following that can be further composed to
create larger DOM objects:

def ulistify(id: str, linkage: dict[str, str]) -> HtmlElement:
# uses a dict to represent the linkage in this example, so slightly
different than the Jinja example
return html`
{
  html`{caption}` for href, caption in
linkage.items()
}`

It's possible to write something similar using f-string syntax now, but it
involves careful tracking of which quotes are being used to support
nesting. The interaction of backticks and expression syntax keeps it simple
in comparison.

An example library in the JavaScript ecosystem that builds out from the JS
tagged template literal support is Lit (https://lit.dev/,  note that I
haven't used this library.) Lit provides some nice functionality by
composing HTML DOM out of DOM fragments; and doing this in a lazy fashion
(virtual DOM, as ReactJS demonstrated; I am not aware of a fast virtual DOM
library in Python as a C Extension, but it would seem like a
straightforward thing to write). Anyway, Lit and other similar libraries in
JS could help understand the scope of the available innovation if we were
to add such functionality.

Format specifiers and ! conversions should presumably still be available in
expressions and thus available from the template literal object to be used
as part of any rendering. However, I'm leaving this open for the moment as
to exactly what this looks like - especially if I overlooked any parsing
issues for such support!

Lastly, it's worth noting that types.InterpolationTemplate could look quite
similar to https://262.ecma-international.org/6.0/#sec-tagged-templates if
it's a constant object - a template literal in other words - which a tagged
function is then applied to with the evaluated expressions.

- Jim


On Mon, Jul 5, 2021 at 1:10 AM Thomas Güttler 
wrote:

>
> Am Fr., 2. Juli 2021 um 12:06 Uhr schrieb Nick Coghlan  >:
>
>>
>>
>> On Fri, 2 Jul 2021, 5:12 pm Thomas Güttler, 
>> wrote:
>>
>>> Hi Nick and all other Python ideas friends,
>>>
>>> yes, you are right. There is not much difference between PEP-501 or my
>>> proposal.
>>>
>>> One argument why I would like to prefer backticks:
>>>
>>> Some IDEs detect that you want to use a f-string automatically:
>>>
>>> You type:
>>>
>>> name = 'Peter'
>>> print('Hello {name...
>>>
>>> and the IDE automatically adds the missing "f" in front of the string:
>>>
>>> name = 'Peter'
>>> print(f'Hello {name...
>>>
>>> This is a handy feature (of PyCharm), which would not work reliably if
>>> there are two different prefixes.
>>>
>>> ---
>>>
>>> You mentioned these things:
>>>
>>> eager rendering: I think deferred rendering would increase the
>>> complexity a lot. And I think it is not needed.
>>>
>>
>> Eager rendering is f-strings. Any templating proposal necessarily
>> involves a delayed rendering step, when the template is combined with the
>> interpolated values.
>>
>> runtime value interpolation: It is up to the receiver of
>>> types.InterpolationTemplate 

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jim Baker]

Supporting JavaScript-style template literals, specifically tagged
templates, could be generally useful. If we look at
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals#tagged_templates,
it's a really simple idea:

The tag function is called with the following arguments: the strings from
the template (the raw string is also available), plus any evaluated
expressions referenced by that string. This idea can be equivalently
expressed in a number of ways in Python. So this might be:

```
from some_html_templating_library import html_template_function as h

num = 42
constraint = "A number >  40"

html_text = h"{num} complies with the constraint
{constraint}"
```

which results in `html_text` being set to `h(template, 42, "A number >
40")`, where `template` is some object that contains at the very least the
original raw string as well as the non-expression strings.

Notably such tagged template literals - so just like JavaScript tagged
templates - are not restricted to HTML or even to returning strings, but
instead could be used for a variety of additional purposes, that I won't
work out this moment:

* Logging strings which don't have to format their expressions - so similar
to current logging when not using f-strings, but with the easy interface of
f-strings, including desired formatting of expressions in simple logger
scenarios
* Internationalization of strings
* SQL queries with sanitization of expressions (to avoid "Bobby Tables"
injection)
* Object literals - something like D"10.2" instead of the somewhat longer
D("10.2"), assuming for the latter `from decimal import Decimal as D` (this
is a pretty weak argument on the face of it, but I can see the possibility
of being more restrictive in the use of monkey patching namespaces with
such literals to avoid unnecessary function call overhead...)
* Code (or AST) construction, following the quasiquote idea from Lisp
* Latex and other minilanguages (is Latex properly a minilanguage? ;) )

For some of these use cases, ideally we don't use `sys._getframe` if this
were to be implemented - assuming f-string style expressions, it's
perfectly feasible to use a lambda to capture any variables as desired
using lexical scoping instead of the dynamic scoping of `sys._getframe` AND
we need deferred evaluation of the expression. Some shorthand for the
lambda keyword could be nice however.

I should mention that when I looked at some similar ideas last summer (
https://github.com/jimbaker/fl-string-pep/, useful for some experiments on
lexical scoping), in part with Guido, I was thinking that an implicit
lambda should be used for all such expressions in tagged template literals.
But my feeling now is that this would not be so Pythonic - we explicitly
mark such usages now, and we should continue to do so. Just need to find
the right character sequence for such shorthand!

On Sat, Jun 12, 2021 at 5:51 AM Jeff Allen  wrote:

> On 10/06/2021 15:46, Ricky Teachey wrote:
>
>
> Makes me wonder if stepping back and thinking, instead, about a way to
> automatically (optionally?) put the current calling context dictionary(ies)
> somewhere that a called function can get at it/them, so it can do what it
> wants with the context.
>
> I think that would violate the expectation that a function call means the
> same thing in different contexts, hence f-strings work by supplying values
> at the "call site" not variables. I know you will find exceptions (like
> import context), but they are not as all-consuming as snapping the entire
> namespace. If you really want to, there's sys._getframe.
>
> Jeff
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/DUEB3HLOW3LI4RXTLZN7JQFFX44FBWSJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DYHW6SFPO4LPOY4MK33TUKKVTOPGGNMH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make ~ (tilde) a binary operator, e.g. __sim__(self, other)

[Jim Baker]

I am no expert on R, but R lazily evaluates arguments to functions; see
https://cran.r-project.org/doc/manuals/r-devel/R-lang.html#Argument-evaluation
(plus
the rest of that page, which is the language spec). Tilde is strictly used
for modeling. Also relevant would be the operator precedence
https://cran.r-project.org/doc/manuals/r-devel/R-lang.html#Infix-and-prefix-operators
-
note that ~ has low precedence, which makes sense in how it is used.

We have a very straightforward way of lazily evaluating these formulas in
Python, not to mention get the correct precedence - encapsulate as a string
and use something like Patsy to parse!

On Mon, Feb 24, 2020 at 11:59 AM David Mertz  wrote:

> Well... also, the meaning in R is quite a bit different from any of the
> meanings suggested by Wolfram.  In fact, although the most common use in R
> is "depends on", it's technically just a generic delayed evaluation without
> any inherent semantics at all.  Or, that is to say, tilde is just a certain
> kind of quotation, and we already have quotation in Python.
>
> On Mon, Feb 24, 2020 at 1:28 PM  wrote:
>
>> Aaron Hall wrote:
>> > The context for this is statistics , so I'll quote Wolfram on tilde in
>> the context of
>> > statistics: http://mathworld.wolfram.com/Tilde.html
>> > "In statistics, the tilde is frequently used to mean "has the
>> distribution (of)," for
>> > instance, X∼N(0,1) means "the stochastic (random) variable X has the
>> distribution N(0,1)
>> > (the standard normal distribution). If X and Y are stochastic variables
>> then X∼Y means "X
>> > has the same distribution as Y."
>>
>> I think that you have refuted your own idea. You have argued that ~ is
>> rightful statistical operator. But Python is not an statistical language.
>> Python is a general purpose programming language while R is a statistical
>> one. They have different domains so what is useful and right in R it is not
>> necessary useful and right in Python. I cannot see a case for a statistical
>> operator in Python.
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/IZMGSVIEHWDASTLUDP5AXLQKMZ4BPDGR/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/6DQC4AOHMVL6RLMLAA3GT3LMJROUULQ5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3G7YVHALBLA6RCNMM2AB3NI5S37Z6BZJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make ~ (tilde) a binary operator, e.g. __sim__(self, other)

[Jim Baker]

Supporting ~ as a binary operator is an interesting idea, especially given
the relatively limited usage of unary ~. However, the big hole in this
proposal for formulas is that there is a de facto standard "minilanguage"
for writing such formulas in Python, namely what Patsy supports:
https://patsy.readthedocs.io/en/latest/formulas.html Patsy is used by
statsmodels and other tools to support the same formulas as we see in R (or
S).

We immediately see the problem with the interaction operator : (colon),
which conflicts with how it is used to support annotations in Python. Given
that this formula minilanguage is comprehensive, this seems to be a fatal
objection.

Note that deferred evaluation is sort of a red herring - it is
straightforward to defer execution in Python's object model, as we see in
SymPy, Pandas dataframes where clauses, and SQLAlchemy, among other
examples.


On Sun, Feb 23, 2020 at 5:37 PM  wrote:

> Aaron Hall wrote:
> > Currently, Python only has ~ (tilde) in the context of a unary operation
> (like
> > -, with __neg__(self), and +, __pos__(self)).
> > ~ currently calls __invert__(self) in the unary context.
> > I think it would be awesome to have in the language, as it would allow
> modelling along the
> > lines of R that we currently only get with text, e.g.:
> > smf.ols(formula='Lottery ~ Literacy + Wealth + Region', data=df)
> > With a binary context for ~, we could write the above string as pure
> Python, with
> > implications for symbolic evaluation (with SymPy) and statistical
> modelling (such as with
> > sklearn or statsmodels) - and other use-cases/DSLs.
> > In LaTeX we call this \sim (Wikipedia indicates this is for "similar
> to").
> > I'm not too particular, but __sim__(self, other) would have the benefits
> of
> > being both short and consistent with LaTeX.
> > This is not a fully baked idea, perhaps there's a good reason we haven't
> added a binary
> > ~.  It seems like I've seen discussion in the past. But I couldn't find
> such
> > discussion. And as I'm currently taking some statistics courses, I'm
> getting
> > R-feature-envy again...
> > What do you think?
> > Aaron Hall
>
> I really do not fully understand your proposal. I do not know nothing
> about R and my statistical knowledge has gone long ago.
>
> However, I think that we cannot expect that Python accommodates every
> existing domain. Let me explain: Python have not special features, syntax,
> operators to deal with SQL, HTML, ini files, OpenGL, etc. These domains,
> and others, are supported via libraries, outside of the language core.
>
> ~ exists in bit-wise context and, as long as I know, it comes from C --I
> have never used it indeed in Python. It is a unary operator because it
> works in that way as a bitwise operator.
>
> I cannot see any improvement in becoming ~ into a binary operator. I
> imagine that a binary ~ would have a completely different meaning from a
> unary ~. I can foresee many problems here.
>
> In my opinion, you should prove that binary ~ has a relevant benefit for
> the whole language, not just for R tasks. It should be useful in some
> different domains and behave consistently --or at least so consistent as
> possible-- in those domains.
>
> Can you, for instance, envision other uses of binary ~ beyond R?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UMXBIM6TSOTR76KOBOJXGVJUDMX7IHBT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/V74C274XICEJKE57IVULBWHF46N3L2BF/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Hexadecimal floating literals

[Jim Baker]

On Thu, Sep 21, 2017 at 7:32 PM, Steven D'Aprano 
wrote:

> On Thu, Sep 21, 2017 at 01:09:11PM -0700, David Mertz wrote:
> > -1
> >
> > Writing a floating point literal requires A LOT more knowledge than
> writing
> > a hex integer.
> >
> > What is the bit length of floats on your specific Python compile?
>
> Are there actually any Python implementations or builds which have
> floats not equal to 64 bits? If not, perhaps it is time to make 64 bit
> floats a language guarantee.
>

Jython passes the hexadecimal float tests in Lib/test/test_float.py, since
Java uses 64-bit IEEE 754 double representation for the storage type of its
double primitive type. (One can further constrain with strictfp, for
intermediate representation, not certain how widely used that would be. I
have never seen it.) In turn, Jython uses such doubles for its PyFloat
implementation.

I wonder if CPython is the only implementation that could potentially
supports other representations, such as found on System/360 (or the
successor z/OS architecture). And I vaguely recall VAX VMS had an
alternative floating point, but is that still around???

- Jim
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Hexadecimal floating literals

[Jim Baker]

On Thu, Sep 21, 2017 at 1:57 AM, Paul Moore  wrote:

> ...
>
> It's also worth remembering that there will be implementations other
> than CPython that will need changes, too - Jython, PyPy, possibly
> Cython, and many editors and IDEs. So setting the bar at "someone who
> wants this will have to step up and provide a patch" seems reasonable
> to me.
>

It would be more or less trivial for Jython to add such support, given that
Java has such support natively, and we already leverage this support in our
current implementation of 2.7. See
https://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#valueOf(java.lang.String)
if curious. We just need to add the correct floating point constant that is
parsed to Java's constant pool, as used by Java bytecode, and it's done.

I'm much more concerned about finishing the rest of 3.x.

- Jim
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Thread-safe generators

[Jim Baker]

This is a bad idea in the generator itself, as commented earlier by others
here.

>From a cross implementation perspective, in Jython, different threads can
call next on a non running generator, *so long as they coordinate with each
other external to any use of this generator*, and this works fine.

But any reliance on gi_running, as seen here, can only be considered to be
possible help in detecting such races; it would not even come close to
preventing a race:
https://github.com/jythontools/jython/blob/master/src/org/python/core/PyGenerator.java#L146

(We don't even bother making gi_running a volatile to get actual
test-and-set style semantics, because really it makes no sense to pretend
otherwise; and why pay the performance penalty?)

The idea of putting generators behind a queue sounds reasonably workable -
the semantics then are the right ones, although implementing this
efficiently is the trick here.


On Sun, Apr 16, 2017 at 11:08 PM, Nick Coghlan  wrote:

> On 17 April 2017 at 08:00, Paul Moore  wrote:
> > On 15 April 2017 at 10:45, Nick Coghlan  wrote:
> >> So I'd be opposed to trying to make generator objects natively thread
> >> aware - as Stephen notes, the GIL is an implementation detail of
> >> CPython, so it isn't OK to rely on it when defining changes to
> >> language level semantics (in this case, whether or not it's OK to have
> >> multiple threads all calling the same generator without some form of
> >> external locking).
> >>
> >> However, it may make sense to explore possible options for offering a
> >> queue.AutoQueue type, where the queue always has a defined maximum
> >> size (defaulting to 1), disallows explicit calls to put(), and
> >> automatically populates itself based on an iterator supplied to the
> >> constructors. Once the input iterator raises StopIteration, then the
> >> queue will start reporting itself as being empty.
> >
> > +1 A generator that can have values pulled from it on different
> > threads sounds like a queue to me, so the AutoQueue class that wraps a
> > generator seems like a natural abstraction to work with. It also means
> > that the cost for thread safety is only paid by those applications
> > that need it.
>
> If someone did build something like this, it would be interesting to
> benchmark it against a more traditional producer thread model, where
> one thread is responsible for adding work items to the queue, while
> others are responsible for draining them.
>
> The trick is that an auto-queue would borrow execution time from the
> consumer threads when new values are needed, so you'd theoretically
> get fewer context switches between threads, but at the cost of
> changing the nature of the workload in a given thread, and hence
> messing with the working set of objects it has active.
>
> It may also pair well with the concurrent.futures.Executor model,
> which is already good for "go handle this predefined list of tasks",
> but currently less useful as a replacement for a message queue with a
> pool of workers.
>
> Setting the latter up yourself is currently still a bit tedious, since:
>
> 1. we don't have a standard threading Pool abstraction in the standard
> library, just the one tucked away as part of multiprocessing
> 2. while queue.Queue has native support for worker pools, we don't
> provide a pre-assembled version that makes it easy to say "here is the
> producer, here are the consumers, wire them together for me"
>
> There are good reasons for that (mainly that it's hard to come up with
> an abstraction that's useful in its own right without becoming so
> complex that you're on the verge of reinventing a task manager like
> celery or a distributed computation manager like dask), but at the
> same time, the notion of "input queue, worker pool, output queue" is
> one that comes up a *lot* across different concurrency models, so
> there's potential value in providing a low-barrier-to-entry
> introduction to that idiom as part of the standard library.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/