On 11/13/2023 11:44 PM, AVI GROSS via Python-list wrote:
Dom,
I hear you.
As you say, writing your own extension in something like C++ may not appeal to
you even if it is faster.
I was wondering if using a generator or something similar in R might make sense.
I mean what happens if you write a function that includes a "yield" or two and
does a part of what you want. It maintains some internal state between invocations. So
you can call it once to setup things then call it repeatedly to keep processing the next
item. You stop calling it when you get a result you want, such as that it has seen what
you want N times.
Since the code stays in memory, it may effectively run faster than some other
kinds of functions calls. It can keep things in internal storage such as not
just how many N you want but how many it has seen.
I'm inclined to just turn the iterable into a set to get the values,
then iterate through those values calling count() on a listified version
of the iterable. If the count >= target, return.
It may not be the fastest one could do but it's simple and probably
pretty fast for many uses. I suppose that for some iterables it would
be better not to turn them into lists, but one could figure out about
that after working out more carefully what cases need to be covered.
Your outer function can maintain a list of the items you want to XOR or
generate a new one dynamically as needed. It can use functional programming
techniques to create a new customized version of the iterator, such as with a
value of N built in. You would then call the outer function and let it use the
inner function till the result is available or until the data in the iterator
runs out or perhaps other tweaks involving two way communication of sorts
between the functions.
I am NOT suggesting this approach is optimal or fast but merely wondering if
something along these lines is worth trying that might speed things up even if
not very fast. Such approaches can be even more effective if what you are
working on need not all be instantiated up front but can be dynamically
calculated or incrementally read from files. With care, you can make multiple
instantiations that each iterate over their own sets of data without
interference.
Just a thought. In a sense, this can be a slightly decent substitute for the
non-standard evaluation in R where you can arrange for lots of your data to not
be interpreted till absolutely needed.
-----Original Message-----
From: Dom Grigonis <dom.grigo...@gmail.com>
Sent: Monday, November 13, 2023 10:12 PM
To: avi.e.gr...@gmail.com
Cc: Grant Edwards <grant.b.edwa...@gmail.com>; Python <python-list@python.org>
Subject: Re: xor operator (DEPRECATED)
Fair point. However, I gave it a shot for the following reason:
I couldn’t find a way to make such performant function. Using python builtin
components still ends up several times slower than builtin `all`. Cython or
numba or similar is not an option as they do not support `truth` values. Or if
they do, it ends up slower than pure python variant.
So all what is left is writing a proper extension. Which I would prefer not to
do for 1 function. I thought maybe `xor`, as in logical XOR functionality in
its vanilla case could be compelling. And after doing a bit of search I see
that very very few languages have that and it seems for a good reason.
Some that do: R, thats all I could find. Although some (if not many) went
through the proposal phase. And yes, none of them have a function that I am
proposing.
So yes, you are right, not a good proposal.
But there still seems to be the need for short-circuiting performant
implementations in python space. The issue is that there are many variants of
what might be needed while there is no efficient solution to sourcing
predicates from python to lower level implementations. Someone mentioned that
numpy experimented with such implementations in C, but they did not get
anywhere with it.
The best I could come up with is cached numba for numpy problems, which does
perform very well and more than worth it if function is re-used. It even ends
up faster than cython or cffi extensions, however can’t have many of those due
to JIT and AOT is currently being deprecated (which wouldn’t solve anything
anyway). However, as I mentioned earlier it does not apply to this case.
So it’s either:
a) Something very clever and flexible implemented that covers most of such
needs and doesn’t require predicates.
b) I welcome any thoughts on this.
DG
On 14 Nov 2023, at 04:27, AVI GROSS via Python-list <python-list@python.org>
wrote:
I was going to ask a dumb question. Has any other language you know of made
something available that does what is being asked for and included it in the
main program environment rather than an add-on?
A secondary mention here has been whether short-circuiting functions like
"any" and "all" have been augmented with something like "has_n" that
evaluates arguments till it has found n or perhaps n+1 of what it wants then
skips the rest. Does any language supply something like that? What would
such a function return and does it have an "any" or an "all" side?
It sounds like if I asked if a list of integers has at least n prime numbers
in "any" mode, it should ignore any that are not primes till it finds n
primes or fails and returns true or false. If in "all" mode, I assume it
would have to be the first n items without a failure.
Fine, but then someone may want to know WHERE you stopped or for you to
return the sublist of the ones that made the match, or even return
everything that was skipped so you can later process that. Consider a long
list of jurors you process to place a dozen that qualify on a jury and then
later you want to choose from among the rest for another jury.
Human minds can come up with an amazing number of ideas including for
"useful" functions or features but I find the vast majority would rarely be
used as nobody remembers it is available and some fairly simple method using
other functions can easily be cobbled together.
-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail....@python.org> On
Behalf Of Grant Edwards via Python-list
Sent: Monday, November 13, 2023 8:19 PM
To: python-list@python.org
Subject: Re: xor operator
On 2023-11-14, Dom Grigonis via Python-list <python-list@python.org> wrote:
Except the 'any' and 'all' builtins are _exactly_ the same as bitwise
or and and applided to many bits. To do something "in line" with that
using the 'xor' operator would return True for an odd number of True
values and False for an even Number of True values.
Fair point.
Have you ever encountered the need for xor for many bits (the one
that I am NOT referring to)? Would be interested in what sort of
case it could be useful.
Yes, it's used all the time in low-level communications protocols,
where it's often implemented in hardware. But, it is also not at all
unusual to implement it in software.
It's also not that unusual for the "count-ones" part of the function
you're asking for to be implemented in hardware by a CPU having an
instruction that counts the number of 1 bits in a register.
GCC has a low-level builtins called __builtin_popcount() and
__builtin-popcountl() that counts the number of 1's in an unsigned
(long) int.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
