Count-Trailing-Zeros(x) --- Can I use Log(x) to see if X is equal to ( Integer * 10 ^ k) ???

[Hen Hanna]

Count how many zeros are at the end of your int:

 defend_zeros(num):
 s = str(num)
 return len(s) - len(s.rstrip("0"))


print(end_zeros(10))   # == 1
print(end_zeros(101))  # == 0
print(end_zeros(245))   # == 0
print(end_zeros(100100))# == 2


   Writing a loop (using   %  and // )  feels like [reinventing 
the wheel] and   seems  intrinsically wrong.


 i like  this code (above)  because   it's not reinventing the 
wheel --- the Python  implementor has already done the loop  (using   %  and // 
),  so i'm just using that efficient  tool  (routine).
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue517371] Add .count() method to tuples

[admin]

Change by admin :


--
github: None -> 36100

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400884] arraymodule: adding count, extend, index, pop, remove

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400884] arraymodule: adding count, extend, index, pop, remove

[admin]

Change by admin :


--
github: None -> 32619

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45146] Add a possibility for asyncio.Condition to determine the count of currently waiting consumers

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Sorry, your use-case looks not convincing but overcomplicated.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

On 26.11.2021 10:56, Ruben Vorderman wrote:
> 
> $ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'')"
> 50 loops, best of 5: 495 nsec per loop

Shouldn't this read:

$ python -m timeit -s "from bytes_sort import bytes_sort" "bytes_sort(b'')"

(-s instead of -c) ?

--
nosy: +lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Tim Peters]

Tim Peters  added the comment:

I agree with closing this - I don't know of real use cases either.

Serhiy, essentially all LSD radix sorts are stable, and rely on that for their 
own correctness. MSD radix sorts vary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Gregory P. Smith]

Gregory P. Smith  added the comment:

General consensus: There isn't a common need for this.

--
nosy: +gregory.p.smith
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I concur with Raymond. It is difficult to find any use case for sorting bytes 
objects (I cannot find any).

As for using radix sort in list.sort() in special case of small integer keys, 
it is difficult to implement, because we should preserve the initial order of 
items with the same key. I am not sure that it is possible to implement stable 
radix sort with linear complexity. In any case the overhead will be more 
significant.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I’m -1 on this. Given that use cases are rare, there is no need to burden the 
code base with an optimization of something we can already do in other ways.

Also, I don’t like that the APIs for list.sort(), bytes.sort(), and 
bytearray.sort() wouldn’t match.  IMO that would do more harm than good.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[STINNER Victor]

STINNER Victor  added the comment:

There are other byte-like objects like memoryview or array.array (for some 
array types). I would suggest writing a function rather than a method.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Eric V. Smith]

Eric V. Smith  added the comment:

Given that the normal sort() machinery wouldn't use this code, I don't think 
there's any advantage to adding .sort() methods to bytes and bytesarray. The 
downside to adding these methods is the increased complexity in the stdlib.

I think the better approach is to put bytes_sort on PyPI.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Mark Dickinson]

Mark Dickinson  added the comment:

> If there are enough use cases for it.

Well, that's the question. :-) *I* can't think of any good use cases, but maybe 
others can. But if we can't come up with some use cases, then this feels like a 
solution looking for a problem, and that makes it hard to justify both the 
short-term effort and the longer-term maintenance costs of adding the 
complexity.

FWIW, given a need to efficiently compute frequency tables for reasonably long 
byte data, I'd probably reach first for NumPy (and numpy.bincount in 
particular):

Python 3.10.0 (default, Nov 12 2021, 12:32:57) [Clang 12.0.5 
(clang-1205.0.22.11)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.28.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import collections, numpy as np

In [2]: t = 
b'MDIAIHHPWIRRPFFPFHSPSRLFDQFFGEHLLESDLFSTATSLSPFYLRPPSFLRAPSWIDTGLSEMRLEKDRFSVNLDVKHFSPEELKVKVLGDVIEVHGKHEERQDEHGFISREFHRKYRI
   ...: PADVDPLAITSSLSSDGVLTVNGPRKQVSGPERTIPITREEKPAVAAAPKK';  t *= 100

In [3]: %timeit np.bincount(np.frombuffer(t, np.uint8))
32.7 µs ± 3.15 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)

In [4]: %timeit collections.Counter(t)
702 µs ± 25.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [5]: %timeit sorted(t)
896 µs ± 64.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

I used it for the median calculation of FASTQ quality scores 
(https://en.wikipedia.org/wiki/FASTQ_format). But in the end I used the 
frequency table to calculate the median more quickly. So as you say, the 
frequency table turned out to be more useful.

Having said that the usefulness depends on how many times 8-bit data is passed 
into sorted. (bytes,bytearrays, most python strings are 8-bit I believe). I 
raised this issue not because I want a .sort() method on bytes or 
bytearrays, but mostly because I think python's sorted function can be improved 
with regards to 8-bit data. I think it is an interesting thing to consider, 
depending on how often this occurs.

For example:
sorted(b'Let\'s test a proper string now. One that has some value to be 
sorted.')
and 
list(bytes_sort(b'Let\'s test a proper string now. One that has some value to 
be sorted.'))

This returns the same result (a list of integers). But the byte_sort 
implementation is 2.5 times faster. So sorted is not optimally implemented here.

Since sorted is now basically throwing everything into list.sort an alternative 
codepath using bytes.sort can be considered. (If there are enough use cases for 
it).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Mark Dickinson]

Mark Dickinson  added the comment:

(Changing the issue type: as I understand it, this is a proposal for a new 
feature, namely new methods bytes.sort and bytearray.sort, rather than a 
performance improvement to an existing feature.)

--
type: performance -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Mark Dickinson]

Mark Dickinson  added the comment:

Can you give example use-cases for sorting a bytes or bytearray object? I see 
value in the intermediate object - the frequency table, but the reconstructed 
sorted bytes object just seems like an inefficient representation of the 
frequency table, and I'm not sure how it would be useful.

As the wikipedia page for counting sort says, the real value is in sorting 
items by keys that are small integers, and the special case where the item is 
identical to the key isn't all that useful:

> In some descriptions of counting sort, the input to be sorted is assumed to 
> be more simply a sequence of integers itself, but this simplification does 
> not accommodate many applications of counting sort.

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

Sorry for the spam. I see I made a typo in the timeit script. Next time I will 
be more dilligent when making these kinds of reports and triple checking it 
before hand, and sending it once. I used -c instead of -s and now all the setup 
time is also included. This confounds the results.

The proper test commands should be:

python -m timeit -s "from bytes_sort import bytes_sort, bytearray_sort_inplace" 
"bytes_sort(b'My string here')"

python -m timeit "bytes(sorted(b'My string here'))"

Using just sorted, to purely compare the sorting algorithms without the 
overhead of creating a new bytes object.
python -m timeit "sorted(b'My string here')"

Correct comparison results
# String = b''
using bytes(sorted: 188 nsec
using sorted:   108 nsec
using byte_sort:125 nsec  # Some overhead here, setting up the countarray
# String = b'abc'
using bytes(sorted: 252 nsec
using sorted:   145 nsec
using byte_sort:136 nsec  # Overhead compared to sorted already negated 
when sorting 3 items(!)
# String = b'Let\'s test a proper string now. One that has some value to be 
sorted.'
using bytes(sorted: 1830 nsec (reported as 1.83 usec)
using sorted:   1550 nsec (reported as 1.55 usec)
using byte_sort: 220 nsec

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

I changed the cython script a bit to use a more naive implementation without 
memset.
Now it is always significantly faster than bytes(sorted(my_bytes)).

$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'')"
50 loops, best of 5: 495 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'abc')"
50 loops, best of 5: 519 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'Let\'s 
test a proper string now. One that has some value to be sorted.')"
50 loops, best of 5: 594 nsec per loop

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Alex Waygood]

Change by Alex Waygood :


--
nosy: +Mark.Shannon, brett.cannon, rhettinger, serhiy.storchaka, tim.peters, 
vstinner, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

Also I didn't know if this should be in Component C-API or Interpreter Core. 
But I guess this will be implemented as C-API calls PyBytes_Sort and 
PyByteArray_SortInplace so I figured C-API is the correct component here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

New submission from Ruben Vorderman :

Python now uses the excellent timsort for most (all?) of its sorting. But this 
is not the fastest sort available for one particular use case.

If the number of possible values in the array is limited, it is possible to 
perform a counting sort: https://en.wikipedia.org/wiki/Counting_sort. In a 
counting sort each value maps to an integer corresponding to its relative 
value. The values are then counted by using key = map_to_key(value); 
count_array[key]. Then from this count_array a new array of sorted values can 
be constructed. (See the wikipedia article for a more detailed explanation).

For the python bytes and bytesarray types this is extremely simple to 
implement. All 256 possible values are can be directly used as keys. 

Rough implementation:
- Use buffer protocol to get pointer to bytes/bytesarray internal c-string
- Declare a count_array: Py_ssize_t[256] count_array . (use memset to set it to 
0)
- Iterate over the c-string and add each value to the countarray. 
count_array[buffer[i]] += 1
- Allocate a new bytes(array) object, or in the case of bytesarray the sorting 
can be performed inplace when bytesarray.sort() is used. 
- Iterate over the count_array. Get the number of values for each key and use 
memset to insert the sequences of keys into position.


The most obvious way to implement this algorithm will be as bytesarray.sort() 
where it is sorted inplace and as bytes.sort() which returns a new sorted bytes 
object. This is much much faster than using bytes(sorted(bytes)).

I made a quick cython implementation for speed testing here: 
https://github.com/rhpvorderman/bytes_sort/blob/main/bytes_sort.pyx

Currently to get a sorted bytestring one has to do bytes(sorted(my_bytes)). 

Test results:

# First make sure there is no regression when sorting an empty string
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes(sorted(b''))"
50 loops, best of 5: 560 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'')"
50 loops, best of 5: 565 nsec per loop

# Test result for very small strings
$ python -m timeit -c "from bytes_sort import bytes_sort" 
"bytes(sorted(b'abc'))"
50 loops, best of 5: 628 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'abc')"
50 loops, best of 5: 578 nsec per loop

# Even on a very small already sorted string, a counting sort is faster.

# Test with a proper string
$ python -m timeit -c "from bytes_sort import bytes_sort" 
"bytes(sorted(b'Let\'s test a proper string now. One that has some value to be 
sorted.'))"
10 loops, best of 5: 2.32 usec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'Let\'s 
test a proper string now. One that has some value to be sorted.')"
50 loops, best of 5: 674 nsec per loop

More than three times faster!

--
components: C API
messages: 407032
nosy: rhpvorderman
priority: normal
severity: normal
status: open
title: Bytes and bytesarrays can be sorted with a much faster count sort.
type: performance
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45902>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18376] show the effective count of process when running the testsuite

[za]

za  added the comment:

The patch seems outdated. Is the ticket still relevant?

--
nosy: +za

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)

[Dong-hee Na]

Dong-hee Na  added the comment:

> There is IBM effort to do this in container level, so that os.cpu_count() 
> will return right result in container

Good news!

--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] On Linux, os.count() should read cgroup cpu.shares and cpu.cfs (CPU count inside docker container)

[Vadym Stupakov]

Vadym Stupakov  added the comment:

> Do we have any news about this?

There is IBM effort to do this in container level, so that os.cpu_count() will 
return right result in container

https://www.phoronix.com/scan.php?page=news_item=Linux-CPU-Namespace

--
nosy: +RedEyed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27637] int.to_bytes(-1, ...) should automatically choose required count of bytes

[Lorenz Panny]

Lorenz Panny  added the comment:

I would like to express my support for making length=None to automatically use 
the minimal possible length. It's true that this will rarely be needed in 
production-grade serialization code, but this functionality is worth its weight 
in gold for quickly written proof-of-concept code or when using Python as a 
"pocket calculator" in an interactive shell.

I'm sure I've personally typed the expression (n.bit_length()+7)//8 
approximately a million times while quickly trying something. It'd be nice if 
Python could just do this simple computation for me instead. The code changes 
required are minimal and there shouldn't be any performance impact.

In fact, in my opinion this should even be the default behaviour, but 3.11 just 
made length=1 the default (see #45155) and changing this now would cause an 
(albeit very mild) API incompatibility.

--
nosy: +lorenz_

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: count consecutive elements

[Bischoop]

I'd wish to have this challenge posted on my blog but because I suck in
IT english, could you guys help me with decribing the task of what the
code supposed to do? 

--
Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue18376] show the effective count of process when running the testsuite

[Irit Katriel]

Change by Irit Katriel :


--
keywords: +easy -patch
versions: +Python 3.11 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45146] Add a possibility for asyncio.Condition to determine the count of currently waiting consumers

[Mykola Mokhnach]

Mykola Mokhnach  added the comment:

This improvement request is also applicable to other asyncio synchronization 
primitives that contain a waiting queue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45146] Add a possibility for asyncio.Condition to determine the count of currently waiting consumers

[Mykola Mokhnach]

New submission from Mykola Mokhnach :

Currently the asyncio.Condition class does not provide any properties that 
would allow to determine how many (if) consumers are waiting for the current 
condition. My scenario is the following:

```
...
FILE_STATS_CONDITIONS: Dict[str, asyncio.Condition] = {}
FILE_STATS_CACHE: Dict[str, Union[FileStat, Exception]] = {}

...

async def _fetch_file_stat(self: T, file_id: str) -> FileStat:
"""
The idea behind this caching is to avoid sending multiple info requests
for the same file id to the server and thus to avoid throttling. This 
is safe, because
stored files are immutable.
"""
try:
file_stat: FileStat
if file_id in FILE_STATS_CONDITIONS:
self.log.debug(f'Reusing the previously cached stat request for 
the file {file_id}')
async with FILE_STATS_CONDITIONS[file_id]:
await FILE_STATS_CONDITIONS[file_id].wait()
cached_result = FILE_STATS_CACHE[file_id]
if isinstance(cached_result, FileStat):
file_stat = cached_result
else:
raise cached_result
else:
FILE_STATS_CONDITIONS[file_id] = asyncio.Condition()
cancellation_exception: Optional[asyncio.CancelledError] = None
async with FILE_STATS_CONDITIONS[file_id]:
file_stat_task = 
asyncio.create_task(self.storage_client.get_file_stat(file_id))
try:
try:
file_stat = await asyncio.shield(file_stat_task)
except asyncio.CancelledError as e:
if len(getattr(FILE_STATS_CONDITIONS[file_id], 
'_waiters', (None,))) == 0:
# it is safe to cancel now if there are no 
consumers in the waiting queue
file_stat_task.cancel()
raise
# we don't want currently waiting consumers to fail 
because of the task cancellation
file_stat = await file_stat_task
cancellation_exception = e
FILE_STATS_CACHE[file_id] = file_stat
except Exception as e:
FILE_STATS_CACHE[file_id] = e
raise
finally:
FILE_STATS_CONDITIONS[file_id].notify_all()
if cancellation_exception is not None:
raise cancellation_exception
# noinspection PyUnboundLocalVariable
self.log.info(f'File stat: {file_stat}')
return file_stat
except ObjectNotFoundError:
self.log.info(f'The file identified by "{file_id}" either does not 
exist or has expired')
raise file_not_found_error(file_id)
finally:
if file_id in FILE_STATS_CONDITIONS and not 
FILE_STATS_CONDITIONS[file_id].locked():
del FILE_STATS_CONDITIONS[file_id]
if file_id in FILE_STATS_CACHE:
del FILE_STATS_CACHE[file_id]
```

Basically I need to use `getattr(FILE_STATS_CONDITIONS[file_id], '_waiters', 
(None,))` to workaround this limitation in order to figure out whether to 
cancel my producer now or later.

It would be nice to have a public property on the Condition class, which would 
basically return the value of `len(condition._waiters)`.

--
components: asyncio
messages: 401434
nosy: asvetlov, mykola-mokhnach, yselivanov
priority: normal
severity: normal
status: open
title: Add a possibility for asyncio.Condition to determine the count of 
currently waiting consumers
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 
<https://bugs.python.org/issue45146>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Tin Tvrtković]

Tin Tvrtković  added the comment:

I was trying to instrument one of our executors to expose some metrics, so I 
tried using the value of the semaphore as the number of idle threads and I 
noticed it was way too large. If this was fixed getting these metrics would be 
easier.

--
nosy: +tinchester

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Antoine Pitrou]

Antoine Pitrou  added the comment:

To be clear, this probably doesn't have any actual consequence, since the right 
number of threads is launched anyway (at least in the example). But it's 
probably worth making the implementation less quirky (also, the semaphore's 
internal counter *might* overflow at some point?).

--
components: +Library (Lib) -asyncio
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Antoine Pitrou]

Change by Antoine Pitrou :


--
versions: +Python 3.11 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35398] SQLite incorrect row count for UPDATE

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

See also bpo-36859.

--
nosy: +erlendaasland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44041] [sqlite3] check that column count is updated correctly for cached statements

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy:  -pablogsal
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44041] [sqlite3] check that column count is updated correctly for cached statements

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 8363ac8607eca7398e568e1336154e1262a995a0 by Erlend Egeberg 
Aasland in branch 'main':
bpo-44041: Add test for sqlite3 column count (GH-25907)
https://github.com/python/cpython/commit/8363ac8607eca7398e568e1336154e1262a995a0


--
nosy: +pablogsal

___
Python tracker 
<https://bugs.python.org/issue44041>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44041] [sqlite3] check that column count is updated correctly for cached statements

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Bennie Swart]

Change by Bennie Swart :


--
keywords: +patch
pull_requests: +24869
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26265

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Bennie Swart]

New submission from Bennie Swart :

The concurrent.futures.ThreadPoolExecutor class, which is the default asyncio 
executor, introduced the _idle_semaphore field in version 3.8 in order to track 
idle threads so they can be reused before increasing the pool size 
unnecessarily.
This semaphore counter becomes unbalanced when the thread pool is 
over-saturated, as can be seen in the file provided. This is due to workers 
always incrementing the count after finishing a job, whereas the executor only 
decrements the count if it is already greater than 0.
This seems to be a logic bug unrelated to the running environment and 
introduced since python 3.8.

--
components: asyncio
files: bug.py
messages: 394017
nosy: asvetlov, bennieswart, yselivanov
priority: normal
severity: normal
status: open
title: ThreadPoolExecutor unbalanced semaphore count
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file50054/bug.py

___
Python tracker 
<https://bugs.python.org/issue44188>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11354] argparse: nargs could accept range of options count

[paul j3]

paul j3  added the comment:

Post parsing testing for the correct number of strings is the easy part.

It's the auto-generate usage that's harder to do right, especially if we wanted 
to enable the metavar tuple option.  Clean usage for '+' is messy enough.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11354] argparse: nargs could accept range of options count

[Alex Kanitz]

Alex Kanitz  added the comment:

Given that people were asking for real-world use cases, here's one: 
high-throughput sequencing, e.g., in RNA-Seq 
(https://en.wikipedia.org/wiki/RNA-Seq), typically yields either one or two 
output files, depending on the type of the sequencing library. As the 
processing of these library types is very similar, bioinformatics tools dealing 
with these inputs thus typically have APIs that take exactly 1 or 2 files as 
inputs. As there is quite a wide range of tools for dealing with such inputs, 
several of which implemented in Python, I believe there is indeed an interest 
in this functionality. On a more conceptual note: it is also consistent with 
similar and often-used functionality in regexes, from which, I suppose, the '+' 
and '*' notation is borrowed.

Currently implementing such a tool, I ran into the argparse limitation 
described here: either I (a) use a positional param with nargs=1 for the first 
file and define an optional param for the second file (inconsistent, 
non-intuitive and semantically incorrect API, because if there IS a second 
file, it is not really optional), (b) use nargs='+', do the admittedly simple 
post-processing/validation and either ignore keep the auto-generated usage 
string(wrong/misleading), hardcode the correct usage string (maintenance burden 
because of several optional params) or apply this patch (or just the auto-usage 
generation function), which seems rather expensive, or (c) have the user pass 
the second file in one string, separated by a comma or similar (also not very 
intuitive and needs some checks to ensure that the filename/s don't actually 
include commas).

--
nosy: +uniqueg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Python cannot count apparently

[Michael F. Stemper]

On 08/02/2021 13.12, Schachner, Joseph wrote:

This code works:
 mystr = "hello"
 for ch in mystr:
 print(ch, end="")

result is: hello

Note that the for loop does not use range.  Strings are iterable, that is they 
support Python's iteration protocol.  So, for ch in mystr: assigns one 
character from mystr to ch each time, each iteration gets the next character.
To prevent each character from appearing on a separate line (in Python 3) you need 
end="".   That is, don't put in the usual end-of-line ending.


Then you agree that the OP:


You must have done something to suppress the newlines after each call to 
print().

So it's quite obvious that the code you posted has very little to do with the 
code you ran. If you really want us to help, please directly copy and paste the 
exact code that you ran. We can't really help you with only guesses as to what 
you did.



--
Michael F. Stemper
Outside of a dog, a book is man's best friend.
Inside of a dog, it's too dark to read.
--
https://mail.python.org/mailman/listinfo/python-list

RE: Python cannot count apparently

[Schachner, Joseph]

This code works:
mystr = "hello"
for ch in mystr:
print(ch, end="")

result is: hello

Note that the for loop does not use range.  Strings are iterable, that is they 
support Python's iteration protocol.  So, for ch in mystr: assigns one 
character from mystr to ch each time, each iteration gets the next character.  
To prevent each character from appearing on a separate line (in Python 3) you 
need end="".   That is, don't put in the usual end-of-line ending.

--- Joseph S.

Teledyne Confidential; Commercially Sensitive Business Data

-Original Message-
From: Michael F. Stemper  
Sent: Monday, February 8, 2021 9:19 AM
To: python-list@python.org
Subject: Re: Python cannot count apparently

On 07/02/2021 13.34, Philipp Daher wrote:
> Hello,
> 
> I recently coded this snippet of code:
> myString=„hello“
> for i in range(len(myString):
>   print(string[i])
> 
> And now for the weird part:
> 
> SOMETIMES, the output is this:
> 
> hello

Strange. When I fix the errors in what you posted:
- wrong character to start the string
- wrong variable name in the call to print()

I get[1]:


... myString="hello"
... for i in range(len(myString)):
...   print( myString[i] )
...
h
e
l
l
o
...

You must have done something to suppress the newlines after each call to 
print().

So it's quite obvious that the code you posted has very little to do with the 
code you ran. If you really want us to help, please directly copy and paste the 
exact code that you ran. We can't really help you with only guesses as to what 
you did.



[1] (leading > replaced with . to fool news clients)
--
Michael F. Stemper
Galatians 3:28
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[Michael F. Stemper]

On 07/02/2021 13.34, Philipp Daher wrote:

Hello,

I recently coded this snippet of code:
myString=„hello“
for i in range(len(myString):
  print(string[i])

And now for the weird part:

SOMETIMES, the output is this:

hello


Strange. When I fix the errors in what you posted:
- wrong character to start the string
- wrong variable name in the call to print()

I get[1]:


... myString="hello"
... for i in range(len(myString)):
...   print( myString[i] )
...
h
e
l
l
o
...

You must have done something to suppress the newlines after
each call to print().

So it's quite obvious that the code you posted has very little
to do with the code you ran. If you really want us to help, please
directly copy and paste the exact code that you ran. We can't
really help you with only guesses as to what you did.



[1] (leading > replaced with . to fool news clients)
--
Michael F. Stemper
Galatians 3:28
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[Peter Pearson]

On Sun, 7 Feb 2021 20:49:52 + (UTC), Kevin M. Wilson wrote:

> Set i = 0 at the begin of the code, that way each entry starts at
> Logical 0 of the array/container/list...

No.  The original code, as posted, was

>>I recently coded this snippet of code:
>>myString=„hello“
>>for i in range(len(myString):
>> print(string[i])

Setting i=0 before the "for" statement would make no difference.
In the first pass through the for loop, i is set to the first
value in the range, which is zero.



-- 
To email me, substitute nowhere->runbox, invalid->com.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[dn via Python-list]

On 08/02/2021 09.49, Kevin M. Wilson via Python-list wrote:
> Set i = 0 at the begin of the code, that way each entry starts at Logical 0 
> of the array/container/list...


FYI: https://docs.python.org/3/library/stdtypes.html#typesseq-range

See also @Chris' contribution regarding the pythonic idiom for iterating
over a container.
-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[Kevin M. Wilson via Python-list]

Set i = 0 at the begin of the code, that way each entry starts at Logical 0 of 
the array/container/list...
"The only way to have experience is by having the experience"!
 

On Sunday, February 7, 2021, 12:56:40 PM MST, Karsten Hilbert 
 wrote:  
 
 Am Sun, Feb 07, 2021 at 07:47:03PM + schrieb Paul Bryan:

> That's not the only problem with the code. There's a missing close-
> paren and a reference to "string" which I presume was meant to be
> "myString".

I know. I wasn't going to spoil everything right away. The
sort of response we would get from OP would tell us what sort
of help might be most suitable :-)

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list
  
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[Karsten Hilbert]

Am Sun, Feb 07, 2021 at 07:47:03PM + schrieb Paul Bryan:

> That's not the only problem with the code. There's a missing close-
> paren and a reference to "string" which I presume was meant to be
> "myString".

I know. I wasn't going to spoil everything right away. The
sort of response we would get from OP would tell us what sort
of help might be most suitable :-)

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[Paul Bryan]

That's not the only problem with the code. There's a missing close-
paren and a reference to "string" which I presume was meant to be
"myString".

Suggest OP create a reproducible case, and paste the code and output
verbatim.


On Sun, 2021-02-07 at 20:40 +0100, Karsten Hilbert wrote:
> Am Sun, Feb 07, 2021 at 08:34:34PM +0100 schrieb Philipp Daher via
> Python-list:
> 
> > I recently coded this snippet of code:
> > myString=„hello“
> 
> I doubt you have (coded *this* snippet of code) -- because
> those quotes wouldn't work.
> 
> Karsten
> --
> GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[Joel Goldstick]

On Sun, Feb 7, 2021 at 2:36 PM Philipp Daher via Python-list
 wrote:
>
> Hello,
>
> I recently coded this snippet of code:
> myString=„hello“
> for i in range(len(myString):
>  print(string[i])
>
> And now for the weird part:
>
> SOMETIMES, the output is this:
>
> hello
>
> And SOMETIMES, the output changes to:
>
> ohell
>
> WHY??? Why do I get different outputs with the EXACT SAME CODE?
>
> Can someone help me please? Thank you
> --
> https://mail.python.org/mailman/listinfo/python-list

Did you type the code using idle, or the default python interactive
console?  If you did, please do that again, and copy and paste your
code and the results here.  There is something amiss.

-- 
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[Chris Angelico]

On Mon, Feb 8, 2021 at 6:36 AM Philipp Daher via Python-list
 wrote:
>
> Hello,
>
> I recently coded this snippet of code:
> myString=„hello“
> for i in range(len(myString):
>  print(string[i])

This code won't work as is. Please *copy and paste* your code when
asking for help.

> And now for the weird part:
>
> SOMETIMES, the output is this:
>
> hello
>
> And SOMETIMES, the output changes to:
>
> ohell

Neither of those makes sense based on the code you're showing, so
again, please *copy and paste* the code and output.

> WHY??? Why do I get different outputs with the EXACT SAME CODE?
>
> Can someone help me please? Thank you

I would recommend that you start by iterating over the string itself,
instead of a range. My crystal ball tells me that you're getting
tangled with negative indexing, but without seeing the code each time,
it's impossible to be sure.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python cannot count apparently

[Karsten Hilbert]

Am Sun, Feb 07, 2021 at 08:34:34PM +0100 schrieb Philipp Daher via Python-list:

> I recently coded this snippet of code:
> myString=„hello“

I doubt you have (coded *this* snippet of code) -- because
those quotes wouldn't work.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

Python cannot count apparently

[Philipp Daher via Python-list]

Hello,

I recently coded this snippet of code:
myString=„hello“
for i in range(len(myString):
 print(string[i])

And now for the weird part:

SOMETIMES, the output is this:

hello

And SOMETIMES, the output changes to:

ohell

WHY??? Why do I get different outputs with the EXACT SAME CODE?

Can someone help me please? Thank you
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Peter Otten]

On 19/01/2021 10:42, Bischoop wrote:

On 2021-01-19, Peter Otten <__pete...@web.de> wrote:

On 19/01/2021 04:45, Bischoop wrote:


I sat to it again and solved it.


Congratulations!


lil = tuple(set(s)) # list of characters in s

li=[0,0,0,0,0,0]  # list for counted repeats


I see a minor problem here. What happens if s contains more than len(li)
characters?



Thanks, I've created it when came to idea how to solve the problem and
then forgot that part, will have to update it with two additional line
of code.


import timeit


Since you seem interested in performance: imagine that the input is a
very long string. Let's call the length N.

How often do you iterate over its characters?



Does it take time :-) I actually left it because seen guy in thread
were comparing their time results, I'm pretty much aware that mine
solution is time consuming and there are better ways to do it but
I wanted to finish what I started without any additional libraries
like itertools etc in the way that my knowledge allows. The idea with
for this tuple just came up in a first seconds when I look at that
and was not bothering about time or performance especially that later
on as you seen I had different problem to solve and which took me quite
a time and when I look at it today I think how couldn't I came with it
earlier and let myself stuck on it.

I'm happy that despite I've asked here I've finish it practically wthout
additional help and yeah ( Thanks to Stefan who just pointed me to look
at it from different prespective instead pointing what was wrong),
I'll iterate here n = x [x for x in lil], with big string it can make
difference. Now, when you asked me that question I came indeed for better
idea to do this. One loop during which I can append character if not in lil.


How often does Tim's solution?


oh here Im stuck and dont understand what you mean?


[Tim Chase]


  def consecutive_counter(seq):
  # I'm not sure if there's something
  # like this already in itertools
  cur = nada = object()
  count = 0
  for x in seq:
  if x == cur:
  count += 1
  else:
  if cur is not nada:
  yield cur, count
  cur = x
  count = 1
  if cur is not nada:
  yield cur, count

  def longest(seq):
  results = []
  biggest = 0
  for item, count in seq:
      if count > biggest:
  results = [item]
  biggest = count
  elif count == biggest:
  results.append(item)
  return results, biggest



Tim's code iterates over the string once whereas you iterate multiple
times. While this is mainly a question of elegance performance is
affected when you have *nested* loops:


for i in lil:
c = 0
h= lil.index(i)
for letter in s:


If you are unlucky and run into a string with 1000 different characters
the code in your inner loop will execute 1000*1000 times while Tim's
will run 1000 times. This is called O(N*N) and O(N).

Not all loops are obvious,


lil.index(letter)


contains another loop, and you now have O(N*N*N) behavior.

This is the worst case, usually the alphabet (li1) will be smaller, but
avoiding nested loops when possible is still a good habit to get into.

--
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Bischoop]

On 2021-01-19, Peter Otten <__pete...@web.de> wrote:
> On 19/01/2021 04:45, Bischoop wrote:
>
>> I sat to it again and solved it.
>
> Congratulations!
>
> > lil = tuple(set(s)) # list of characters in s
> >
> > li=[0,0,0,0,0,0]  # list for counted repeats
>
> I see a minor problem here. What happens if s contains more than len(li) 
> characters?
>

Thanks, I've created it when came to idea how to solve the problem and
then forgot that part, will have to update it with two additional line
of code.

>> import timeit
>
> Since you seem interested in performance: imagine that the input is a 
> very long string. Let's call the length N.
>
> How often do you iterate over its characters?
>

Does it take time :-) I actually left it because seen guy in thread
were comparing their time results, I'm pretty much aware that mine
solution is time consuming and there are better ways to do it but
I wanted to finish what I started without any additional libraries
like itertools etc in the way that my knowledge allows. The idea with
for this tuple just came up in a first seconds when I look at that
and was not bothering about time or performance especially that later
on as you seen I had different problem to solve and which took me quite
a time and when I look at it today I think how couldn't I came with it
earlier and let myself stuck on it.

I'm happy that despite I've asked here I've finish it practically wthout
additional help and yeah ( Thanks to Stefan who just pointed me to look
at it from different prespective instead pointing what was wrong),
I'll iterate here n = x [x for x in lil], with big string it can make 
difference. Now, when you asked me that question I came indeed for better
idea to do this. One loop during which I can append character if not in lil.

> How often does Tim's solution?

oh here Im stuck and dont understand what you mean?

--
Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Peter Otten]

On 19/01/2021 04:45, Bischoop wrote:


I sat to it again and solved it.


Congratulations!

> lil = tuple(set(s)) # list of characters in s
>
> li=[0,0,0,0,0,0]  # list for counted repeats

I see a minor problem here. What happens if s contains more than len(li)
characters?


import timeit


Since you seem interested in performance: imagine that the input is a
very long string. Let's call the length N.

How often do you iterate over its characters?
How often does Tim's solution?
--
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Bischoop]

On 2021-01-14, Stefan Ram  wrote:
>
>   If you want to know why, maybe you should insert print
>   statements to see the values of critical variables and
>   expression, especially in loops. 
>
>   Then compare the printed values with your expectations.
>
>   Also, decompose into meaningful functions and test each
>   function separately.
>
>

I sat to it again and solved it.

import timeit

run1 = '''

s = 'aabskbad'


lil = tuple(set(s)) # list of characters in s

li=[0,0,0,0,0,0]  # list for counted repeats


for i in lil:
c = 0
h= lil.index(i)
for letter in s:
if letter == i:
c += 1
if c > li[lil.index(letter)]:
li[lil.index(letter)] = c
else:
c=0
continue

m = max(li)

for index, j in enumerate(li):
if li[index] == m:
print(f'{lil[index]} appears {m} consecutive times')


'''
print(timeit.timeit(stmt=run1, number=1))



output:
c appears 4 consecutive times
a appears 4 consecutive times
0.00013008200039621443


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Peter Otten]

On 17/01/2021 02:15, Dan Stromberg wrote:

IMO a good set of tests is much more important than type annotations ;)


   def get_longest(string: str) -> typing.Tuple[int, typing.List[str]]:
   """Get the longest run of a single consecutive character."""


May I ask why you artificially limit the signature to str?
If there were a signature, shouldn't it be something like

Iterable[E] --> Tuple[int, List[E]]

where E may be anything that has an equality operation?
As someone who has not yet looked seriously into typing: is there a way
to spell this?


   if not string:
   return (0, [])
   grouped = itertools.groupby(string)
   grouped_with_lengths = [(len(list(value)), key) for key, value in
grouped]
   max_count_and_letter = max(grouped_with_lengths)
   max_count = max_count_and_letter[0]
   result = (max_count, sorted(list_ for count, list_ in
grouped_with_lengths if count == max_count))
   return result


If you want to dabble some more here's a test case you might strive for
your function to pass:

"""
>>> class A:
... def __init__(self, x):
... self.x = x
... def __eq__(self, other):
... return self.x == other.x
... def __repr__(self):
... return self.x

>>> get_longest(map(A, "aaabbcccdaa"))
(3, [a, c])

"""
--
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Dan Stromberg]

On Thu, Jan 14, 2021 at 2:01 PM Wolfram Hinderer via Python-list <
python-list@python.org> wrote:

> Am 13.01.2021 um 22:20 schrieb Bischoop:
> > I want to  to display a number or an alphabet which appears mostly
> > consecutive in a given string or numbers or both
> > Examples
> > s= ' aabskaaabad'
> > output: c
> > # c appears 4 consecutive times
> >   8bbakebaoa
> > output: b
> > #b appears 2 consecutive times
> >
> >
> You can let itertools.groupy find the groups.
>
> max((len(tuple(group)), key) for key, group in itertools.groupby(s))
> # (4, 'c')
>

Does anyone else find the documentation on itertools.groupby kind of
lacking?

I think it makes sense now that I've played around with it though.

Here's a revised solution:
  def get_longest(string: str) -> typing.Tuple[int, typing.List[str]]:
  """Get the longest run of a single consecutive character."""
  if not string:
  return (0, [])
  grouped = itertools.groupby(string)
  grouped_with_lengths = [(len(list(value)), key) for key, value in
grouped]
  max_count_and_letter = max(grouped_with_lengths)
      max_count = max_count_and_letter[0]
  result = (max_count, sorted(list_ for count, list_ in
grouped_with_lengths if count == max_count))
  return result
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Tim Chase]

On 2021-01-16 03:32, Bischoop wrote:
>> The OP didn't specify what should happen in that case, so it would
>> need some clarification.
> 
> In that case maybe good solution would be to return three of them?

That's the solution I chose in my initial reply, you get a tuple back
of ([list of longest matches], length_of_longest_match)

-tkc


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Bischoop]

On 2021-01-14, Tim Chase  wrote:
>
> seems to only return one value so seems to get odd results if I
> specify something like
>
>   get_longest("aaabcccbbb")
>
> which at least here tells me that "c" is the longest run, even though
> aaa, bbb, and ccc are all runs of length 3.  The OP didn't specify
> what should happen in that case, so it would need some clarification.
>
>

In that case maybe good solution would be to return three of them?

--
Thanks

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Wolfram Hinderer via Python-list]

Am 13.01.2021 um 22:20 schrieb Bischoop:

I want to  to display a number or an alphabet which appears mostly
consecutive in a given string or numbers or both
Examples
s= ' aabskaaabad'
output: c
# c appears 4 consecutive times
  8bbakebaoa
output: b
#b appears 2 consecutive times



You can let itertools.groupy find the groups.

max((len(tuple(group)), key) for key, group in itertools.groupby(s))
# (4, 'c')
--
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Bischoop]

On 2021-01-13, Bischoop  wrote:

I know what was wrong:

> m = s.index(i)

I forgot that m will return first index of i.

So tried that way but still getting out of index although I that that
I'm making sure not to get out of index.

s = 'aabskaaabadh'

c = 0
t = list(set(s)) # list of characters in s
li=[0,0,0,0,0,0]  # list for counted repeats
print(t)
for x in t:
h = t.index(x)
for index, i in enumerate(s):
maximus = len(s)
if i == x:
c += 1
if index < maximus:
if s[index +1] != x:  # if next element is not x
if c > li[h]:   #update c if bigger than existing
li[h] = c
c = 0
else:
if c > li[h]:
li[h] = c


for i in t:
n = t.index(i)
print(i,li[n])

print(f'{s[li.index(max(li))]} appears {max(li)} consecutive times')


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Dan Stromberg]

On Wed, Jan 13, 2021 at 6:20 PM Dan Stromberg  wrote:

> On Wed, Jan 13, 2021 at 5:59 PM Tim Chase 
> wrote:
>
>> On 2021-01-13 21:20, Bischoop wrote:
>> > I want to  to display a number or an alphabet which appears mostly
>> > consecutive in a given string or numbers or both
>> > Examples
>> > s= ' aabskaaabad'
>> > output: c
>> > # c appears 4 consecutive times
>> >  8bbakebaoa
>> > output: b
>> > #b appears 2 consecutive times
>>
>
> I'm kind of partial to:
>
> import collections
> import typing
>
>
> def get_longest(string: str) -> typing.Tuple[int, str]:
>
I just realized this has a kinda big bug. Don't use!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Tim Chase]

On 2021-01-13 18:20, Dan Stromberg wrote:
> I'm kind of partial to:
> 
> import collections
> import typing
> 
> 
> def get_longest(string: str) -> typing.Tuple[int, str]:
> """Get the longest run of a single consecutive character."""
> dict_: typing.DefaultDict[str, int] =
> collections.defaultdict(int) for left_ch, right_ch in zip(string,
> string[1:]): if left_ch == right_ch:
> dict_[left_ch] += 1
> 
> maximum = max((value, key) for key, value in dict_.items())
> 
> return maximum

seems to only return one value so seems to get odd results if I
specify something like

  get_longest("aaabcccbbb")

which at least here tells me that "c" is the longest run, even though
aaa, bbb, and ccc are all runs of length 3.  The OP didn't specify
what should happen in that case, so it would need some clarification.

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Dan Stromberg]

On Wed, Jan 13, 2021 at 5:59 PM Tim Chase 
wrote:

> On 2021-01-13 21:20, Bischoop wrote:
> > I want to  to display a number or an alphabet which appears mostly
> > consecutive in a given string or numbers or both
> > Examples
> > s= ' aabskaaabad'
> > output: c
> > # c appears 4 consecutive times
> >  8bbakebaoa
> > output: b
> > #b appears 2 consecutive times
>

I'm kind of partial to:

import collections
import typing


def get_longest(string: str) -> typing.Tuple[int, str]:
"""Get the longest run of a single consecutive character."""
dict_: typing.DefaultDict[str, int] = collections.defaultdict(int)
for left_ch, right_ch in zip(string, string[1:]):
if left_ch == right_ch:
dict_[left_ch] += 1

maximum = max((value, key) for key, value in dict_.items())

return maximum

HTH
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Tim Chase]

On 2021-01-13 21:20, Bischoop wrote:
> I want to  to display a number or an alphabet which appears mostly
> consecutive in a given string or numbers or both
> Examples
> s= ' aabskaaabad'
> output: c
> # c appears 4 consecutive times
>  8bbakebaoa
> output: b
> #b appears 2 consecutive times

I'd break the problem into two parts:

1) iterate over your thing (in this case, a string) and emit the item
and its correpsonding count of consecutive matches.

2) feed that into something that finds the longest run(s) output by
that.

So off the cuff, something like

  def consecutive_counter(seq):
  # I'm not sure if there's something
  # like this already in itertools
  cur = nada = object()
  count = 0
  for x in seq:
  if x == cur:
  count += 1
  else:
  if cur is not nada:
      yield cur, count
  cur = x
  count = 1
  if cur is not nada:
  yield cur, count

  def longest(seq):
  results = []
  biggest = 0
  for item, count in seq:
  if count > biggest:
  results = [item]
  biggest = count
  elif count == biggest:
  results.append(item)
  return results, biggest

  for s in (
  "",
  "a",
  "aaa",
  "aaabbb",
  "aabskaaabad",
  "aabskaaakbad",
  ):
  print("Testing %r" % s)
  print(repr(longest(consecutive_counter(s

-tkc




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Bischoop]

On 2021-01-13, Bischoop  wrote:
> t = set(s) # set of characters in s

I have this one changed to:
t= list(set(s))
-- 
https://mail.python.org/mailman/listinfo/python-list

count consecutive elements

[Bischoop]

I want to  to display a number or an alphabet which appears mostly
consecutive in a given string or numbers or both
Examples
s= ' aabskaaabad'
output: c
# c appears 4 consecutive times
 8bbakebaoa
output: b
#b appears 2 consecutive times

I thought about set the string then and for each element loop the string
to check if equal with element set element if so count =+1 and check if
next to it is not equal add counter value to list with same index as in
set.
However I'm fighting to code it and the counting numbers are not right:


s = 'aabskaaabad'

c = 0
t = set(s) # set of characters in s
li=[0,0,0,0,0,0]  # list for counted repeats

for x in t:
h = t.index(x)
for i in s:
if i == x:
m = s.index(i)
c +=1
if s[m+1] != x:  # if next element is not x
if c > li[h]: 
li[h]= c
c = 0

for i in t:
n = t.index(i)
print(i,li[n])

--
Thanks
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue40949] test_socket altered the execution environment: threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

[STINNER Victor]

STINNER Victor  added the comment:

I didn't see the failure recently, I close the issue.

--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41739] test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41739] test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)

[STINNER Victor]

STINNER Victor  added the comment:

I created bpo-41833 "threading.Thread: use target name if the name parameter is 
omitted" to help me to debug this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41739] test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)

[STINNER Victor]

STINNER Victor  added the comment:

Previous issues:

* bpo-30131: "test_logging now joins queue threads"
* bpo-30830: test_logging uses threading_setup/cleanup, logging.config.listen() 
calls server_close(), socketserver.ThreadingMixIn.server_close()
* bpo-31235: Fix ResourceWarning in test_logging

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41739] test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)

[STINNER Victor]

STINNER Victor  added the comment:

See also https://pythondev.readthedocs.io/unstable_tests.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41739] test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)

[STINNER Victor]

STINNER Victor  added the comment:

s390x Debian 3.x:
https://buildbot.python.org/all/#/builders/49/builds/75

Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 
2)
Warning -- Dangling thread: <_MainThread(MainThread, started 4396138073968)>
Warning -- Dangling thread: 

--

___
Python tracker 
<https://bugs.python.org/issue41739>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39451] enum.Enum reference count leaks

[Ethan Furman]

Ethan Furman  added the comment:

Thanks, Dan, for checking this out.  As you noted, a garbage collection solved 
the issue, so I'm not going to worry about it.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41739] test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41739] test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)

[STINNER Victor]

STINNER Victor  added the comment:

Same isuse on s390x RHEL7 LTO + PGO 3.x:
https://buildbot.python.org/all/#/builders/244/builds/45

Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 
2)
Warning -- Dangling thread: <_MainThread(MainThread, started 4396860012288)>
Warning -- Dangling thread: 

--

___
Python tracker 
<https://bugs.python.org/issue41739>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41739] test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)

[STINNER Victor]

New submission from STINNER Victor :

s390x RHEL8 LTO + PGO 3.x:
https://buildbot.python.org/all/#/builders/442/builds/399

0:01:43 load avg: 4.23 [229/423/1] test_logging failed (env changed) -- 
running: test_concurrent_futures (48.1 sec)
Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 
2)
Warning -- Dangling thread: 
Warning -- Dangling thread: <_MainThread(MainThread, started 4395867015984)>

--
components: Tests
messages: 376509
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_logging: threading_cleanup() failed to cleanup 1 threads (count: 1, 
dangling: 2)
versions: Python 3.10

___
Python tracker 
<https://bugs.python.org/issue41739>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40949] test_socket altered the execution environment: threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40949] test_socket altered the execution environment: threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

[STINNER Victor]

STINNER Victor  added the comment:

I tried various commands to reproduce the "test_socket (env changed)" but I 
failed to find a reliable way to trigger this bug. I ran test.bisect_cmd for 
3h, with other commands run in parallel to stress the machine (system load 
between 8 and 30).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40949] test_socket altered the execution environment: threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

[STINNER Victor]

STINNER Victor  added the comment:

Oh, by design, NetworkConnectionNoServer.test_create_connection() has a race 
condition:

port = socket_helper.find_unused_port()
with self.assertRaises(OSError) as cm:
socket.create_connection((HOST, port))

If another process starts to listen to the "unused" port between 
find_unused_port() call and the create_connection() call, create_connection() 
succeed and the test fails.

Maybe I should just add a comment mentioning the race condition. Or we should 
find a more reliable way to get an error.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40949] test_socket altered the execution environment: threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

[STINNER Victor]

STINNER Victor  added the comment:

Another error while running "./python -m test test_socket --fail-env-changed 
-j20 -r -F -w":

NetworkConnectionNoServer.test_create_connection() fails with "AssertionError: 
OSError not raised".

0:01:13 load avg: 69.29 [ 12/1] test_socket failed (1 min 9 sec) -- running: 
test_socket (...)

/home/vstinner/cpython/Lib/test/test_socket.py:5120: ResourceWarning: unclosed 

  socket.create_connection((HOST, port))
ResourceWarning: Enable tracemalloc to get the object allocation traceback

test test_socket failed -- Traceback (most recent call last):
  File "/home/vstinner/cpython/Lib/test/test_socket.py", line 5120, in 
test_create_connection
socket.create_connection((HOST, port))
AssertionError: OSError not raised

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40949] test_socket altered the execution environment: threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

[STINNER Victor]

STINNER Victor  added the comment:

More logs before testSourceAddress failure:

(...)
testSmallReadNonBlocking (test.test_socket.UnbufferedFileObjectClassTestCase) 
... ok
testUnbufferedRead (test.test_socket.UnbufferedFileObjectClassTestCase) ... ok
testWriteNonBlocking (test.test_socket.UnbufferedFileObjectClassTestCase) ... ok
testAttributes (test.test_socket.LineBufferedFileObjectClassTestCase) ... ok
testMakefileAfterMakefileClose 
(test.test_socket.LineBufferedFileObjectClassTestCase) ... ok
testReadAfterTimeout (test.test_socket.LineBufferedFileObjectClassTestCase) ... 
ok
testCloseAfterMakefile (test.test_socket.SmallBufferedFileObjectClassTestCase) 
... ok
testMakefileAfterMakefileClose 
(test.test_socket.SmallBufferedFileObjectClassTestCase) ... ok
testUnbufferedRead (test.test_socket.SmallBufferedFileObjectClassTestCase) ... 
ok
testAttributes (test.test_socket.UnicodeReadFileObjectClassTestCase) ... ok
testClosedAttr (test.test_socket.UnicodeReadFileObjectClassTestCase) ... ok
testMakefileAfterMakefileClose 
(test.test_socket.UnicodeReadFileObjectClassTestCase) ... ok
testReadAfterTimeout (test.test_socket.UnicodeReadFileObjectClassTestCase) ... 
ok
testReadline (test.test_socket.UnicodeReadFileObjectClassTestCase) ... ok
testCloseAfterMakefile (test.test_socket.UnicodeWriteFileObjectClassTestCase) 
... ok
testClosedAttr (test.test_socket.UnicodeWriteFileObjectClassTestCase) ... ok
testFullRead (test.test_socket.UnicodeWriteFileObjectClassTestCase) ... ok
testMakefileAfterMakefileClose 
(test.test_socket.UnicodeWriteFileObjectClassTestCase) ... ok
testReadAfterTimeout (test.test_socket.UnicodeWriteFileObjectClassTestCase) ... 
ok
testRealClose (test.test_socket.UnicodeWriteFileObjectClassTestCase) ... ok
testFullRead (test.test_socket.UnicodeReadWriteFileObjectClassTestCase) ... ok
testMakefileAfterMakefileClose 
(test.test_socket.UnicodeReadWriteFileObjectClassTestCase) ... ok
testReadAfterTimeout (test.test_socket.UnicodeReadWriteFileObjectClassTestCase) 
... ok
testReadline (test.test_socket.UnicodeReadWriteFileObjectClassTestCase) ... ok
testSmallRead (test.test_socket.UnicodeReadWriteFileObjectClassTestCase) ... ok
testUnbufferedRead (test.test_socket.UnicodeReadWriteFileObjectClassTestCase) 
... ok
test_create_connection (test.test_socket.NetworkConnectionNoServer) ... ok
test_create_connection_timeout (test.test_socket.NetworkConnectionNoServer) ... 
ok
testSourceAddress (test.test_socket.NetworkConnectionAttributesTest) ... 
Warning -- Unraisable exception
Exception ignored in thread started by: >
Traceback (most recent call last):
  File "/home/vstinner/cpython/Lib/test/test_socket.py", line 396, in clientRun
self.clientTearDown()
  File "/home/vstinner/cpython/Lib/test/test_socket.py", line 5164, in 
clientTearDown
self.cli.close()
AttributeError: 'NetworkConnectionAttributesTest' object has no attribute 'cli'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40949] test_socket altered the execution environment: threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

[STINNER Victor]

STINNER Victor  added the comment:

While running "./python -m test.bisect_cmd test_socket --fail-env-changed -v -o 
bisect" 6x in parallel (with a different filename for the -o option), I got 
this error:

testSourceAddress (test.test_socket.NetworkConnectionAttributesTest) ... 
Warning -- Unraisable exception
Exception ignored in thread started by: >
Traceback (most recent call last):
  File "/home/vstinner/cpython/Lib/test/test_socket.py", line 396, in clientRun
self.clientTearDown()
  File "/home/vstinner/cpython/Lib/test/test_socket.py", line 5164, in 
clientTearDown
self.cli.close()
AttributeError: 'NetworkConnectionAttributesTest' object has no attribute 'cli'


By the way, I was also running "./python -m test -j20 -r -F" in parallel and it 
failed with the following error which isn't very helpful :-(

test test_socket failed -- multiple errors occurred; run in verbose mode for 
details

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40949] test_socket altered the execution environment: threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

[STINNER Victor]

STINNER Victor  added the comment:

PPC64LE RHEL7 Refleaks 3.9:
https://buildbot.python.org/all/#/builders/704/builds/87

0:12:25 load avg: 8.94 [242/425/1] test_socket failed (env changed) (2 min 43 
sec) -- running: test_decimal (2 min 26 sec), test_peg_generator (12 min 23 
sec), test_signal (6 min 20 sec), test_weakref (2 min 41 sec), 
test_multiprocessing_fork (2 min 36 sec), test_venv (2 min 33 sec)
beginning 6 repetitions
123456
.Warning -- threading_cleanup() failed to cleanup 0 threads (count: 0, 
dangling: 5)
Warning -- Dangling thread: 
Warning -- Dangling thread: 
Warning -- Dangling thread: 
Warning -- Dangling thread: <_MainThread(MainThread, started 70367377380144)>
Warning -- Dangling thread: 
.

--
title: test_socket: threading_cleanup() failed to cleanup 0 threads (count: 0, 
dangling: 5) -> test_socket altered the execution environment: 
threading_cleanup() failed to cleanup 0 threads (count: 0, dangling: 5)

___
Python tracker 
<https://bugs.python.org/issue40949>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[STINNER Victor]

STINNER Victor  added the comment:

> For time.time in Windows, Python 3.10 should switch to using 
> GetSystemTimePreciseAsFileTime [3] instead of GetSystemTimeAsFileTime. 

It's already tracked by bpo-19007: "precise time.time() under Windows 8: use 
GetSystemTimePreciseAsFileTime".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Eryk Sun]

Eryk Sun  added the comment:

> Does the same apply for time.monotonic? I would argue that the 
> difference in behavior between Linux/macOS and Windows is 
> unreasonable; given that time.monotonic exists for measuring time 
> intervals

For time.monotonic in Windows, Python could switch to using 
QueryUnbiasedInterruptTime [1], which excludes periods in which the system is 
suspended or hibernated. In Windows 10, QueryUnbiasedInterruptTimePrecise [2] 
can be used, which directly reads the counter in the hardware timer instead of 
the value at the last timer interrupt.

For time.time in Windows, Python 3.10 should switch to using 
GetSystemTimePreciseAsFileTime [3] instead of GetSystemTimeAsFileTime. 

For time.perf_counter in Windows, QueryPerformanceCounter remains the best 
option since it uses the invariant TSC in the CPU if available.

---

Note that, unlike GetTickCount64, the resolution for QueryUnbiasedInterruptTime 
isn't lpTimeIncrement from GetSystemTimeAdjustment [4]. lpTimeIncrement is the 
initial default and upper bound for the interrupt period, which can be adjusted 
down with timeBeginPeriod or the undocumented system call NtSetTimerResolution. 
The former can lower the interrupt period from the default of about 10-16 ms 
(usually 15.625 ms, i.e. 64 interrupts/second) down to about 1 ms, while 
NtSetTimerResolution goes down to about 0.5 ms. It's not free, however, else it 
would be the default setting. More frequent timer interrupts can decrease 
system performance and increase power consumption.

---

[1] 
https://docs.microsoft.com/en-us/windows/win32/api/realtimeapiset/nf-realtimeapiset-queryunbiasedinterrupttime

[2] 
https://docs.microsoft.com/en-us/windows/win32/api/realtimeapiset/nf-realtimeapiset-queryunbiasedinterrupttimeprecise

[3] 
https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getsystemtimepreciseasfiletime

[4] 
https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getsystemtimeadjustment

--
nosy: +eryksun
versions: +Python 3.10, Python 3.9 -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Rishav Kundu]

Rishav Kundu  added the comment:

> perf_counter behavior during system suspend is undefined

Does the same apply for time.monotonic? I would argue that the difference in 
behavior between Linux/macOS and Windows is unreasonable; given that 
time.monotonic exists for measuring time intervals (which are not necessarily 
required to be of short duration – unlike perf_counter).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Rishav Kundu]

Rishav Kundu  added the comment:

[1]: https://docs.python.org/3.8/library/time.html#time.clock_gettime

[2]: 
https://developer.apple.com/documentation/kernel/1646199-mach_continuous_time 
effectively uses CLOCK_MONOTONIC_RAW on macOS.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[STINNER Victor]

STINNER Victor  added the comment:

> While I agree that the behavior of perf_counter should be consistent across 
> macOS/Linux and Windows wrt system suspend,

perf_counter behavior during system suspend is undefined.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Rishav Kundu]

Rishav Kundu  added the comment:

While I agree that the behavior of perf_counter should be consistent across 
macOS/Linux and Windows wrt system suspend, you can already access those clocks 
by using time.clock_gettime [1] with appropriate clock IDs (CLOCK_MONOTONIC_RAW 
on macOS and CLOCK_BOOTTIME on Linux)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[nooB]

nooB  added the comment:

> perf_counter documentation says "It does include time elapsed during sleep 
> and is system-wide." where "sleep" here means time.sleep()

Apologies for misinterpreting the documentation. A clock function that includes 
system suspend time can be useful. Consider this as a feature request.

I noticed that on both Linux and Mac OS, the functions time.monotonic and 
time.perf_counter are exactly same. They do not count time elapsed during 
suspend.

--- CODE -

import time
for clock in "monotonic", "perf_counter":
print(f"{clock}: {time.get_clock_info(clock)}")

--- MAC OUTPUT 

monotonic: namespace(adjustable=False, implementation='mach_absolute_time()', 
monotonic=True, resolution=1e-09)
perf_counter: namespace(adjustable=False, 
implementation='mach_absolute_time()', monotonic=True, resolution=1e-09)

--- LINUX OUTPUT --

monotonic: namespace(adjustable=False, 
implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, 
resolution=1e-09)
perf_counter: namespace(adjustable=False, 
implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, 
resolution=1e-09)

--

In Windows, perf_counter uses "QueryPerformanceCounter()" which includes "the 
time when the machine was in a sleep state such as standby, hibernate, or 
connected standby"
refer: 
https://docs.microsoft.com/en-us/windows/win32/sysinfo/acquiring-high-resolution-time-stamps

I learnt that "mach_continuous_time()" in Mac OS, 
"clock_gettime(CLOCK_BOOTTIME)" in Linux and "QueryPerformanceCounter()" in 
Windows all include system suspend time. It would be nice if perf_counter can 
be tweaked to provide similar behaviour across platforms. If it is not 
recommended, the mentioned functions can be exposed as a separate clock 
function.

Thanks.

--

___
Python tracker 
<https://bugs.python.org/issue41303>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Ronald Oussoren]

Ronald Oussoren  added the comment:

mach_absolute_time returns time in ticks, there's a separate API that returns 
the resolution of this clock (which is already used).

The manpage explicitly says that mach_absolute_time and CLOCK_UPTIME_RAW are 
the same clock:

 CLOCK_UPTIME_RAW   clock that increments monotonically, in the same manner 
as CLOCK_MONOTONIC_RAW, but that does
not increment while the system is asleep.  The returned 
value is identical to the result of
mach_absolute_time() after the appropriate 
mach_timebase conversion is applied.

Switching from mach_absolute_time to CLOCK_UPTIME_RAW would therefore bring us 
no improvements, and would complicate the code base because clock_gettime is 
only available starting from macOS 10.12.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[STINNER Victor]

STINNER Victor  added the comment:

> On macOS CLOCK_UPTIME_RAW is the same as mach_absolute_time (according to the 
> manpage).

https://developer.apple.com/documentation/kernel/1462446-mach_absolute_time 
says:

"mach_absolute_time: Returns current value of a clock that increments 
monotonically in tick units (starting at an arbitrary point), this clock does 
not increment while the system is asleep."

"Discussion: Prefer to use the equivalent 
clock_gettime_nsec_np(CLOCK_UPTIME_RAW) in nanoseconds."

where clock_gettime_nsec_np() is: "As a non-portable extension, the 
clock_gettime_nsec_np() function will return the clock value in 64-bit 
nanoseconds."

It doesn't say anything about NTP, whereas CLOCK_MONOTONIC_RAW seems to a clock 
which is not adjusted by NTP:

"""
CLOCK_MONOTONIC_RAW

clock that increments monotonically, tracking the time
since an arbitrary point like CLOCK_MONOTONIC.  How-
ever, this clock is unaffected by frequency or time
adjustments.  It should not be compared to other
system time sources.
"""

So right, it sounds like mach_absolute_time() returns *seconds*.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Ronald Oussoren]

Ronald Oussoren  added the comment:

On macOS CLOCK_UPTIME_RAW is the same as mach_absolute_time (according to the 
manpage).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[STINNER Victor]

STINNER Victor  added the comment:

> There is a clock with similar behaviour as the Linux clock: CLOCK_UPTIME_RAW

On Linux, CLOCK_UPTIME_RAW is not adjusted by NTP and it should not be used for 
a clock using *seconds*. NTP ensures that a clock provides seconds and is not 
slower or faster.

On macOS, if you want a clock which is incremented while the system is asleep, 
CLOCK_MONOTONIC sounds like a better choice.

I don't know if time.perf_counter() and CLOCK_MONOTONIC have similar effective 
resolution on macOS. time.perf_counter() should have a better resolution, but 
can have a worse accurary, than time.monotonic().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Ronald Oussoren]

Ronald Oussoren  added the comment:

@Nathaniel: I hadn't noticed that CLOCK_MONOTONIC on macOS behaves different 
from that clock on Linux. Sigh.

That means there's little reason to switch to CLOCK_MONOTONIC on macOS, that 
would just result in different behaviour between Linux and macOS. 

There is a clock with similar behaviour as the Linux clock: CLOCK_UPTIME_RAW, 
but switching to that instead of mach_absolute_time would just complicate the 
code base because we still support macOS 10.9 where clock_gettime is not 
available.

BTW. I'm against using mach_continuous_time, if a change is needed it should be 
to clock_gettime as that's the more portable API.   And given the stated goal 
of time.perf_counter() a change is IMHO not necessary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[STINNER Victor]

STINNER Victor  added the comment:

> (On which note: Please don't switch macOS to use 
> clock_gettime(CLOCK_MONOTONIC); that would be a breaking change for us!)

Right, I propose to add new clock(s) since people may rely on the current 
clock(s) specifications.

> At least Linux, macOS, FreeBSD, and Windows all have a way to access a 
> monotonic clock that stops ticking while the system is suspended.

Python also tries to support AIX, OpenBSD, Solaris, Android, etc.
https://pythondev.readthedocs.io/platforms.html#best-effort-and-unofficial-platforms

I'm not sure that all "supported" platforms provide such clock. It's ok if such 
clock is not available on all platforms. People can fallback on 
monotonic/perf_counter depending on their need. For example, previously, it was 
common to write something like:

try: from time import monotonic
except ImportError: from time import time as monotonic # Python 2 or clock not 
available

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Nathaniel Smith]

Nathaniel Smith  added the comment:

I made a record of my investigations here: 
https://github.com/python-trio/trio/issues/1586

One reason we might care about this is that asyncio ought to be using a clock 
that stops ticking while suspended.

(On which note: Please don't switch macOS to use 
clock_gettime(CLOCK_MONOTONIC); that would be a breaking change for us!)

At least Linux, macOS, FreeBSD, and Windows all have a way to access a 
monotonic clock that stops ticking while the system is suspended.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com