[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

New submission from Dong-hee Na :

This is a similar case with https://bugs.python.org/issue41902
A quite possible usecase when the user use range with big number

./python -m pyperf compare_to longrange_master.json longrange_opt.json
Mean +- std dev: [longrange_master] 6.45 us +- 0.09 us -> [longrange_opt] 5.72 
us +- 0.07 us: 1.13x faster (-11%)

--
assignee: corona10
components: Interpreter Core
files: bench_longrange.py
messages: 379578
nosy: corona10, methane, vstinner
priority: normal
severity: normal
status: open
title: Micro optimization for longrange iteration if step is 1
type: performance
versions: Python 3.10
Added file: https://bugs.python.org/file49539/bench_longrange.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Change by Dong-hee Na :


--
keywords: +patch
pull_requests: +21886
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/22971

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Sorry, but I do not think it is worth. If there are any issues with multiplying 
by 1, it may be worth to optimize multiplication of integers in general. But 
iterating range(1 << 1000, (1 << 1000) + 100) looks a pretty artificial example.

--
nosy: +mark.dickinson, rhettinger, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

At bpo-41092, we decide not to optimize mulitply operation itself.

I approach as same case that step=1 is quite a lot usecase until user designate 
step value.

If range with longobject is not that much use case, I can agree with your 
opinion

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

s/
I approach as same case that step=1 is quite a lot usecase until user designate 
step value.
/My assumtion is that step=1 is quite often usecase until user designate step 
value.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

if somebody try to sum very large value with range.
sum(range(1 << 1000, (1 << 1000) + 100))
it can be affected by this patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I concur with Serhiy

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Inada Naoki]

Inada Naoki  added the comment:

I agree that 1<<1000 is artificial example. But isn't this patch affects more 
regular patterns like [[] for _ in range(1)]?

If we reject this, shouldn't we revert GH-22479 too?
I believe iterating range object is much more common use  case than range.index.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

My particular opinion is that I would advise against having more branching, 
especially for code that is rare: it augments the possibilities of branch 
mispredictions, the changes of untested code paths, and the possibilities of 
unknown interactions. In this case, I agree that the code is simple enough, but 
I still feel that the cost may not be worth the extra maintenance needs of 
those branches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Inada Naoki]

Inada Naoki  added the comment:

I had not noticed that there are rangeobject and longrangeobject. It doesn't 
affect to range(1) definitely.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

@methane

To be more precise, [[] for _ in range(1)] will not effort this patch 
because range(1) does not create longrangeiter object.
This patch will affect to if the number is veeery large.

On the other hand, GH-22479 is affect to all index API() whether the number is 
large or small.

p.s by the way, Naoki is the last name or first name? ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

s / will not effort / will not be affected

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

@pablogsal, @serhiy.storchaka

> I would advise against having more branching, especially for code that is 
> rare:

About bpo-41902, I believe those patches affect all range objects which is 
created with step is one .

I believe the usage of creation of range object with range(start), range(start, 
stop) is more than range(start, stop, step) unless there is significant proof 
that users prefer to create with range(start, stop, step)

So I think that those patches are worth to do it.

but PR 22971 is only affecting to longrange iteration.
it means if there no worth to optimize range(start), range(start, stop) when 
the start and stop is big number, this issue could be rejected.

And if the patch is rejected, I would like to suggest write a comment on the 
codeline for the future contributors why we decided to not optimizing this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Inada Naoki]

Inada Naoki  added the comment:

> On the other hand, GH-22479 is affect to all index API() whether the number 
> is large or small.

Then, no need to revert GH-22479 for consistency. Thanks.

> p.s by the way, Naoki is the last name or first name? ;)

It is difficult to say what is first name. Inada is surname and Naoki is given 
name. Surname comes first in Japanese.

Traditionally, we used reversed name order when write name in English to follow 
English-world manner. But China and Korea don't reverse name. And Japan is 
stopping reversed name order too. Japanese junior high schools teach 
surname-given name since about 2000.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

Sorry for the offtopic

@methane

Looks like Inada san is the right expression. ;)

> But China and Korea don't reverse name.

This is a very intereting fact, the national system of Korea are using 
surname-given name order or printing with a distinc section.

A good example is the passport,
surname: Na
givenname: Dong-hee


But the education of Korea teaches to use given-surname order when going abroad 
;)
So I always use the given-surname order when I have to write my name in English.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Inada Naoki]

Inada Naoki  added the comment:

Oh, I didn't know that. Thank you.

I thought Chinese and Korean use surname-given name order because of "Xí 
Jìnpíng", "Moon Jae-in", and "Kim Jong-un". Japanese previous P.M. used "Shinzo 
Abe" English notation, but changed it to "Abe Shinzo" recently.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I think that In sum(range(1 << 1000, (1 << 1000) + 100)) it is dwarfed by 
repeated addition of long integers in sum(). I expect the the effect of this 
optimization is even less that 13% in this case. In any case, for sum(range(a, 
b)) it is better to use formula (b-a)*(a+b-1)//2 if it is performance critical 
to you.

Iterating range object is pretty rare in Python (in most cases you iterate a 
sequence itself or enumerate object), and iterating range object for large 
integer is especially rare. Also, in real code you do not just iterate for the 
sake of iterating, you execute some code in every iteration. And it will make 
the effect of the optimization of this case much smaller, closer to 1% or 0.1% 
that to 13%.

Last week Inada-san played with my old optimization patch for using free lists 
for integers and decided to close the issue. He, and I, and other core 
developers agreed that it is not worth. Besides that optimization would help in 
much more cases and had good effect in microbenchmarks. And I closed also my 
other optimization patch few days ago. And did it many times in past. It is 
fanny to microoptimize the code, add special cases here and there, but it makes 
code larger and less maintenable, and a lot of such microoptimizations can have 
negative cumulative effect on the performance of general code. Sometimes the 
work of the core developer is to say "No" to his own code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

> Sometimes the work of the core developer is to say "No" to his own code.

Thank you for the careful sentence.
For clear the air, I never think that my patch should be accepted.
This is why I always get reviews from other core devs and I always accept the 
review result.
(You may know my a lot of rejected PR ;))

but for the future record, we always record why someone tried this and why 
rejected(e.g why not worthed to do it)
so this is the reason why I talked about why I tried this optimization.

In the future, if someone submits a similar patch we can give this issue to 
them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

I will close this issue by tomorrow with rejected ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42147] Micro optimization for longrange iteration if step is 1

[Dong-hee Na]

Dong-hee Na  added the comment:

I close this issue with rejected status.
Thank you serhiy, pablogsal and Inada-san for discussion ;)

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com