Entering a very large number

[ast]

Hi

I found this way to put a large number in
a variable.

C = int(
"28871482380507712126714295971303939919776094592797"
"22700926516024197432303799152733116328983144639225"
"94197780311092934965557841894944174093380561511397"
"4215424169339729054237110027510420801349667317"
"5515285922696291677532547505856101949404200039"
"90443211677661994962953925045269871932907037356403"
"22737012784538991261203092448414947289768854060249"
"76768122077071687938121709811322297802059565867")

It works but is it not optimal since there is a
string to int conversion.

I was not able to put an integer directly because
character '\' for line cut doesnt work inside an
integer

C = \
28871482380507712126714295971303939919776094592797\
22700926516024197432303799152733116328983144639225\
...
76768122077071687938121709811322297802059565867

doesn't work

Do you have a better idea ?

Thx


--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Wolfgang Maier]

On 03/23/2018 01:16 PM, ast wrote:

Hi

I found this way to put a large number in
a variable.

C = int(
"28871482380507712126714295971303939919776094592797"
"22700926516024197432303799152733116328983144639225"
"94197780311092934965557841894944174093380561511397"
"4215424169339729054237110027510420801349667317"
"5515285922696291677532547505856101949404200039"
"90443211677661994962953925045269871932907037356403"
"22737012784538991261203092448414947289768854060249"
"76768122077071687938121709811322297802059565867")



A very simple improvement would be to use a single
triple-quoted string. Assuming you are copy/pasting
the number from somewhere that will save a lot of your
time.


It works but is it not optimal since there is a
string to int conversion.

I was not able to put an integer directly because
character '\' for line cut doesnt work inside an
integer

C = \
28871482380507712126714295971303939919776094592797\
22700926516024197432303799152733116328983144639225\
...
76768122077071687938121709811322297802059565867

doesn't work

Do you have a better idea ?

Thx




--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Rustom Mody]

On Friday, March 23, 2018 at 5:46:56 PM UTC+5:30, ast wrote:
> Hi
> 
> I found this way to put a large number in
> a variable.

What stops you from entering the number on one single (v long) line?

In case there is a religious commitment to PEP 8 dicta, the recommended 
meditation is this line (also from PEP8):

"However, know when to be inconsistent -- sometimes style guide recommendations 
just aren't applicable"
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Wolfgang Maier]

On 03/23/2018 01:30 PM, Wolfgang Maier wrote:

On 03/23/2018 01:16 PM, ast wrote:

Hi

I found this way to put a large number in
a variable.

C = int(
"28871482380507712126714295971303939919776094592797"
"22700926516024197432303799152733116328983144639225"
"94197780311092934965557841894944174093380561511397"
"4215424169339729054237110027510420801349667317"
"5515285922696291677532547505856101949404200039"
"90443211677661994962953925045269871932907037356403"
"22737012784538991261203092448414947289768854060249"
"76768122077071687938121709811322297802059565867")



A very simple improvement would be to use a single
triple-quoted string. Assuming you are copy/pasting
the number from somewhere that will save a lot of your
time.


Like this, for example:

n = int(
''.join("""
37107287533902102798797998220837590246510135740250
46376937677490009712648124896970078050417018260538
74324986199524741059474233309513058123726617309629
91942213363574161572522430563301811072406154908250
23067588207539346171171980310421047513778063246676
89261670696623633820136378418383684178734361726757
28112879812849979408065481931592621691275889832738
44274228917432520321923589422876796487670272189318
47451445736001306439091167216856844588711603153276
70386486105843025439939619828917593665686757934951
62176457141856560629502157223196586755079324193331
64906352462741904929101432445813822663347944758178
92575867718337217661963751590579239728245598838407
58203565325359399008402633568948830189458628227828
80181199384826282014278194139940567587151170094390
35398664372827112653829987240784473053190104293586
86515506006295864861532075273371959191420517255829
71693888707715466499115593487603532921714970056938
54370070576826684624621495650076471787294438377604
53282654108756828443191190634694037855217779295145
36123272525000296071075082563815656710885258350721
45876576172410976447339110607218265236877223636045
17423706905851860660448207621209813287860733969412
""".split())
)

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[ast]

Le 23/03/2018 à 13:43, Rustom Mody a écrit :

On Friday, March 23, 2018 at 5:46:56 PM UTC+5:30, ast wrote:

Hi

I found this way to put a large number in
a variable.


What stops you from entering the number on one single (v long) line?



It is not beautiful and not very readable. It is better to
have a fixed number of digits per line (eg 50)

import this

Beautiful is better than ugly.
Readability counts.




In case there is a religious commitment to PEP 8 dicta, the recommended
meditation is this line (also from PEP8):

"However, know when to be inconsistent -- sometimes style guide recommendations just 
aren't applicable"



Yes I am using pylint which flags too long lines (80 characters)
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[ast]

Le 23/03/2018 à 13:30, Wolfgang Maier a écrit :

On 03/23/2018 01:16 PM, ast wrote:




A very simple improvement would be to use a single
triple-quoted string. Assuming you are copy/pasting
the number from somewhere that will save a lot of your
time.


no, it seems that sone \n are inserted inside the number

>>> C = int("""
1234
5678""")

Traceback (most recent call last):
  File "", line 3, in 
5678""")
ValueError: invalid literal for int() with base 10: '\n1234\n5678'


--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Johannes Bauer]

On 23.03.2018 14:01, ast wrote:

> It is not beautiful and not very readable. It is better to
> have a fixed number of digits per line (eg 50)

Oh yes, because clearly a 400-digit number becomes VERY beautiful and
readable once you add line breaks to it.

Cheers,
Joe

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Antoon Pardon]

On 23-03-18 14:01, ast wrote:
> Le 23/03/2018 à 13:43, Rustom Mody a écrit :
>> On Friday, March 23, 2018 at 5:46:56 PM UTC+5:30, ast wrote:
>>> Hi
>>>
>>> I found this way to put a large number in
>>> a variable.
>>
>> What stops you from entering the number on one single (v long) line?
>
>
> It is not beautiful and not very readable. It is better to
> have a fixed number of digits per line (eg 50)

Numbers that large are not readable, no matter how you put then in your
code. Such a blob of digits just doesn't carry much meaning to humans.

What meaningful information from number can you easily retrieve from
representing the number in some kind of table form that you can't from
just writing the number on one line?

-- 
Antoon.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[ast]

Le 23/03/2018 à 13:55, Wolfgang Maier a écrit :

On 03/23/2018 01:30 PM, Wolfgang Maier wrote:

On 03/23/2018 01:16 PM, ast wrote:




n = int(
     ''.join("""
37107287533902102798797998220837590246510135740250
46376937677490009712648124896970078050417018260538
74324986199524741059474233309513058123726617309629

...


45876576172410976447339110607218265236877223636045
17423706905851860660448207621209813287860733969412
""".split())
)



yes, good idea

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[ast]

Le 23/03/2018 à 14:16, Antoon Pardon a écrit :

On 23-03-18 14:01, ast wrote:

Le 23/03/2018 à 13:43, Rustom Mody a écrit :

On Friday, March 23, 2018 at 5:46:56 PM UTC+5:30, ast wrote:




What meaningful information from number can you easily retrieve from
representing the number in some kind of table form that you can't from
just writing the number on one line?



digit n° 103 is 1
digit n° 150 is 7
...

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Antoon Pardon]

On 23-03-18 14:35, ast wrote:
> Le 23/03/2018 à 14:16, Antoon Pardon a écrit :
>> On 23-03-18 14:01, ast wrote:
>>> Le 23/03/2018 à 13:43, Rustom Mody a écrit :
 On Friday, March 23, 2018 at 5:46:56 PM UTC+5:30, ast wrote:
>
>
>> What meaningful information from number can you easily retrieve from
>> representing the number in some kind of table form that you can't from
>> just writing the number on one line?
>>
>
> digit n° 103 is 1
> digit n° 150 is 7
> ...
>
In what way is that meaningful information? How will that information
help you in any way with writing your program?

-- 
Antoon.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Richard Damon]

On 3/23/18 9:35 AM, ast wrote:

Le 23/03/2018 à 14:16, Antoon Pardon a écrit :

On 23-03-18 14:01, ast wrote:

Le 23/03/2018 à 13:43, Rustom Mody a écrit :

On Friday, March 23, 2018 at 5:46:56 PM UTC+5:30, ast wrote:




What meaningful information from number can you easily retrieve from
representing the number in some kind of table form that you can't from
just writing the number on one line?



digit n° 103 is 1
digit n° 150 is 7
...

If the value of the specific digits is meaningful, they you likely don't 
really have a number, but a digital representation, for which a string 
is probably the better way to define it, and take the cost of the 
conversion as part of the cost to be pretty.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Thomas Jollans]

On 2018-03-23 14:01, ast wrote:
> Le 23/03/2018 à 13:43, Rustom Mody a écrit :
>> On Friday, March 23, 2018 at 5:46:56 PM UTC+5:30, ast wrote:
>>> Hi
>>>
>>> I found this way to put a large number in
>>> a variable.
>>
>> What stops you from entering the number on one single (v long) line?
> 
> 
> It is not beautiful and not very readable. It is better to
> have a fixed number of digits per line (eg 50)
> 
> import this
> 
> Beautiful is better than ugly.
> Readability counts.

I would tend to read this as a reason not to have extremely large
numbers in your code in the first place if you can avoid it.

> 
> 
>>
>> In case there is a religious commitment to PEP 8 dicta, the recommended
>> meditation is this line (also from PEP8):
>>
>> "However, know when to be inconsistent -- sometimes style guide
>> recommendations just aren't applicable"
>>
> 
> Yes I am using pylint which flags too long lines (80 characters)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Peter J. Holzer]

On 2018-03-23 14:12:27 +0100, ast wrote:
> Le 23/03/2018 à 13:55, Wolfgang Maier a écrit :
> > On 03/23/2018 01:30 PM, Wolfgang Maier wrote:
> > > On 03/23/2018 01:16 PM, ast wrote:

[quoted from the first mail in this thread:]
> > > > It works but is it not optimal since there is a
> > > > string to int conversion.


> > n = int(
> >      ''.join("""
> > 37107287533902102798797998220837590246510135740250
> > 46376937677490009712648124896970078050417018260538
> > 74324986199524741059474233309513058123726617309629
> ...
> 
> > 45876576172410976447339110607218265236877223636045
> > 17423706905851860660448207621209813287860733969412
> > """.split())
> > )
> > 
> 
> yes, good idea

Not if you want to avoid that string to int conversion (as you stated).

That is still there, but in addition you now split the string into a
list and then join the list into a different string.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Steven D'Aprano]

On Sun, 25 Mar 2018 00:05:56 +0100, Peter J. Holzer wrote:

[...]
>> yes, good idea
> 
> Not if you want to avoid that string to int conversion (as you stated).
> 
> That is still there, but in addition you now split the string into a
> list and then join the list into a different string.

I'm glad I wasn't the only one who spotted that.

There's something very curious about somebody worried about efficiency 
choosing a *less* efficient solution than what they started with. To 
quote W.A. Wulf:

"More computing sins are committed in the name of efficiency (without 
necessarily achieving it) than for any other single reason — including 
blind stupidity."

As Donald Knuth observed:

"We should forget about small efficiencies, say about 97% of the time: 
premature optimization is the root of all evil."

The Original Poster (OP) is concerned about saving, what, a tenth of a 
microsecond in total? Hardly seems worth the effort, especially if you're 
going to end up with something even slower.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 25/03/2018 02:47, Steven D'Aprano wrote:

On Sun, 25 Mar 2018 00:05:56 +0100, Peter J. Holzer wrote:

[...]

yes, good idea


Not if you want to avoid that string to int conversion (as you stated).

That is still there, but in addition you now split the string into a
list and then join the list into a different string.


I'm glad I wasn't the only one who spotted that.

There's something very curious about somebody worried about efficiency
choosing a *less* efficient solution than what they started with. To
quote W.A. Wulf:

"More computing sins are committed in the name of efficiency (without
necessarily achieving it) than for any other single reason — including
blind stupidity."

As Donald Knuth observed:

"We should forget about small efficiencies, say about 97% of the time:
premature optimization is the root of all evil."

The Original Poster (OP) is concerned about saving, what, a tenth of a
microsecond in total? Hardly seems worth the effort, especially if you're
going to end up with something even slower.


Using CPython on my machine, doing a string to int conversion that 
specific number took 200 times as long as doing a normal assignment.


That conversion took 4 microseconds.

Not significant if it's only done once. But it might be executed a 
million times.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Christian Gollwitzer]

Am 25.03.18 um 14:32 schrieb bartc:
Using CPython on my machine, doing a string to int conversion that 
specific number took 200 times as long as doing a normal assignment.


That conversion took 4 microseconds.

Not significant if it's only done once. But it might be executed a 
million times.


Honestly, why should it be executed a million times? Do you have a 
million different 400 digit numbers as constants in your code? If so, I 
suggest to store them in a database file accompanied with the code.


If there are few different only, then don't do the conversion a million 
times. Convert them at module initialization and assign them to a global 
variable.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Joe Pfeiffer]

ast  writes:

> Hi
>
> I found this way to put a large number in
> a variable.
>
> C = int(
> "28871482380507712126714295971303939919776094592797"
> "22700926516024197432303799152733116328983144639225"
> "94197780311092934965557841894944174093380561511397"
> "4215424169339729054237110027510420801349667317"
> "5515285922696291677532547505856101949404200039"
> "90443211677661994962953925045269871932907037356403"
> "22737012784538991261203092448414947289768854060249"
> "76768122077071687938121709811322297802059565867")
>
> It works but is it not optimal since there is a
> string to int conversion.
>
> I was not able to put an integer directly because
> character '\' for line cut doesnt work inside an
> integer
>
> C = \
> 28871482380507712126714295971303939919776094592797\
> 22700926516024197432303799152733116328983144639225\
> ...
> 76768122077071687938121709811322297802059565867

After following the thread for a while...  you will, of course, simply
have to do a string to int conversion no matter what approach you take
to writing it.  The number is a string of digits; it has to be converted
to the internal representation.  Even if you write

C = 
28871482380507712126714295971303939919776094592797227009265160241974323037991527331163289831446392259419778031109293496555784189494417409338056151139742154241693397290542371100275104208013496673175515285922696291677532547505856101949404200039904432116776619949629539250452698719329070373564032273701278453899126120309244841494728976885406024976768122077071687938121709811322297802059565867

the conversion happens.  Of all the variations people have proposed to
try to make the number more readable, the one you started with above
seems clearest to me.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 25/03/2018 15:01, Christian Gollwitzer wrote:

Am 25.03.18 um 14:32 schrieb bartc:
Using CPython on my machine, doing a string to int conversion that 
specific number took 200 times as long as doing a normal assignment.


That conversion took 4 microseconds.

Not significant if it's only done once. But it might be executed a 
million times.


Honestly, why should it be executed a million times?


Because it's inside a function that is called a million times?

 Do you have a
million different 400 digit numbers as constants in your code? If so, I 
suggest to store them in a database file accompanied with the code.


If there are few different only, then don't do the conversion a million 
times. Convert them at module initialization and assign them to a global 
variable.


That's just another workaround. You don't really want the global 
namespace polluted with names that belong inside functions. And you 
might not want to do all that initialisation on behalf of hundreds of 
functions that may or may not be called. Neither do you want the code at 
module level to be cluttered with all these giant constants.


The real problem is in writing a very long constant that doesn't 
comfortably fit on one screen line, and where the editor used doesn't 
offer any help (in displaying on multiple lines) and neither does the 
language.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Fran Litterio]

On 3/25/2018 10:53 AM, Joe Pfeiffer wrote:

After following the thread for a while...  you will, of course, simply
have to do a string to int conversion no matter what approach you take
to writing it.  The number is a string of digits; it has to be converted
to the internal representation.  Even if you write

C = 
28871482380507712126714295971303939919776094592797227009265160241974323037991527331163289831446392259419778031109293496555784189494417409338056151139742154241693397290542371100275104208013496673175515285922696291677532547505856101949404200039904432116776619949629539250452698719329070373564032273701278453899126120309244841494728976885406024976768122077071687938121709811322297802059565867

the conversion happens.


But doesn't that string-to-int conversion happen at compile-time not 
run-time?

--
Fran
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 25/03/2018 15:53, Joe Pfeiffer wrote:

ast  writes:



C = int(
"28871482380507712126714295971303939919776094592797"
"22700926516024197432303799152733116328983144639225"
"94197780311092934965557841894944174093380561511397"
"4215424169339729054237110027510420801349667317"
"5515285922696291677532547505856101949404200039"
"90443211677661994962953925045269871932907037356403"
"22737012784538991261203092448414947289768854060249"
"76768122077071687938121709811322297802059565867")



After following the thread for a while...  you will, of course, simply
have to do a string to int conversion no matter what approach you take
to writing it.


What, even with you write this:

  C = 12

?


The number is a string of digits; it has to be converted
to the internal representation.


Which is usually done by a compiler, and it will only do it once not 
each time the line is encountered in a running program. (Although a 
compiler will also do the conversion when the line is never executed!)



Even if you write

C = 
28871482380507712126714295971303939919776094592797227009265160241974323037991527331163289831446392259419778031109293496555784189494417409338056151139742154241693397290542371100275104208013496673175515285922696291677532547505856101949404200039904432116776619949629539250452698719329070373564032273701278453899126120309244841494728976885406024976768122077071687938121709811322297802059565867

the conversion happens.


But not at runtime.

--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Grant Edwards]

On 2018-03-25, bartc  wrote:
> On 25/03/2018 02:47, Steven D'Aprano wrote:
>
>> The Original Poster (OP) is concerned about saving, what, a tenth of a
>> microsecond in total? Hardly seems worth the effort, especially if you're
>> going to end up with something even slower.
>
> Using CPython on my machine, doing a string to int conversion that 
> specific number took 200 times as long as doing a normal assignment.
>
> That conversion took 4 microseconds.
>
> Not significant if it's only done once. But it might be executed a 
> million times.

Which adds up to 4 seconds.

Still not worth spending hours (or even a few minutes) to optimize.

-- 
Grant

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Joe Pfeiffer]

bartc  writes:

> On 25/03/2018 15:53, Joe Pfeiffer wrote:
>> ast  writes:
>
>>> C = int(
>>> "28871482380507712126714295971303939919776094592797"
>>> "22700926516024197432303799152733116328983144639225"
>>> "94197780311092934965557841894944174093380561511397"
>>> "4215424169339729054237110027510420801349667317"
>>> "5515285922696291677532547505856101949404200039"
>>> "90443211677661994962953925045269871932907037356403"
>>> "22737012784538991261203092448414947289768854060249"
>>> "76768122077071687938121709811322297802059565867")
>
>> After following the thread for a while...  you will, of course, simply
>> have to do a string to int conversion no matter what approach you take
>> to writing it.
>
> What, even with you write this:
>
>   C = 12
>
> ?

I'm having a hard time parsing that sentence.  If you're pointing out
that when someone writes an int of reasonable length they don't mess
with strings and explicit int conversions, of course you're right.  This
question is specficially about "very large" numbers.

>> The number is a string of digits; it has to be converted
>> to the internal representation.
>
> Which is usually done by a compiler, and it will only do it once not
> each time the line is encountered in a running program. (Although a
> compiler will also do the conversion when the line is never executed!)

If you're using compiled python, yes (though I don't really know how the
bytecode represents long numbers like this, or whether it would optimize
away the explicit int conversion).  Since it's a constant string, it
would be a *really* easy optimization.

>> Even if you write
>>
>> C =
>> 28871482380507712126714295971303939919776094592797227009265160241974323037991527331163289831446392259419778031109293496555784189494417409338056151139742154241693397290542371100275104208013496673175515285922696291677532547505856101949404200039904432116776619949629539250452698719329070373564032273701278453899126120309244841494728976885406024976768122077071687938121709811322297802059565867
>>
>> the conversion happens.
>
> But not at runtime.

If it's not in a loop, only once at runtime in any case.  Does the
interpreter save away constants as it goes, so it would only have to be
done once in any case?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 25/03/2018 16:47, Grant Edwards wrote:

On 2018-03-25, bartc  wrote:

On 25/03/2018 02:47, Steven D'Aprano wrote:


The Original Poster (OP) is concerned about saving, what, a tenth of a
microsecond in total? Hardly seems worth the effort, especially if you're
going to end up with something even slower.


Using CPython on my machine, doing a string to int conversion that
specific number took 200 times as long as doing a normal assignment.

That conversion took 4 microseconds.

Not significant if it's only done once. But it might be executed a
million times.


Which adds up to 4 seconds.

Still not worth spending hours (or even a few minutes) to optimize.


If this is a program that will only ever be used by one person, and that 
person will only ever run it once, and you know that bit is executed 1M 
time and not 50M, then you might be right.


Remember that 4 seconds might be on top of dozens of other things that 
don't appear to be worth optimising.


The chances are however that a program in development might be run 
hundreds or thousands of times.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[ast]

Le 25/03/2018 à 03:47, Steven D'Aprano a écrit :

On Sun, 25 Mar 2018 00:05:56 +0100, Peter J. Holzer wrote:






The Original Poster (OP) is concerned about saving, what, a tenth of a
microsecond in total? Hardly seems worth the effort, especially if you're
going to end up with something even slower.




I regret that many answers are malicious, as your.

The question was how to enter a large number without
going through a string, no matter why.
This question is absolutely legitimate, contrary to
some people think.
It seems that it is not feasible unless writting it in
a single line, which is awful.
OK, go on with the string, and Wolfgang solution using
triple quote is good.





--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Peter J. Holzer]

On 2018-03-25 19:18:23 +0200, ast wrote:
> Le 25/03/2018 à 03:47, Steven D'Aprano a écrit :
> > The Original Poster (OP) is concerned about saving, what, a tenth of a
> > microsecond in total? Hardly seems worth the effort, especially if you're
> > going to end up with something even slower.
> > 
> > 
> 
> I regret that many answers are malicious, as your.

I just looked up "malicious" in Merriam-Webster, just in case it has
some meaning I wasn't previously aware of.

Nope. It says: 

: having or showing a desire to cause harm to someone
: given to, marked by, or arising from malice

and malice is defined as:

: desire to cause pain, injury, or distress to another
: intent to commit an unlawful act or cause harm without legal
justification or excuse

Can you explain, why you think that Steven (or anybody else in this
thread) wants to cause you harm, pain, injury or distress, or how his
answer constitutes an unlawful act?

> The question was how to enter a large number without
> going through a string, no matter why.

And the answer is: Write it all on one line.

> This question is absolutely legitimate, contrary to
> some people think.

It certainly is. OTOH, the question why you would want to do this is
also legitimate. So is the question why you think that the few
microseconds it takes to parse the string are not acceptable.

And finally:

> It seems that it is not feasible unless writting it in
> a single line, which is awful.
> OK, go on with the string, and Wolfgang solution using
> triple quote is good.

If you think that your solution is not good enough, why do you think
Wolfgang's solution is better? It still converts a string to an int and
it does additional operations on the string. So it is slower and harder
to understand than what you already had. (It also cannot be properly
indented, but you may not care about that.)

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Richard Damon]

On 3/25/18 8:32 AM, bartc wrote:

On 25/03/2018 02:47, Steven D'Aprano wrote:

On Sun, 25 Mar 2018 00:05:56 +0100, Peter J. Holzer wrote:

[...]

yes, good idea


Not if you want to avoid that string to int conversion (as you stated).

That is still there, but in addition you now split the string into a
list and then join the list into a different string.


I'm glad I wasn't the only one who spotted that.

There's something very curious about somebody worried about efficiency
choosing a *less* efficient solution than what they started with. To
quote W.A. Wulf:

"More computing sins are committed in the name of efficiency (without
necessarily achieving it) than for any other single reason — including
blind stupidity."

As Donald Knuth observed:

"We should forget about small efficiencies, say about 97% of the time:
premature optimization is the root of all evil."

The Original Poster (OP) is concerned about saving, what, a tenth of a
microsecond in total? Hardly seems worth the effort, especially if you're
going to end up with something even slower.


Using CPython on my machine, doing a string to int conversion that 
specific number took 200 times as long as doing a normal assignment.


That conversion took 4 microseconds.

Not significant if it's only done once. But it might be executed a 
million times.




The other half of that thought is how does the 4 microseconds to create 
the constant compare to the operations USING that number. My guess is 
that for most things the usage will swamp the initialization, even if 
that is somewhat inefficient.


One option, which in my opinion is more readable, is if you really want 
the efficency is to write the number on a single line, with a place 
guide comment above, something like


# 0112233445
# 12345678901234567890123456789012345678901234567890

foo = 13452356547242573457246245673472457723245634564564

This makes it very clear what the 47th digit of the number is (run the 
guide numbers in the direction that makes sense for you)


Yes, you will have a very long line (several in fact with the guide 
comment), but that 'uglyness' is worth it if the time cost turns out to 
be significant. After all, correctness outweighs elegance (but maybe not 
clarity, but the long line isn't unclear, just a bit ugly), so if the 
time cost make the program violate the specs to be useful, you need a 
bit of ugly to get it to work. (The other option would be to initialize 
into a global variable, which has its own type of ugly)).

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 26/03/2018 00:27, Richard Damon wrote:

On 3/25/18 8:32 AM, bartc wrote:


Using CPython on my machine, doing a string to int conversion that 
specific number took 200 times as long as doing a normal assignment.


That conversion took 4 microseconds.

Not significant if it's only done once. But it might be executed a 
million times.




The other half of that thought is how does the 4 microseconds to create 
the constant compare to the operations USING that number. My guess is 
that for most things the usage will swamp the initialization, even if 
that is somewhat inefficient.


Calling a function that sets up C using 'C = 288714...' on one line, and 
that then calculates D=C+C, takes 0.12 seconds to call 100 times.


To do D=C*C, takes 2.2 seconds (I've subtracted the function call 
overhead of 0.25 seconds; there might not be any function call).


If I instead initialise C using 'C = int("288712...")', then timings 
increase as follows:


0.12  =>   3.7 seconds
2.2   =>   5.9 seconds

So the overhead /can/ be substantial, and /can/ be significant compared 
with doing bignum calculations.


Of course, once initialised, C might be used a hundred times, then the 
overhead is less significant. But it is not small enough to just dismiss.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Richard Damon]

On 3/25/18 9:37 PM, bartc wrote:

On 26/03/2018 00:27, Richard Damon wrote:

On 3/25/18 8:32 AM, bartc wrote:


Using CPython on my machine, doing a string to int conversion that 
specific number took 200 times as long as doing a normal assignment.


That conversion took 4 microseconds.

Not significant if it's only done once. But it might be executed a 
million times.




The other half of that thought is how does the 4 microseconds to 
create the constant compare to the operations USING that number. My 
guess is that for most things the usage will swamp the 
initialization, even if that is somewhat inefficient.


Calling a function that sets up C using 'C = 288714...' on one line, 
and that then calculates D=C+C, takes 0.12 seconds to call 100 times.


To do D=C*C, takes 2.2 seconds (I've subtracted the function call 
overhead of 0.25 seconds; there might not be any function call).


If I instead initialise C using 'C = int("288712...")', then timings 
increase as follows:


0.12  =>   3.7 seconds
2.2   =>   5.9 seconds

So the overhead /can/ be substantial, and /can/ be significant 
compared with doing bignum calculations.


Of course, once initialised, C might be used a hundred times, then the 
overhead is less significant. But it is not small enough to just dismiss.


And my point is that writing a program to just add or multiply once two 
FIXED big long numbers (since they are in the source code, that seems 
fixed), a million times seems  unlikely (and even then the cost isn't 
that bad, since that sounds like a run once program). And of course once 
the answer has been computed, it needs to be output somehow, which 
likely will add to the time significantly. The question would be do you 
have a REAL PRACTICAL case where this overhead actually makes a 
significant difference, or are we just talking a theoretical toy. It is 
a fact that it seems the language has no notation to express such a 
large number as a number in source code on mulitple lines (except as a 
string). I would think it would take something more than a toy program 
to persuade for something notation to do this directly (and there are 
alternatives that have been mentioned that aren't that bad for this sort 
of corner case).


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Rustom Mody]

On Monday, March 26, 2018 at 12:55:43 AM UTC+5:30, Peter J. Holzer wrote:
> On 2018-03-25 19:18:23 +0200, ast wrote:
> > Le 25/03/2018 à 03:47, Steven D'Aprano a écrit :
> > > The Original Poster (OP) is concerned about saving, what, a tenth of a
> > > microsecond in total? Hardly seems worth the effort, especially if you're
> > > going to end up with something even slower.
> > > 
> > > 
> > 
> > I regret that many answers are malicious, as your.

tl;dr:

I plead guilty with my almost first post in this thread containing: «In case 
there is a religious commitment to PEP 8 dicta… »

Apologies!

> 
> I just looked up "malicious" in Merriam-Webster, just in case it has
> some meaning I wasn't previously aware of.
> 
> Nope. It says: 
> 
> : having or showing a desire to cause harm to someone
> : given to, marked by, or arising from malice
> 
> and malice is defined as:
> 
> : desire to cause pain, injury, or distress to another
> : intent to commit an unlawful act or cause harm without legal
> justification or excuse
> 

O come! The OP is apparently not an English speaker; just do
s/malicious/gratuitously unkind/
or just
s/malicious/rude/

My excuse if any is that I was using “religious” in a precise way as
“belief-system” and the over-rigorous commitment to 72 (or whatever) length
lines can be a belief→commitment that produces egregious results

The deeper difficulty is that it is impossible to be human and not have some
non-verifiable beliefs. And yet when these beliefs harden into ossified
religions the results can be unimaginably pernicious

In particular, the complement set: belief-system - classic-religion
can contain the most pernicious examples:  Just compare the number killed in
the name of Islam or Christianity with those killed by communism and divide by
the number of years these 'religions' held sway

Some other examples:

1. Irreligion is a more pernicious belief-system than (classic) religion:
https://youtu.be/6PbYoQw8M48

2. Science as a delusional belief-system:
https://youtu.be/JKHUaNAxsTg

3. New-Atheism as a colonialist project: 
https://aquasanju.wordpress.com/2015/08/08/new-atheists-are-privilege-deniers-sushant-taing/

It is particularly striking that 2 above was banned! Shows how identical are the
subconscious psychological drivers of medieval popes and today's scientists 
(so-called)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Steven D'Aprano]

On Mon, 26 Mar 2018 02:37:44 +0100, bartc wrote:

> Calling a function that sets up C using 'C = 288714...' on one line, and
> that then calculates D=C+C, takes 0.12 seconds to call 100 times.
> 
> To do D=C*C, takes 2.2 seconds (I've subtracted the function call
> overhead of 0.25 seconds; there might not be any function call).

Bart, this is a contrived example that really proves nothing. This is 
literally the sort of premature optimization that Knuth etc so frequently 
warn about. In *almost any* real program, you're not going to be 
calculating C+C over and over and over and over again, millions of times 
in a row, while doing *literally* nothing else.

If your point is that, *hypothetically*, there could be some amazingly 
unlikely set of circumstances where that's exactly what I will need to 
do, then fine, I'll deal with those circumstances when it happens and not 
a moment before.

And I'll probably deal with it by calculating C+C in Python, then hard-
coding D= that number, reducing the runtime cost of the addition to zero.


> If I instead initialise C using 'C = int("288712...")', then timings
> increase as follows:

Given that the original number given had 397 digits and has a bit length 
of 1318, I must admit to some curiosity as to how exactly you managed to 
cast it to a C int (32 bits on most platforms).

It is too big for an int, a long (64 bits), a long-long (128 bits) or 
even a long-long-long-long-long-long-long-long-long-long-long-long-long-
long-long-long (1024 bits), if such a thing even exists.


So what exactly did you do?



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 26/03/2018 10:34, Steven D'Aprano wrote:

On Mon, 26 Mar 2018 02:37:44 +0100, bartc wrote:



If I instead initialise C using 'C = int("288712...")', then timings
increase as follows:


Given that the original number given had 397 digits and has a bit length
of 1318, I must admit to some curiosity as to how exactly you managed to
cast it to a C int (32 bits on most platforms).

It is too big for an int, a long (64 bits), a long-long (128 bits) or
even a long-long-long-long-long-long-long-long-long-long-long-long-long-
long-long-long (1024 bits), if such a thing even exists.


So what exactly did you do?


I'm not sure why you think the language C came into it. I did this:

def fn():
C = int(
"28871482380507712126714295971303939919776094592797"
"22700926516024197432303799152733116328983144639225"
"94197780311092934965557841894944174093380561511397"
"4215424169339729054237110027510420801349667317"
"5515285922696291677532547505856101949404200039"
"90443211677661994962953925045269871932907037356403"
"22737012784538991261203092448414947289768854060249"
"76768122077071687938121709811322297802059565867")

#   C = 2887148238050771212671429... [truncated for this post]

D=C+C

for i in range(100):
fn()

The purpose was to establish how such int("...") conversions compare in 
overheads with actual arithmetic with the resulting numbers.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 26/03/2018 03:35, Richard Damon wrote:

On 3/25/18 9:37 PM, bartc wrote:


So the overhead /can/ be substantial, and /can/ be significant 
compared with doing bignum calculations.


Of course, once initialised, C might be used a hundred times, then the 
overhead is less significant. But it is not small enough to just dismiss.


And my point is that writing a program to just add or multiply once two 
FIXED big long numbers (since they are in the source code, that seems 
fixed), a million times seems  unlikely (and even then the cost isn't 
that bad, since that sounds like a run once program).


Similar overheads occur when you use string=>int even on small numbers:

This code:

C = int("12345")
D = C+C  # or C*C; about the same results

takes 5 times as long (using my CPython 3.6.x on Windows) as:

C = 12345
D = C+C

Your arguments that this doesn't really matter would equally apply here.

Yet you don't see Python code full of 'int("43")' instead of just '43' 
on the basis that the extra overhead is not significant, as the program 
might be run only once.


A slightly worrying attitude in a language that has issues with 
performance, but no longer a surprising one.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Steven D'Aprano]

On Mon, 26 Mar 2018 11:31:22 +0100, bartc wrote:

> On 26/03/2018 10:34, Steven D'Aprano wrote:

>> So what exactly did you do?
> 
> I'm not sure why you think the language C came into it.

My misunderstanding. Sorry.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Richard Damon]

On 3/26/18 6:45 AM, bartc wrote:

On 26/03/2018 03:35, Richard Damon wrote:

On 3/25/18 9:37 PM, bartc wrote:


So the overhead /can/ be substantial, and /can/ be significant 
compared with doing bignum calculations.


Of course, once initialised, C might be used a hundred times, then 
the overhead is less significant. But it is not small enough to just 
dismiss.


And my point is that writing a program to just add or multiply once 
two FIXED big long numbers (since they are in the source code, that 
seems fixed), a million times seems  unlikely (and even then the cost 
isn't that bad, since that sounds like a run once program).


Similar overheads occur when you use string=>int even on small numbers:

This code:

    C = int("12345")
    D = C+C  # or C*C; about the same results

takes 5 times as long (using my CPython 3.6.x on Windows) as:

    C = 12345
    D = C+C

Your arguments that this doesn't really matter would equally apply here.

Yet you don't see Python code full of 'int("43")' instead of just '43' 
on the basis that the extra overhead is not significant, as the 
program might be run only once.


A slightly worrying attitude in a language that has issues with 
performance, but no longer a surprising one.


Strawman. The original argument was that there was sufficient clarity 
grounds to want to express the big number as a string. That base 
argument doesn't hold here, and in fact for small number the construct 
clearly makes the code less clear, so the construct fails there. If the 
numbers were formatted in a way that using a string, and doing some 
manipulations on it first, and then converting made sense, you might be 
able to make the argument, but then again it comes down to efficiency vs 
clarity, and for that, as long as efficiency hasn't suffered enough to 
start to become a correctness issue, clarity has an edge.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Richard Damon]

On 3/26/18 6:31 AM, bartc wrote:

On 26/03/2018 10:34, Steven D'Aprano wrote:

On Mon, 26 Mar 2018 02:37:44 +0100, bartc wrote:



If I instead initialise C using 'C = int("288712...")', then timings
increase as follows:


Given that the original number given had 397 digits and has a bit length
of 1318, I must admit to some curiosity as to how exactly you managed to
cast it to a C int (32 bits on most platforms).

It is too big for an int, a long (64 bits), a long-long (128 bits) or
even a long-long-long-long-long-long-long-long-long-long-long-long-long-
long-long-long (1024 bits), if such a thing even exists.


So what exactly did you do?


I'm not sure why you think the language C came into it. I did this:

def fn():
    C = int(
    "28871482380507712126714295971303939919776094592797"
    "22700926516024197432303799152733116328983144639225"
    "94197780311092934965557841894944174093380561511397"
    "4215424169339729054237110027510420801349667317"
    "5515285922696291677532547505856101949404200039"
    "90443211677661994962953925045269871932907037356403"
    "22737012784538991261203092448414947289768854060249"
    "76768122077071687938121709811322297802059565867")

#   C = 2887148238050771212671429... [truncated for this post]

    D=C+C

for i in range(100):
    fn()

The purpose was to establish how such int("...") conversions compare 
in overheads with actual arithmetic with the resulting numbers.


Of course if this was done in C with a version that had builtin bignum 
ints or an aggresive enough optimizer (or a Python that did a similar 
level of optimizations) this function would just test the speed of 
starting the program, as it actually does nothing and can be optimized 
away. Yes, something like this can beused to measure the base time to do 
something, but the real question should be is that time significant 
compared to the other things that the program is doing, Making a 200x 
improvement on code that takes 1% of the execution time saves you 
0.995%, not normally worth it unless your program is currently running 
at 100.004% of the allowed (or acceptable) timing, if acceptable timing 
can even be defined that precisely.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Steven D'Aprano]

On Mon, 26 Mar 2018 11:45:33 +0100, bartc wrote:

> Similar overheads occur when you use string=>int even on small numbers:
> 
> This code:
> 
>  C = int("12345")
>  D = C+C  # or C*C; about the same results
> 
> takes 5 times as long (using my CPython 3.6.x on Windows) as:
> 
>  C = 12345
>  D = C+C
> 
> Your arguments that this doesn't really matter would equally apply here.

Efficiency-wise, you are correct.

But that's not why we (well, some of us...) write C = 12345 instead of 

C = int("12345")

or 
C = (((int("1")*10 + int("2"))*10 + int("3"))*10 + int("4")
 )*10 + int(5)

Even that second one would probably be an insignificant runtime cost for 
99% of real applications.

No, the reason why we write C = 12345 is because it is the most straight-
forward, natural, idiomatic way to assign 12345, and therefore the least 
surprising and easiest to read and understand. The fact that it is also 
faster is a bonus.

Contrariwise, we *do* write:

D = Decimal("12.345")

and similar idioms.

When it comes to short integer values, there's no possible benefit to 
either readability or performance to write it as a string and convert. It 
is *harder* to read int("12345") than simply 12345.

But when it comes to huge ints with hundreds or thousands of digits, 
that's not the case. In the absence of special syntax to support huge 
ints, the most readable way to include a HUGE literal int is as a string, 
and then perform our own processing:

C = """123 456 789 012 345 678 901 234 567 ...
   567 890 123 456 789 012 345 678 901 ...
   ..."""
C = parse_and_convert(C)

and so on. As the author, you get to decide how many extra work you are 
prepared to do in the conversion step, in order to buy you extra 
readability. For example, formatting the number in groups of three digits.

Now, if we had a specialist language that was focused specifically on 
huge ints with thousands of digits, then it might be worth building in 
special syntax for it. Python is already half-way there: recent versions 
support using underscores in ints to make it easier to group digits. All 
we really need now is support for multi-line ints.

But for a generalist language like Python, it's probably not too clever 
to try to support ever smaller niche requirements. Especially for an open 
source project, manpower is always at short supply. There are far more 
important priorities to attend to.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 26/03/2018 13:30, Richard Damon wrote:

On 3/26/18 6:31 AM, bartc wrote:


The purpose was to establish how such int("...") conversions compare 
in overheads with actual arithmetic with the resulting numbers.


Of course if this was done in C with a version that had builtin bignum 
ints or an aggresive enough optimizer (or a Python that did a similar 
level of optimizations) this function would just test the speed of 
starting the program, as it actually does nothing and can be optimized 
away.


Which is a nuisance. /You/ are trying to measure how long it takes to 
perform a task, the compiler is demonstrating how long it takes to /not/ 
perform it! So it can be very unhelpful.


Hence my testing with CPython 3.6, rather than on something like PyPy 
which can give results that are meaningless. Because, for example, real 
code doesn't repeatedly execute the same pointless fragment millions of 
times. But a real context is too complicated to set up.


 Yes, something like this can beused to measure the base time to do
something, but the real question should be is that time significant 
compared to the other things that the program is doing, Making a 200x 
improvement on code that takes 1% of the execution time saves you 
0.995%, not normally worth it unless your program is currently running 
at 100.004% of the allowed (or acceptable) timing, if acceptable timing 
can even be defined that precisely.


I'm usually concerned with optimisation in a more general sense than a 
specific program.


Such a with a library function (where you don't know how it's going to 
be used); or with a particular byte-code in an interpreter (you don't 
know how often it will be encountered); or a generated code sequence in 
a compiler.


But even 200x improvement on something that takes 1% of the time can be 
worthwhile if it is just one of dozens of such improvements. Sometimes 
these small, incremental changes in performance can add up.


And even if it was just 1%, the aggregate savings across one million 
users of the program can be substantial, even if the individuals won't 
appreciate it. 1% extra battery life might be a handy five minutes for 
example.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Joe Pfeiffer]

Steven D'Aprano  writes:

> On Mon, 26 Mar 2018 02:37:44 +0100, bartc wrote:
>
>> If I instead initialise C using 'C = int("288712...")', then timings
>> increase as follows:
>
> Given that the original number given had 397 digits and has a bit length 
> of 1318, I must admit to some curiosity as to how exactly you managed to 
> cast it to a C int (32 bits on most platforms).
>
> It is too big for an int, a long (64 bits), a long-long (128 bits) or 
> even a long-long-long-long-long-long-long-long-long-long-long-long-long-
> long-long-long (1024 bits), if such a thing even exists.
>
> So what exactly did you do?

Curiously, the first time I read the original post I also somehow
confused it with C code.  But no, the variable is named C, it isn't
written in the C language.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Chris Angelico]

On Mon, Mar 26, 2018 at 11:46 PM, bartc  wrote:
> On 26/03/2018 13:30, Richard Damon wrote:
>>
>> On 3/26/18 6:31 AM, bartc wrote:
>
>
>>> The purpose was to establish how such int("...") conversions compare in
>>> overheads with actual arithmetic with the resulting numbers.
>>>
>> Of course if this was done in C with a version that had builtin bignum
>> ints or an aggresive enough optimizer (or a Python that did a similar level
>> of optimizations) this function would just test the speed of starting the
>> program, as it actually does nothing and can be optimized away.
>
>
> Which is a nuisance. /You/ are trying to measure how long it takes to
> perform a task, the compiler is demonstrating how long it takes to /not/
> perform it! So it can be very unhelpful.

Yeah. It's so annoying that compilers work so hard to make your code
fast, when all you want to do is measure exactly how slow it is.
Compiler authors are stupid.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Christian Gollwitzer]

Am 26.03.18 um 12:31 schrieb bartc:

On 26/03/2018 10:34, Steven D'Aprano wrote:

So what exactly did you do?


I did this:

def fn():
     C = int(
     "28871482380507712126714295971303939919776094592797"
     "22700926516024197432303799152733116328983144639225"
     "94197780311092934965557841894944174093380561511397"
     "4215424169339729054237110027510420801349667317"
     "5515285922696291677532547505856101949404200039"
     "90443211677661994962953925045269871932907037356403"
     "22737012784538991261203092448414947289768854060249"
     "76768122077071687938121709811322297802059565867")

#   C = 2887148238050771212671429... [truncated for this post]

     D=C+C

for i in range(100):
     fn()

The purpose was to establish how such int("...") conversions compare in 
overheads with actual arithmetic with the resulting numbers.


I still claim that it is not necessary to perform the conversion on each 
function call. For instance, the following change speeds it up by a 
factor of 7 on my machine:


===
def fn(C = int(
"28871482380507712126714295971303939919776094592797"
"22700926516024197432303799152733116328983144639225"
"94197780311092934965557841894944174093380561511397"
"4215424169339729054237110027510420801349667317"
"5515285922696291677532547505856101949404200039"
"90443211677661994962953925045269871932907037356403"
"22737012784538991261203092448414947289768854060249"
"76768122077071687938121709811322297802059565867")):
#   C = 2887148238050771212671429... [truncated for this post]

D=C+C
return D


for i in range(100):
fn()

#print(fn())
===

Out of pure laziness, I haven't compared it to the version with the long 
string. Maybe tha gurus here find yet another elegant way to assign the 
constant only once; my trial with "fn.C = foobar" was not successful.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Richard Damon]

On 3/26/18 8:46 AM, bartc wrote:

On 26/03/2018 13:30, Richard Damon wrote:

On 3/26/18 6:31 AM, bartc wrote:


The purpose was to establish how such int("...") conversions compare 
in overheads with actual arithmetic with the resulting numbers.


Of course if this was done in C with a version that had builtin 
bignum ints or an aggresive enough optimizer (or a Python that did a 
similar level of optimizations) this function would just test the 
speed of starting the program, as it actually does nothing and can be 
optimized away.


Which is a nuisance. /You/ are trying to measure how long it takes to 
perform a task, the compiler is demonstrating how long it takes to 
/not/ perform it! So it can be very unhelpful.


Hence my testing with CPython 3.6, rather than on something like PyPy 
which can give results that are meaningless. Because, for example, 
real code doesn't repeatedly execute the same pointless fragment 
millions of times. But a real context is too complicated to set up.
The bigger issue is that these sort of micro-measurements aren't 
actually that good at measuring real quantitative performance costs. 
They can often give qualitative indications, but the way modern 
computers work, processing environment is extremely important in 
performance, so these sorts of isolated measure can often be misleading. 
The problem is that if you measure operation a, and then measure 
operation b, if you think that doing a then b in the loop that you will 
get a time of a+b, you will quite often be significantly wrong, as cache 
performance can drastically affect things. Thus you really need to do 
performance testing as part of a practical sized exercise, not a micro 
one, in order to get a real measurement.


 Yes, something like this can beused to measure the base time to do
something, but the real question should be is that time significant 
compared to the other things that the program is doing, Making a 200x 
improvement on code that takes 1% of the execution time saves you 
0.995%, not normally worth it unless your program is currently 
running at 100.004% of the allowed (or acceptable) timing, if 
acceptable timing can even be defined that precisely.


I'm usually concerned with optimisation in a more general sense than a 
specific program.


Such a with a library function (where you don't know how it's going to 
be used); or with a particular byte-code in an interpreter (you don't 
know how often it will be encountered); or a generated code sequence 
in a compiler.


But even 200x improvement on something that takes 1% of the time can 
be worthwhile if it is just one of dozens of such improvements. 
Sometimes these small, incremental changes in performance can add up.


And even if it was just 1%, the aggregate savings across one million 
users of the program can be substantial, even if the individuals won't 
appreciate it. 1% extra battery life might be a handy five minutes for 
example.


Yes, but if you find where you are really spending your time, a similar 
effort may give significantly larger improvements.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Steven D'Aprano]

On Mon, 26 Mar 2018 23:49:07 -0400, Richard Damon wrote:

> The bigger issue is that these sort of micro-measurements aren't
> actually that good at measuring real quantitative performance costs.
> They can often give qualitative indications, but the way modern
> computers work, processing environment is extremely important in
> performance, so these sorts of isolated measure can often be misleading.
> The problem is that if you measure operation a, and then measure
> operation b, if you think that doing a then b in the loop that you will
> get a time of a+b, you will quite often be significantly wrong, as cache
> performance can drastically affect things. Thus you really need to do
> performance testing as part of a practical sized exercise, not a micro
> one, in order to get a real measurement.


I think that is so well said, and so important, that I'm replying to the 
post just to quote it even though I have nothing substantial to add to it.




-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 26/03/2018 16:31, Chris Angelico wrote:

On Mon, Mar 26, 2018 at 11:46 PM, bartc  wrote:

On 26/03/2018 13:30, Richard Damon wrote:


On 3/26/18 6:31 AM, bartc wrote:




The purpose was to establish how such int("...") conversions compare in
overheads with actual arithmetic with the resulting numbers.


Of course if this was done in C with a version that had builtin bignum
ints or an aggresive enough optimizer (or a Python that did a similar level
of optimizations) this function would just test the speed of starting the
program, as it actually does nothing and can be optimized away.



Which is a nuisance. /You/ are trying to measure how long it takes to
perform a task, the compiler is demonstrating how long it takes to /not/
perform it! So it can be very unhelpful.


Yeah. It's so annoying that compilers work so hard to make your code
fast, when all you want to do is measure exactly how slow it is.
Compiler authors are stupid.


In some ways, yes they are. If they were in charge of Formula 1 pre-race 
speed trials, all cars would complete the circuit in 0.00 seconds with 
an average speed of infinity mph.


Because they can see that they all start and end at the same point so 
there is no reason to actually go around the track.


And in actual computer benchmarks, the compilers are too stupid to 
realise that this is not real code doing a useful task that is being 
done, and the whole thing is optimised away as being pointless.


Optimisation is a very bad idea on microbenchmarks if the results are 
going to be misleading.



--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[bartc]

On 27/03/2018 04:49, Richard Damon wrote:

On 3/26/18 8:46 AM, bartc wrote:


Hence my testing with CPython 3.6, rather than on something like PyPy 
which can give results that are meaningless. Because, for example, 
real code doesn't repeatedly execute the same pointless fragment 
millions of times. But a real context is too complicated to set up.


The bigger issue is that these sort of micro-measurements aren't 
actually that good at measuring real quantitative performance costs. 
They can often give qualitative indications, but the way modern 
computers work, processing environment is extremely important in 
performance, so these sorts of isolated measure can often be misleading. 
The problem is that if you measure operation a, and then measure 
operation b, if you think that doing a then b in the loop that you will 
get a time of a+b, you will quite often be significantly wrong, as cache 
performance can drastically affect things. Thus you really need to do 
performance testing as part of a practical sized exercise, not a micro 
one, in order to get a real measurement.


That might apply to native code, where timing behaviour of a complicated 
 chip like x86 might be unintuitive.


But my comments were specifically about byte-code executed with CPython. 
Then the behaviour is a level or two removed from the hardware and with 
slightly different characteristics.


(Since the program you are actually executing is the interpreter, not 
the Python program, which is merely data. And whatever aggressive 
optimisations are done to the interpreter code, they are not affected by 
the Python program being run.)


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Richard Damon]

On 3/30/18 6:41 AM, bartc wrote:

On 27/03/2018 04:49, Richard Damon wrote:

On 3/26/18 8:46 AM, bartc wrote:


Hence my testing with CPython 3.6, rather than on something like 
PyPy which can give results that are meaningless. Because, for 
example, real code doesn't repeatedly execute the same pointless 
fragment millions of times. But a real context is too complicated to 
set up.


The bigger issue is that these sort of micro-measurements aren't 
actually that good at measuring real quantitative performance costs. 
They can often give qualitative indications, but the way modern 
computers work, processing environment is extremely important in 
performance, so these sorts of isolated measure can often be 
misleading. The problem is that if you measure operation a, and then 
measure operation b, if you think that doing a then b in the loop 
that you will get a time of a+b, you will quite often be 
significantly wrong, as cache performance can drastically affect 
things. Thus you really need to do performance testing as part of a 
practical sized exercise, not a micro one, in order to get a real 
measurement.


That might apply to native code, where timing behaviour of a 
complicated  chip like x86 might be unintuitive.


But my comments were specifically about byte-code executed with 
CPython. Then the behaviour is a level or two removed from the 
hardware and with slightly different characteristics.


(Since the program you are actually executing is the interpreter, not 
the Python program, which is merely data. And whatever aggressive 
optimisations are done to the interpreter code, they are not affected 
by the Python program being run.)


But cache behavior may very well still influence it, as a small section 
of byte code may only exercise a small part of the interpreter, and thus 
it might be able to all (or mostly)) live in cache, and thus run faster, 
while a broader program, uses more of the interpreter, and may no longer 
fit in the cache. In some ways, this can be much amplified over a fully 
compiled code as very small changes in byte code can have much bigger 
effects over what gets accessed. You probably do get less opportunity 
for things to speed up by combining pieces, but still plenty of 
opportunity to get slowdowns.


Another factor that you run into is that lookup time can be a factor, 
just the mere presence of lots of other code in the test module, even if 
not executing, can impact the speed it runs at.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

Re: Entering a very large number

[Chris Angelico]

On Fri, Mar 30, 2018 at 9:30 PM, bartc  wrote:
> On 26/03/2018 16:31, Chris Angelico wrote:
>> Yeah. It's so annoying that compilers work so hard to make your code
>> fast, when all you want to do is measure exactly how slow it is.
>> Compiler authors are stupid.
>
>
> In some ways, yes they are. If they were in charge of Formula 1 pre-race
> speed trials, all cars would complete the circuit in 0.00 seconds with an
> average speed of infinity mph.

Actually, I think this would be a vast improvement to Formula 1
racing. We just declare out-of-bounds driving to be "undefined
behaviour" in the C sense and permit it to crash at run-time, and then
optimize the actual driving away. The net effect of the race is to
burn certain amounts of fuel; the cars themselves are immaterial. A
properly optimized F1 race, therefore, is a bonfire.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list