Re: Python 2.7 range Function provokes a Memory Error

[Stephen Tucker]

Hi again,

I tried xrange, but I got an error telling me that my integer was too big
for a C long.

Clearly, xrange in Py2 is not capable of dealing with Python (that is,
possibly very long) integers.

I am raising this because,

(a) IF xrange in Py3 is a simple "port" from Py2, then it won't handle
Python integers either.

AND

(b) IF xrange in Py3 is intended to be equivalent to range (which, even in
Py2, does handle Python integers)

THEN

It could be argued that xrange in Py3 needs some attention from the
developer(s).

Stephen Tucker.


On Thu, Mar 2, 2023 at 6:24 PM Jon Ribbens via Python-list <
python-list@python.org> wrote:

> On 2023-03-02, Stephen Tucker  wrote:
> > The range function in Python 2.7 (and yes, I know that it is now
> > superseded), provokes a Memory Error when asked to deiliver a very long
> > list of values.
> >
> > I assume that this is because the function produces a list which it then
> > iterates through.
> >
> > 1. Does the  range  function in Python 3.x behave the same way?
>
> No, in Python 3 it is an iterator which produces the next number in the
> sequence each time.
>
> > 2. Is there any equivalent way that behaves more like a  for loop (that
> is,
> > without producing a list)?
>
> Yes, 'xrange' in Python 2 behaves like 'range' in Python 3.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Python 2.7 range Function provokes a Memory Error

[Stephen Tucker]

Hi,

The range function in Python 2.7 (and yes, I know that it is now
superseded), provokes a Memory Error when asked to deiliver a very long
list of values.

I assume that this is because the function produces a list which it then
iterates through.

1. Does the  range  function in Python 3.x behave the same way?

2. Is there any equivalent way that behaves more like a  for loop (that is,
without producing a list)?

To get round the problem I have written my own software that is used in a
for  loop.

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Precision Tail-off?

[Stephen Tucker]

As a follow-up to my previous message, I have just produced the following
log on IDLE, for your information:
--
>>> math.e ** (math.log
(12345678900) / 3)
4.979338592181741e+16
>>> 10 ** (math.log10 (12345678900)
/ 3)
4.979338592181736e+16
>>> 12345678900 ** (1.0 / 3.0)
4.979338592181734e+16
>>> 123456789e42 ** (1.0 / 3.0)
4.979338592181734e+16
------

Stephen Tucker.


On Fri, Feb 17, 2023 at 10:27 AM Stephen Tucker 
wrote:

> Thanks, one and all, for your reponses.
>
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
>
> Consider an integer N consisting of a finitely-long string of digits in
> base 10.
>
> Consider the infinitely-precise cube root of N (yes I know that it could
> never be computed unless N is the cube of an integer, but this is a
> mathematical argument, not a computational one), also in base 10. Let's
> call it RootN.
>
> Now consider appending three zeroes to the right-hand end of N (let's call
> it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).
>
> The *only *difference between RootN and RootNZZZ is that the decimal
> point in RootNZZZ is one place further to the right than the decimal point
> in RootN.
>
> None of the digits in RootNZZZ's string should be different from the
> corresponding digits in RootN.
>
> I rest my case.
>
> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.
>
> Stephen Tucker.
>
>
> On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson 
> wrote:
>
>> On Tue, 14 Feb 2023 11:17:20 +, Oscar Benjamin wrote:
>> > On Tue, 14 Feb 2023 at 07:12, Stephen Tucker 
>> wrote:
>> [snip]
>> >> I have just produced the following log in IDLE (admittedly, in Python
>> >> 2.7.10 and, yes I know that it has been superseded).
>> >>
>> >> It appears to show a precision tail-off as the supplied float gets
>> bigger.
>> [snip]
>> >>
>> >> For your information, the first 20 significant figures of the cube
>> root in
>> >> question are:
>> >>49793385921817447440
>> >>
>> >> Stephen Tucker.
>> >> --
>> >> >>> 123.456789 ** (1.0 / 3.0)
>> >> 4.979338592181744
>> >> >>> 1234567890. ** (1.0 / 3.0)
>> >> 49793385921817.36
>> >
>> > You need to be aware that 1.0/3.0 is a float that is not exactly equal
>> > to 1/3 ...
>> [snip]
>> > SymPy again:
>> >
>> > In [37]: a, x = symbols('a, x')
>> >
>> > In [38]: print(series(a**x, x, Rational(1, 3), 2))
>> > a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
>> >
>> > You can see that the leading relative error term from x being not
>> > quite equal to 1/3 is proportional to the log of the base. You should
>> > expect this difference to grow approximately linearly as you keep
>> > adding more zeros in the base.
>>
>> Marvelous.  Thank you.
>>
>>
>> --
>> To email me, substitute nowhere->runbox, invalid->com.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Precision Tail-off?

[Stephen Tucker]

Thanks, one and all, for your reponses.

This is a hugely controversial claim, I know, but I would consider this
behaviour to be a serious deficiency in the IEEE standard.

Consider an integer N consisting of a finitely-long string of digits in
base 10.

Consider the infinitely-precise cube root of N (yes I know that it could
never be computed unless N is the cube of an integer, but this is a
mathematical argument, not a computational one), also in base 10. Let's
call it RootN.

Now consider appending three zeroes to the right-hand end of N (let's call
it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).

The *only *difference between RootN and RootNZZZ is that the decimal point
in RootNZZZ is one place further to the right than the decimal point in
RootN.

None of the digits in RootNZZZ's string should be different from the
corresponding digits in RootN.

I rest my case.

Perhaps this observation should be brought to the attention of the IEEE. I
would like to know their response to it.

Stephen Tucker.


On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson 
wrote:

> On Tue, 14 Feb 2023 11:17:20 +, Oscar Benjamin wrote:
> > On Tue, 14 Feb 2023 at 07:12, Stephen Tucker 
> wrote:
> [snip]
> >> I have just produced the following log in IDLE (admittedly, in Python
> >> 2.7.10 and, yes I know that it has been superseded).
> >>
> >> It appears to show a precision tail-off as the supplied float gets
> bigger.
> [snip]
> >>
> >> For your information, the first 20 significant figures of the cube root
> in
> >> question are:
> >>49793385921817447440
> >>
> >> Stephen Tucker.
> >> --
> >> >>> 123.456789 ** (1.0 / 3.0)
> >> 4.979338592181744
> >> >>> 1234567890. ** (1.0 / 3.0)
> >> 49793385921817.36
> >
> > You need to be aware that 1.0/3.0 is a float that is not exactly equal
> > to 1/3 ...
> [snip]
> > SymPy again:
> >
> > In [37]: a, x = symbols('a, x')
> >
> > In [38]: print(series(a**x, x, Rational(1, 3), 2))
> > a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
> >
> > You can see that the leading relative error term from x being not
> > quite equal to 1/3 is proportional to the log of the base. You should
> > expect this difference to grow approximately linearly as you keep
> > adding more zeros in the base.
>
> Marvelous.  Thank you.
>
>
> --
> To email me, substitute nowhere->runbox, invalid->com.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Precision Tail-off?

[Stephen Tucker]

Hi,

I have just produced the following log in IDLE (admittedly, in Python
2.7.10 and, yes I know that it has been superseded).

It appears to show a precision tail-off as the supplied float gets bigger.

I have two questions:
1. Is there a straightforward explanation for this or is it a bug?
2. Is the same behaviour exhibited in Python 3.x?

For your information, the first 20 significant figures of the cube root in
question are:
   49793385921817447440

Stephen Tucker.
--
>>> 123.456789 ** (1.0 / 3.0)
4.979338592181744
>>> 123456.789 ** (1.0 / 3.0)
49.79338592181744
>>> 123456789. ** (1.0 / 3.0)
497.9338592181743
>>> 123456789000. ** (1.0 / 3.0)
4979.338592181743
>>> 12345678900. ** (1.0 / 3.0)
49793.38592181742
>>> 1234567890. ** (1.0 / 3.0)
497933.8592181741
>>> 123456789. ** (1.0 / 3.0)
4979338.59218174
>>> 123456789000. ** (1.0 / 3.0)
49793385.9218174
>>> 12345678900. ** (1.0 / 3.0)
497933859.2181739
>>> 1234567890. ** (1.0 / 3.0)
4979338592.181739
>>> 123456789. ** (1.0 / 3.0)
49793385921.81738
>>> 123456789000. ** (1.0 / 3.0)
497933859218.1737
>>> 12345678900. ** (1.0 / 3.0)
4979338592181.736
>>> 1234567890. ** (1.0 / 3.0)
49793385921817.36
>>> 123456789. ** (1.0 / 3.0)
497933859218173.56
>>> 123456789000. ** (1.0 / 3.0)
4979338592181735.0
>>> 12345678900. ** (1.0 / 3.0)
4.979338592181734e+16
>>> 1234567890. ** (1.0 / 3.0)
4.979338592181734e+17
>>> 123456789. ** (1.0 /
3.0)
4.979338592181733e+18
>>> 123456789000. ** (1.0 /
3.0)
4.979338592181732e+19
>>> 12345678900. **
(1.0 / 3.0)
4.9793385921817313e+20
--
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: IDLE "Codepage" Switching?

[Stephen Tucker]

Thanks for these responses.

I was encouraged to read that I'm not the only one to find this all
confusing.

I have investigated a little further.

1. I produced the following IDLE log:

>>> mylongstr = ""
>>> for thisCP in range (1, 256):
mylongstr += chr (thisCP) + " " + str (ord (chr (thisCP))) + ", "


>>> print mylongstr
1, 2, 3, 4, 5, 6, 7, 8, 9,
 10, 11, 12,
 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
31,   32, ! 33, " 34, # 35, $ 36, % 37, & 38, ' 39, ( 40, ) 41, * 42, + 43,
, 44, - 45, . 46, / 47, 0 48, 1 49, 2 50, 3 51, 4 52, 5 53, 6 54, 7 55, 8
56, 9 57, : 58, ; 59, < 60, = 61, > 62, ? 63, @ 64, A 65, B 66, C 67, D 68,
E 69, F 70, G 71, H 72, I 73, J 74, K 75, L 76, M 77, N 78, O 79, P 80, Q
81, R 82, S 83, T 84, U 85, V 86, W 87, X 88, Y 89, Z 90, [ 91, \ 92, ] 93,
^ 94, _ 95, ` 96, a 97, b 98, c 99, d 100, e 101, f 102, g 103, h 104, i
105, j 106, k 107, l 108, m 109, n 110, o 111, p 112, q 113, r 114, s 115,
t 116, u 117, v 118, w 119, x 120, y 121, z 122, { 123, | 124, } 125, ~
126, 127, ﾀ 128, ﾁ 129, ﾂ 130, ﾃ 131, ﾄ 132, ﾅ 133, ﾆ 134, ﾇ 135, ﾈ 136, ﾉ
137, ﾊ 138, ﾋ 139, ﾌ 140, ﾍ 141, ﾎ 142, ﾏ 143, ﾐ 144, ﾑ 145, ﾒ 146, ﾓ 147,
ﾔ 148, ﾕ 149, ﾖ 150, ﾗ 151, ﾘ 152, ﾙ 153, ﾚ 154, ﾛ 155, ﾜ 156, ﾝ 157, ﾞ
158, ﾟ 159, ﾠ 160, ﾡ 161, ﾢ 162, ﾣ 163, ﾤ 164, ﾥ 165, ﾦ 166, ﾧ 167, ﾨ 168,
ﾩ 169, ﾪ 170, ﾫ 171, ﾬ 172, ﾭ 173, ﾮ 174, ﾯ 175, ﾰ 176, ﾱ 177, ﾲ 178, ﾳ
179, ﾴ 180, ﾵ 181, ﾶ 182, ﾷ 183, ﾸ 184, ﾹ 185, ﾺ 186, ﾻ 187, ﾼ 188, ﾽ 189,
ﾾ 190, ﾿ 191, À 192, Á 193, Â 194, Ã 195, Ä 196, Å 197, Æ 198, Ç 199, È
200, É 201, Ê 202, Ë 203, Ì 204, Í 205, Î 206, Ï 207, Ð 208, Ñ 209, Ò 210,
Ó 211, Ô 212, Õ 213, Ö 214, × 215, Ø 216, Ù 217, Ú 218, Û 219, Ü 220, Ý
221, Þ 222, ß 223, à 224, á 225, â 226, ã 227, ä 228, å 229, æ 230, ç 231,
è 232, é 233, ê 234, ë 235, ì 236, í 237, î 238, ï 239, ð 240, ñ 241, ò
242, ó 243, ô 244, õ 245, ö 246, ÷ 247, ø 248, ù 249, ú 250, û 251, ü 252,
ý 253, þ 254, ÿ 255,
>>>

2. I copied and pasted the IDLE log into a text file and ran a program on
it that told me about every byte in the log.

3. I discovered the following:

Bytes 001 to 127 (01 to 7F hex) inclusive were printed as-is;

Bytes 128 to 191 (80 to BF) inclusive were output as UTF-8-encoded
characters whose codepoints were FF00 hex more than the byte values (hence
the strange glyphs);

Bytes 192 to 255 (C0 to FF) inclusive were output as UTF-8-encoded
characters - without any offset being added to their codepoints in the
meantime!

I thought you might just be interested in this - there does seem to be some
method in IDLE's mind, at least.

Stephen Tucker.








On Wed, Jan 18, 2023 at 9:41 AM Peter J. Holzer  wrote:

> On 2023-01-17 22:58:53 -0500, Thomas Passin wrote:
> > On 1/17/2023 8:46 PM, rbowman wrote:
> > > On Tue, 17 Jan 2023 12:47:29 +, Stephen Tucker wrote:
> > > > 2. Does the IDLE in Python 3.x behave the same way?
> > >
> > > fwiw
> > >
> > > Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux
> > > Type "help", "copyright", "credits" or "license()" for more
> information.
> > > str = ""
> > > for c in range(140, 169):
> > >  str += chr(c) + " "
> > >
> > > print(str)
> > > Œ   Ž ‘ ’ “ ” • – — ˜ ™ š › œ   ž Ÿ   ¡ ¢ £ ¤ ¥
> > > ¦ § ¨
> > >
> > >
> > > I don't know how this will appear since Pan is showing the icon for a
> > > character not in its set.  However, even with more undefined characters
> > > the printable one do not change. I get the same output running Python3
> > > from the terminal so it's not an IDLE thing.
> >
> > I'm not sure what explanation is being asked for here.  Let's take
> Python3,
> > so we can be sure that the strings are in unicode.  The font being used
> by
> > the console isn't mentioned, but there's no reason it should have glyphs
> for
> > any random unicode character.
>
> Also note that the characters between 128 (U+0080) and 159 (U+009F)
> inclusive aren't printable characters. They are control characters.
>
> hp
>
> --
>_  | Peter J. Holzer| Story must make more sense than reality.
> |_|_) ||
> | |   | h...@hjp.at |-- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |   challenge!"
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

IDLE "Codepage" Switching?

[Stephen Tucker]

I have four questions.

1. Can anybody explain the behaviour in IDLE (Python version 2.7.10)
reported below? (It seems that the way it renders a given sequence of bytes
depends on the sequence.)

2. Does the IDLE in Python 3.x behave the same way?

3. If it does, is this as it should behave?

4. If it is, then why is it as it should behave?
==
>>> mylongstr = ""
>>> for thisCP in range (157, 169):
mylongstr += chr (thisCP) + " "


>>> print mylongstr
ﾝ ﾞ ﾟ ﾠ ﾡ ﾢ ﾣ ﾤ ﾥ ﾦ ﾧ ﾨ
>>> mylongstr = ""
>>> for thisCP in range (158, 169):
mylongstr += chr (thisCP) + " "


>>> print mylongstr
ž Ÿ   ¡ ¢ £ ¤ ¥ ¦ § ¨
>>> mylongstr = ""
>>> for thisCP in range (157, 169):
    mylongstr += chr (thisCP) + " "


>>> print mylongstr
ﾝ ﾞ ﾟ ﾠ ﾡ ﾢ ﾣ ﾤ ﾥ ﾦ ﾧ ﾨ
==

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: file.read Method Documentation (Python 2.7.10)

[Stephen Tucker]

Chris,

Thanks for this clarification.

I have not found documentation that disagrees with you. I simply observe
that the documentation that I have alluded to earlier in this chain
(section 5.9 File Objects) could have been made clearer by the addition of
a note along the lines that the behaviour of a file's read method (in
particular, what the unit of information is that it reads (that is, "byte",
"UTF-8 encoded character", or whatever)) depends on the way in which the
file has been opened.

Thank you, Chris (and others) for your attention to my request. I consider
this enquiry closed.

Stephen.




On Wed, Jan 11, 2023 at 5:36 PM Chris Angelico  wrote:

> On Thu, 12 Jan 2023 at 04:31, Stephen Tucker 
> wrote:
> > 1. Create BOM.txt
> > 2. Input three bytes at once from BOM.txt and print them
> > 3. Input three bytes one at a time from BOM.txt and print them
>
> All of these correctly show that a file, in binary mode, reads and writes
> bytes.
>
> > 4. Input three bytes at once from BOM.txt and print them
> > >>> import codecs
> > >>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
>
> This is now a codecs file, NOT a vanilla file object. See its docs here:
>
> https://docs.python.org/2.7/library/codecs.html#codecs.open
>
> The output is "codec-dependent" but I would assume that UTF-8 will
> yield Unicode text strings.
>
> > 5. Attempt to input three bytes one at a time from BOM.txt and print them
> > -
> >
> > >>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
> > >>> myBOM_4 = myfil.read (1)
> > >>> myBOM_4
> > u'\ufeff'
>
> > A. The attempt at Part 5 actually inputs all three bytes when we ask it
> to input just the first one!
>
> On the contrary; you asked it for one *character* and it read one
> character.
>
> Where were you seeing documentation that disagreed with this?
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: file.read Method Documentation (Python 2.7.10)

[Stephen Tucker]

Chris,

Thanks for your reply.

I hope the evidence below (taken from IDLE) clarifies my issue:

Stephen.

==


1. Create BOM.txt
-

>>> myfil = open ("BOM.txt", "wb")
>>> myfil.write ("\xef" + "\xbb" + "\xbf")
>>> myfil.close()

2. Input three bytes at once from BOM.txt and print them


>>> myfil = open ("BOM.txt", "rb")
>>> myBOM = myfil.read (3)
>>> myBOM
'\xef\xbb\xbf'
>>> myfil.close()

3. Input three bytes one at a time from BOM.txt and print them
--

>>> myfil = open ("BOM.txt", "rb")
>>> myBOM_1 = myfil.read (1)
>>> myBOM_2 = myfil.read (1)
>>> myBOM_3 = myfil.read (1)
>>> myBOM_1
'\xef'
>>> myBOM_2
'\xbb'
>>> myBOM_3
'\xbf'
>>> myfil.close()

4. Input three bytes at once from BOM.txt and print them


>>> import codecs
>>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
>>> myBOM = unicode (myfil.read (3))
>>> myBOM
u'\ufeff'
>>> myfil.close ()

5. Attempt to input three bytes one at a time from BOM.txt and print them
-

>>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
>>> myBOM_4 = myfil.read (1)
>>> myBOM_5 = myfil.read (1)
>>> myBOM_6 = myfil.read (1)
>>> myBOM_4
u'\ufeff'
>>> myBOM_5
u''
>>> myBOM_6
u''
>>> myfil.close()

Notes

A. The attempt at Part 5 actually inputs all three bytes when we ask it to
input just the first one!

B. The outcome from Part 5 shows that, actually, the request to input text
in Part 4 brought about a response from the program something like this:

   Input the UTF-8-encoded character as the first "byte";
   As expected, after reaching the end of the file, continue supplying an
empty string for each of the requested extra bytes.

==


On Wed, Jan 11, 2023 at 11:00 AM Chris Angelico  wrote:

> On Wed, 11 Jan 2023 at 21:31, Stephen Tucker 
> wrote:
> >
> > Chris -
> >
> > In the Python 2.7.10 documentation, I am referring to section 5.
> Built-in Types, subsection 5.9 File Objects.
> >
> > In that subsection, I have the following paragraph:
> >
> > file.read([size])
> >
> > Read at most size bytes from the file (less if the read hits EOF before
> obtaining size bytes). If the size argument is negative or omitted, read
> all data until EOF is reached. The bytes are returned as a string object.
> An empty string is returned when EOF is encountered immediately. (For
> certain files, like ttys, it makes sense to continue reading after an EOF
> is hit.) Note that this method may call the underlying C function fread()
> more than once in an effort to acquire as close to size bytes as possible.
> Also note that when in non-blocking mode, less data than was requested may
> be returned, even if no size parameter was given.
> >
>
> Yes, so it should be that number of bytes, which is what it does, isn't it?
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: file.read Method Documentation (Python 2.7.10)

[Stephen Tucker]

Chris -

In the Python 2.7.10 documentation, I am referring to section 5. Built-in
Types, subsection 5.9 File Objects.

In that subsection, I have the following paragraph:

file.read([*size*])

Read at most *size* bytes from the file (less if the read hits EOF before
obtaining *size* bytes). If the *size* argument is negative or omitted,
read all data until EOF is reached. The bytes are returned as a string
object. An empty string is returned when EOF is encountered immediately.
(For certain files, like ttys, it makes sense to continue reading after an
EOF is hit.) Note that this method may call the underlying C function
fread() more than once in an effort to acquire as close to *size* bytes as
possible. Also note that when in non-blocking mode, less data than was
requested may be returned, even if no *size* parameter was given.

Note

This function is simply a wrapper for the underlying fread() C function,
and will behave the same in corner cases, such as whether the EOF value is
cached.
Stephen.

On Mon, Jan 9, 2023 at 6:25 PM Chris Angelico  wrote:

> On Tue, 10 Jan 2023 at 01:36, Stephen Tucker 
> wrote:
> >
> > Dear Python-list,
> >
> > Yes, I know that Python 2.x is no longer supported.
> >
> > I have found that the documentation for this method is misleading when
> the
> > file being read is UTF-8-encoded:
> >
> >Instead of reading *size* bytes, the method reads *size *UTF-8 byte
> > *sequences*.
> >
> > Has this error been corrected in the Python 3.x documentation?
> >
>
> What documentation is this? The builtin 'file' type doesn't know
> anything about encodings, and only ever returns bytes.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

file.read Method Documentation (Python 2.7.10)

[Stephen Tucker]

Dear Python-list,

Yes, I know that Python 2.x is no longer supported.

I have found that the documentation for this method is misleading when the
file being read is UTF-8-encoded:

   Instead of reading *size* bytes, the method reads *size *UTF-8 byte
*sequences*.

Has this error been corrected in the Python 3.x documentation?

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Are these good ideas?

[Stephen Tucker]

Hi,

I have two related issues I'd like comments on.

Issue 1 - Global Values

Some years ago, I had a situation where
(a) I could supply low-level functions that carry out tasks,
(b) I needed those functions to communicate with each other, but
(c) I had no access to the module that invoked my functions.

In order to achieve this, I hit on the idea that I also supply a module
that I describe as a "global values" module. This module …
(i) … defines the values that the functions use to communicate with each
other;
(ii) … is the subject of an import statement in each of my functions;
(iii) … initialises its values at the start of each run (since the import
only carries out an actual import once per session);
(iv) … acts as a repository for the values thereafter.

This solution works well.

Given that I am not particularly concerned about efficiency,
(1) Is this a reasonable way to achieve this goal?
(2) Do you know of any better ways?

Issue 2 - Passed Parameters

I am now facing another situation where I am wanting to pass 6 or 7
parameters down through several layers of logic (function A calling
function B calling ... ) and for results to be passed back.

Having had the idea described above, I am considering using it again to
save all the parameter-and-results passing.

I see nothing wrong with doing that, but I may well be missing something!

Comments, please!

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: on GNU EMACS's python-mode, loading entire buffer

[Stephen Berman]

On Sun, 04 Sep 2022 16:47:07 -0300 Meredith Montgomery  
wrote:

> Meredith Montgomery  writes:
>
>> Meredith Montgomery  writes:
>>
>> [...]
>>
>>> I would also be interested in a command that restarts the REPL afresh
>>> and reloads my buffer --- sort of like keyboard's [F5] of the IDLE.
>>
>> A partial solution for this is the following procedure.
>>
>> (defun python-revert-and-send-buffer-to-repl ()
>>   "Revert current buffer and sends it to the Python REPL."
>>   (interactive)
>>   (revert-buffer "ignore-auto-no" "no-confirm")
>>   (python-shell-send-buffer))
>>
>> We can map this to the F5-key and that improves things.  But a restart
>> of the REPL would be the ideal.  (Sometimes we really want to start
>> afresh.  Sometimes.  Most often we don't want that.)
>
> It's not easy to restart the REPL.  You can send "quit()" to it and
> invoke run-python again interactively by typing out one command after
> another, but if you write a procedure such as this one below, it doesn't
> work: it gives me the impression that there's a timing issue, that is,
> perhaps the procedure is too fast and something happens before it
> should.
>
> (defun python-save-send-buffer-to-repl ()
>   (interactive)
>   (save-buffer)
>   (python-shell-send-string "quit()")
>   (run-python)
>   (python-shell-send-buffer)
>   (python-shell-switch-to-shell))

It does seem like a timing issue.  This works for me:

(defun python-save-send-buffer-to-repl ()
  (interactive)
  (save-buffer)
  (python-shell-send-string "quit()")
  (sit-for 0.1)
  (run-python)
  (python-shell-send-buffer)
  (python-shell-switch-to-shell))

But if I decrease the wait to 0.05 it doesn't work.

Steve Berman
-- 
https://mail.python.org/mailman/listinfo/python-list

Discerning "Run Environment"

[Stephen Tucker]

Hi,

I am a Windows 10 user still using Python 2.x (for good reasons, I assure
you.)

I have a Python 2.x module that I would like to be able to use in a variety
of Python 2.x programs. The module outputs characters to the user that are
only available in the Unicode character set.

I have found that the selection of characters in that set that are
available to my software depends on whether, for example, the program is
being run during an IDLE session or at a Command Prompt.

I am therefore needing to include logic in this module that (a) enables it
to output appropriate characters depending on whether it is being run
during an IDLE session or at a command prompt, and (b) enables it to
discern which of these two "run environments" it is running in.

Goal (a) is achieved easily by using string.replace to replace unavailable
characters with available ones where necessary.

The best way I have found so far to achieve goal (b) is to use sys.modules
and ascertain whether any modules contain the string "idlelib". If they do,
that I assume that the software is being run in an IDLE session.

I suspect that there is a more Pythonic (and reliable?) way of achieving
goal (b).

Can anyone please tell me if there is, and, if there is, what it is?

Thanks.

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Printing Unicode strings in a list

[Stephen Tucker]

To Cameron Simpson,

Thanks for your in-depth and helpful reply. I have noted it and will be
giving it close attention when I can.

The main reason why I am still using Python 2.x is that my colleagues are
still using a GIS system that has a Python programmer's interface - and
that interface uses Python 2.x.

The team are moving to an updated version of the system whose Python
interface is Python 3.x.

However, I am expecting to retire over the next 8 months or so, so I do not
need to be concerned with Python 3.x - my successor will be doing that.

Stephen.


On Thu, Apr 28, 2022 at 2:07 PM Cameron Simpson  wrote:

> On 28Apr2022 12:32, Stephen Tucker  wrote:
> >Consider the following log from a run of IDLE:
> >==
> >
> >Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)]
> >on win32
> >Type "copyright", "credits" or "license()" for more information.
> >>>> print (u"\u2551")
> >║
> >>>> print ([u"\u2551"])
> >[u'\u2551']
> >>>>
> >==
> >
> >Yes, I am still using Python 2.x - I have good reasons for doing so and
> >will be moving to Python 3.x in due course.
>
> Love to hear those reasons. Not suggesting that they are invalid.
>
> >I have the following questions arising from the log:
> >1. Why does the second print statement not produce [ ║]  or ["║"] ?
>
> Because print() prints the str() or each of its arguments, and str() of
> a list if the same as its repr(), which is a list of the repr()s of
> every item in the list. Repr of a Unicode string looks like what you
> have in Python 2.
>
> >2. Should the second print statement produce [ ║]  or ["║"] ?
>
> Well, to me its behaviour is correct. Do you _want_ to get your Unicode
> glyph? in quotes? That is up to you. But consider: what would be sane
> output if the list contained the string "], [3," ?
>
> >3. Given that I want to print a list of Unicode strings so that their
> >characters are displayed (instead of their Unicode codepoint definitions),
> >is there a more Pythonic way of doing it than concatenating them into a
> >single string and printing that?
>
> You could print them with empty separators:
>
> print(s1, s2, .., sep='')
>
> To do that in Python 2 you need to:
>
> from __future__ import print_function
>
> at the top of your Python file. Then you've have a Python 3 string print
> function. In Python 2, pint is normally a statement and you don't need
> the brackets:
>
> print u"\u2551"
>
> but print() is genuinely better as a function anyway.
>
> >4. Does Python 3.x exhibit the same behaviour as Python 2.x in this
> respect?
>
> Broadly yes, except that all strings are Unicode strings and we don't
> bothing with the leading "u" prefix.
>
> Cheers,
> Cameron Simpson 
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Printing Unicode strings in a list

[Stephen Tucker]

Hi PythonList Members,

Consider the following log from a run of IDLE:

==

Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit (Intel)]
on win32
Type "copyright", "credits" or "license()" for more information.
>>> print (u"\u2551")
║
>>> print ([u"\u2551"])
[u'\u2551']
>>>

==

Yes, I am still using Python 2.x - I have good reasons for doing so and
will be moving to Python 3.x in due course.

I have the following questions arising from the log:

1. Why does the second print statement not produce [ ║]  or ["║"] ?

2. Should the second print statement produce [ ║]  or ["║"] ?

3. Given that I want to print a list of Unicode strings so that their
characters are displayed (instead of their Unicode codepoint definitions),
is there a more Pythonic way of doing it than concatenating them into a
single string and printing that?

4. Does Python 3.x exhibit the same behaviour as Python 2.x in this respect?

Thanks in anticipation.

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Problem Installing Pipenv

[Stephen P. Molnar]

I have Python 3.9 installed in Windows 10 on my Laptop and have installed
pipenv v-2021.11.5. When I invoke the python shell command, it fails with
thee following errors:

 

 

C:\Users\SPM\Apps\Spyder-5.2.1>pipenv shell

Traceback (most recent call last):

  File "C:\Python39\lib\runpy.py", line 197, in _run_module_as_main

return _run_code(code, main_globals, None,

  File "C:\Python39\lib\runpy.py", line 87, in _run_code

exec(code, run_globals)

  File "C:\Python39\Scripts\pipenv.exe\__main__.py", line 7, in 

  File "C:\Python39\lib\site-packages\pipenv\vendor\click\core.py", line
1137, in __call__

return self.main(*args, **kwargs)

  File "C:\Python39\lib\site-packages\pipenv\vendor\click\core.py", line
1062, in main

rv = self.invoke(ctx)

  File "C:\Python39\lib\site-packages\pipenv\vendor\click\core.py", line
1668, in invoke

return _process_result(sub_ctx.command.invoke(sub_ctx))

  File "C:\Python39\lib\site-packages\pipenv\vendor\click\core.py", line
1404, in invoke

return ctx.invoke(self.callback, **ctx.params)

  File "C:\Python39\lib\site-packages\pipenv\vendor\click\core.py", line
763, in invoke

return __callback(*args, **kwargs)

  File "C:\Python39\lib\site-packages\pipenv\vendor\click\decorators.py",
line 84, in new_func

return ctx.invoke(f, obj, *args, **kwargs)

  File "C:\Python39\lib\site-packages\pipenv\vendor\click\core.py", line
763, in invoke

return __callback(*args, **kwargs)

 File "C:\Python39\lib\site-packages\pipenv\cli\command.py", line 419, in
shell

do_shell(

  File "C:\Python39\lib\site-packages\pipenv\core.py", line 2309, in
do_shell

shell = choose_shell()

TypeError: choose_shell() missing 1 required positional argument: 'project'

 

Stephen P. Molnar, Ph.D.

(614)312-7528

Skype: smolnar1

 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a better way to create a list of None objects?

[Stephen Tucker]

Thanks for this feedback, Chris, Matthieu. Both are spot on - and thanks
for the timing comparison, Matthieu. I suppose I didn't think to try the
solution you suggest because I didn't think that I would end up with a
single list, but 8 of them.

OK, I'll stop wriggling.

Stephen.

On Thu, Aug 12, 2021 at 10:22 AM Matthieu Dartiailh 
wrote:

> You can achieve the same result by writing:
> [None] * 8
>
> Comparing both cases in IPython I get:
>
> In [1]: %timeit list((None,)*8)
> 110 ns ± 0.785 ns per loop (mean ± std. dev. of 7 runs, 1000 loops
> each)
>
> In [2]: %timeit [None] * 8
> 88.2 ns ± 0.432 ns per loop (mean ± std. dev. of 7 runs, 1000 loops
> each)
>
> So the list multiplication appears a bit faster.
>
> Best
>
> Matthieu
>
> Le 8/12/2021 à 10:57 AM, Stephen Tucker a écrit :
> > Hi,
> >
> > I thought I'd share the following piece of code that I have recently
> written
> > (a) to check that what I have done is reasonable - even optimum,
> > (b) to inform others who might be wanting to do similar things, and
> > (c) to invite comment from the community.
> >
> > ---
> >
> > #
> >
> > # Yes: Create an empty list of Band Limits for this language
> >
> > #
> >
> > # Note. The rather complicated logic on the right-hand side of the
> >
> > #   assignment below is used here because none of the following
> >
> > #   alternatives had the desired effect:
> >
> > #
> >
> > # Logic Effect
> >
> > #
> >
> > # [None * 8]TypeError: unsupported operand type(s) for *: ...
> >
> > # [(None) * 8]  TypeError: unsupported operand type(s) for *: ...
> >
> > # [((None)) * 8]TypeError: unsupported operand type(s) for *: ...
> >
> > # [(None,) * 8] [(None, None, None, None, None, None, None, None)]
> >
> > # list ((None) * 8) TypeError: unsupported operand type(s) for *: ...
> >
> > #
> >
> > diclll_BLim [thisISO_] = list ((None,) * 8)
> >
> >
> > ---
> >
> > Thanks in anticipation.
> >
> > Stephen Tucker.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Is there a better way to create a list of None objects?

[Stephen Tucker]

Hi,

I thought I'd share the following piece of code that I have recently written
(a) to check that what I have done is reasonable - even optimum,
(b) to inform others who might be wanting to do similar things, and
(c) to invite comment from the community.

---

#

# Yes: Create an empty list of Band Limits for this language

#

# Note. The rather complicated logic on the right-hand side of the

#   assignment below is used here because none of the following

#   alternatives had the desired effect:

#

# Logic Effect

#

# [None * 8]TypeError: unsupported operand type(s) for *: ...

# [(None) * 8]  TypeError: unsupported operand type(s) for *: ...

# [((None)) * 8]TypeError: unsupported operand type(s) for *: ...

# [(None,) * 8] [(None, None, None, None, None, None, None, None)]

# list ((None) * 8) TypeError: unsupported operand type(s) for *: ...

#

diclll_BLim [thisISO_] = list ((None,) * 8)


---

Thanks in anticipation.

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Trying to read from a text file to generate a graph

[Stephen Berman]

[Resending to the list only, since I couldn't post it without subscribing.]

On Wed, 28 Jul 2021 11:58:21 -0400 "Steve"  wrote:

> I forgot about the no-file rule...
>
>> On 28Jul2021 02:55, Steve  wrote:
>> I am going though a struggle with this and just don't see where it
>> fails.  I am using the Dual Bar Graph.py program from
>> https://matplotlib.org/stable/gallery/index.html website.  The file
>> from the web site works so that shows that all my installations are
>> complete.
>>
>> My program, LibreGraphics 05.py program runs but the graph is all
>> smutched up.  I am pulling data from the EXCEL-FILE.txt into the
>> program, selecting three values for each line and creating three
>> variables formatted as is shown in the original demo file.
>>
>> When you run the program, choose 112 when prompted. You will see the
>> values of the variables I want to pass to the graph section of the
>> code.  If the values are hardcoded, the graphs look good.  When the
>> variables generated by my section of the code, it does not.

The problem is due to the values of Sensors, TestStrips and SampleNumber
being strings; what you want is for them to be lists, as in the
assignments you commented out.  And since the data from the file is read
in as strings, you have to cast the elements of the Sensors and
TestStrips lists to integers, since you want the numerical values.  The
following code does the job:

Sensors = []
TestStrips = []
SampleNumber = []

x = 1
SensorNumber = input("Enter senaor number: ")
with open("_EXCEL-FILE.txt", 'r') as infile:
for lineEQN in infile:
if (lineEQN[0:1]== "."):
SN = lineEQN[44:48].strip()
if (SensorNumber == SN):
SN = x
SampleNumber.append(SN)

sv = lineEQN[25:29].strip()
Sensors.append(int(sv))

tv = lineEQN[32:37].strip()
TestStrips.append(int(tv))

x += 1

labels = SampleNumber

Add the rest of your code from the second half to make the desired bar
chart.

Steve Berman
-- 
https://mail.python.org/mailman/listinfo/python-list

Are there Python modules that allow a program to write to the screen?

[Stephen Tucker]

Hi,

I have old software written in GWBASIC that I use to plot diagrams on the
screen.

In Windows 10, I have to resort to using the DOSBox emulator to run it.

I would dearly like to re-write it in Python - ideally Python 2.7.

What, if anything, is available?

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Simple question - end a raw string with a single backslash ?

[Stephen Tucker]

For a neatish way to get a string to end with a single backslash, how about
   mystr = r"abc\ "[:-1]
(Note the space at the end of the rough-quoted string.)



Virus-free.
www.avast.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Mon, Oct 19, 2020 at 9:26 AM Antoon Pardon  wrote:

>
>
> Op 13/10/20 om 15:14 schreef Serhiy Storchaka:
> > 13.10.20 11:52, Tony Flury via Python-list пише:
> >> I am trying to write a simple expression to build a raw string that ends
> >> in a single backslash. My understanding is that a raw string should
> >> ignore attempts at escaping characters but I get this :
> >>
> >>  >>> a = r'end\'
> >>File "", line 1
> >>  a = r'end\'
> >>^
> >> SyntaxError: EOL while scanning string literal
> >>
> >> I interpret this as meaning that the \' is actually being interpreted as
> >> a literal quote - is that a bug ?
> >
> > r'You can\'t end raw string literal with a single "\"'
> >
> > If backslash be true inner in a raw string, the above literal would end
> > after \'. It would be very hard to write a raw string containing both \'
> > and \", and even \''' and \""" (there are such strings in the stdlib).
> >
> > So you have problem either with trailing backslash, or with inner
> > backslash followed by quotes. Both problems cannot be solved at the same
> > time. Python parser works as it works because initially it was easier to
> > implement, and now this cannot be changed because it would break some
> > amount of correct code.
>
> IMO the way python does this is broken.
>
>  >>> st=r'You can\'t end raw string literal with a single "\"'
>  >>> print(st)
>
> Now either the \ is special or it is not.
>
> 1) If it is special, it should change how the ' is treated but not
> appear itself.
>
> 2) If it is not special, it should just appear and not change how the '
> is treated.
>
> What python does here is a combination of both. The \ appears and it
> changes how the ' is treated. That is IMO broken.
>
> --
> Antoon Pardon.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Bulletproof json.dump?

[Stephen Rosen]

On Mon, Jul 6, 2020 at 6:37 AM Adam Funk  wrote:

> Is there a "bulletproof" version of json.dump somewhere that will
> convert bytes to str, any other iterables to list, etc., so you can
> just get your data into a file & keep working?
>

Is the data only being read by python programs? If so, consider using
pickle: https://docs.python.org/3/library/pickle.html
Unlike json dumping, the goal of pickle is to represent objects as exactly
as possible and *not* to be interoperable with other languages.


If you're using json to pass data between python and some other language,
you don't want to silently convert bytes to strings.
If you have a bytestring of utf-8 data, you want to utf-8 decode it before
passing it to json.dumps.
Likewise, if you have latin-1 data, you want to latin-1 decode it.
There is no universal and correct bytes-to-string conversion.

On Mon, Jul 6, 2020 at 9:45 AM Chris Angelico  wrote:

> Maybe what we need is to fork out the default JSON encoder into two,
> or have a "strict=True" or "strict=False" flag. In non-strict mode,
> round-tripping is not guaranteed, and various types will be folded to
> each other - mainly, many built-in and stdlib types will be
> represented in strings. In strict mode, compliance with the RFC is
> ensured (so ValueError will be raised on inf/nan), and everything
> should round-trip safely.
>

Wouldn't it be reasonable to represent this as an encoder which is provided
by `json`? i.e.

from json import dumps, UnsafeJSONEncoder
...
json.dumps(foo, cls=UnsafeJSONEncoder)

Emphasizing the "Unsafe" part of this and introducing people to the idea of
setting an encoder also seems nice.


On Mon, Jul 6, 2020 at 9:12 AM Chris Angelico  wrote:

> On Mon, Jul 6, 2020 at 11:06 PM Jon Ribbens via Python-list
>  wrote:
> >

> The 'json' module already fails to provide round-trip functionality:
> >
> > >>> for data in ({True: 1}, {1: 2}, (1, 2)):
> > ... if json.loads(json.dumps(data)) != data:
> > ... print('oops', data, json.loads(json.dumps(data)))
> > ...
> > oops {True: 1} {'true': 1}
> > oops {1: 2} {'1': 2}
> > oops (1, 2) [1, 2]
>
> There's a fundamental limitation of JSON in that it requires string
> keys, so this is an obvious transformation. I suppose you could call
> that one a bug too, but it's very useful and not too dangerous. (And
> then there's the tuple-to-list transformation, which I think probably
> shouldn't happen, although I don't think that's likely to cause issues
> either.)


Ideally, all of these bits of support for non-JSON types should be opt-in,
not opt-out.
But it's not worth making a breaking change to the stdlib over this.

Especially for new programmers, the notion that
deserialize(serialize(x)) != x
just seems like a recipe for subtle bugs.

You're never guaranteed that the deserialized object will match the
original, but shouldn't one of the goals of a de/serialization library be
to get it as close as is reasonable?


I've seen people do things which boil down to

json.loads(x)["some_id"] == UUID(...)

plenty of times. It's obviously wrong and the fix is easy, but isn't making
the default json encoder less strict just encouraging this type of bug?

Comparing JSON data against non-JSON types is part of the same category of
errors: conflating JSON with dictionaries.
It's very easy for people to make this mistake, especially since JSON
syntax is a subset of python dict syntax, so I don't think `json.dumps`
should be encouraging it.

On Tue, Jul 7, 2020 at 6:52 AM Adam Funk  wrote:

> Here's another "I'd expect to have to deal with this sort of thing in
> Java" example I just ran into:
>
> >>> r = requests.head(url, allow_redirects=True)
> >>> print(json.dumps(r.headers, indent=2))
> ...
> TypeError: Object of type CaseInsensitiveDict is not JSON serializable
> >>> print(json.dumps(dict(r.headers), indent=2))
> {
>   "Content-Type": "text/html; charset=utf-8",
>   "Server": "openresty",
> ...
> }
>

Why should the JSON encoder know about an arbitrary dict-like type?
It might implement Mapping, but there's no way for json.dumps to know that
in the general case (because not everything which implements Mapping
actually inherits from the Mapping ABC).
Converting it to a type which json.dumps understands is a reasonable
constraint.

Also, wouldn't it be fair, if your object is "case insensitive" to
serialize it as
  { "CONTENT-TYPE": ... } or { "content-type": ... } or ...
?

`r.headers["content-type"]` presumably gets a hit.
`json.loads(json.dumps(dict(r.headers)))["content-type"]` will get a
KeyError.

This seems very much out of scope for the json package because it's not
clear what it's supposed to do with this type.
Libraries should ask users to specify what they mean and not make
potentially harmful assumptions.

Best,
-Stephen
-- 
https://mail.python.org/mailman/listinfo/python-list

PEP 622

[Stephen Carboni]

Why are OR patterns going to use the bitwise operator instead of
logical or operator?

https://www.python.org/dev/peps/pep-0622/#combining-multiple-patterns-or-patterns

-Steve
-- 
https://mail.python.org/mailman/listinfo/python-list

How to handle async and inheritance?

[Stephen Rosen]

Hi all,

I'm looking at a conflict between code sharing via inheritance and async
usage. I would greatly appreciate any guidance, ideas, or best practices
which might help.

I'll speak here in terms of a toy example, but, if anyone wants to look at
the real code, I'm working on webargs. [1] Specifically, we have a `Parser`
class and an `AsyncParser` subclass, and the two have a lot of code
duplication to handle async/await. [2]


I've got an inheritance structure like this:

class MyAbstractType: ...
class ConcreteType(MyAbstractType): ...
class AsyncConcreteType(MyAbstractType): ...

One of my goals, of course, is to share code between ConcreteType and
AsyncConcreteType via their parent.
But the trouble is that there are functions defined like this:

class MyAbstractType:
def foo(self):
x = self.bar()
y = self.baz(x)
... # some code here, let's say 20 lines

class AsyncConcreteType(MyAbstractType):
async def foo(self):
x = await self.bar()
y = self.baz(x)
... # the same 20 lines as above, but with an `await` added
every-other line


I'm aware that I'm looking at "function color" and that my scenario is
pitting two language features -- inheritance and async -- against one
another. But I don't see a clean way out if we want to support an
"async-aware" version of a class with synchronous methods.

What I tried already, which I couldn't get to work, was to either fiddle
with things like `inspect` to see if the current function is async or to
use a class variable to indicate that the current class is the async
version. The idea was to write something like

class MyAbstractType:
_use_async_calls = False
def foo(self):
x = self._await_if_i_am_async(self.bar)
y = self.baz(x)
...

and that way, the async subclass just needs to change signatures to be
async with little stubs and set the flag,

class AsyncConcreteType(MyAbstractType):
_use_async_calls = True
async def foo(self):
return super().foo()
async def bar(self):
return super().bar()

but this (some of you are ahead of me on this, I'm sure!) did not work at
all. I couldn't find any way to write `_await_if_i_am_async`, other than
possibly doing some weird things with `exec`.
Once I concluded that the python wouldn't let me decide whether or not to
use await at runtime, at least with tools of which I'm aware, I basically
gave up on that route.


However, it seems like there should be some clever technique for defining
`MyAbstractType.foo` such that it awaits on certain calls *if* there's some
indication that it should do so. It's obviously possible with `exec`, but I
don't want to convert all of the core codepaths into giant `exec` blocks.
Perhaps there's a way which is safer and more maintainable though?


If anyone has experience in this space and can offer up a good solution, I
would love to hear about it.
And if someone wants to go above-and-beyond and look at webargs, and
suggest a better way for us to support aiohttp, I'd obviously welcome that
kind of help as well!

Thanks in advance, and best regards,
-Stephen


[1] https://github.com/marshmallow-code/webargs
[2]
https://github.com/marshmallow-code/webargs/blob/6668d267fa4135cf3f653e422bd168298f2213a8/src/webargs/asyncparser.py#L24
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Stephen Rosen]

Worth noting: by assertTrue he probably meant assertEqual.
But I'd recommend using assertIn [1] if you're using unittest to check
output written to stdout/stderr.
That way, your tests are slightly more robust to changes in the exact
output.


pytest may also be helpful for this (or any!) type of testing.
Disclaimer/warning: pytest can be confusing even for experienced python
programmers because it does some fancy things.
But if you put in the time to learn it, it's very popular because of the
way it structures testsuites and code reuse (i.e. fixtures).

It can do a lot to help you, and provides output capturing out of the box
[2] as well as some handy tools for building temporary testing directories
[3].


[1]
https://docs.python.org/3.6/library/unittest.html#unittest.TestCase.assertIn
[2] https://docs.pytest.org/en/stable/capture.html
[3] https://docs.pytest.org/en/stable/tmpdir.html

On Fri, Jun 19, 2020 at 2:18 PM Terry Reedy  wrote:

> On 6/17/2020 12:34 PM, Tony Flury via Python-list wrote:
>
> > In a recent application that I wrote (where output to the console was
> > important), I tested it using the 'unittest' framework, and by patching
> > sys.stderr to be a StringIO - that way my test case could inspect what
> > was being output.
>
> Tony's code with hard returns added so that code lines remain separated
> instead of wrapping.
>
> with patch('sys.stderr', StringIO()) as stderr:
>application.do_stuff()  self.assertTrue(stderr.getvalue(),
>'Woops - that didn\'t work')
> This doc, worth reading more than once, is
> https://docs.python.org/3/library/unittest.mock.html#the-patchers
>
> --
> Terry Jan Reedy
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Exceptions versus Windows ERRORLEVEL

[Stephen Tucker]

Thanks, Eryk - this is very helpful.

Stephen.

On Mon, Apr 6, 2020 at 6:43 AM Eryk Sun  wrote:

> On 4/3/20, Stephen Tucker  wrote:
> >
> > Does an exception raised by a Python 3.x program on a Windows machine set
> > ERRORLEVEL?
>
> ERRORLEVEL is an internal state of the CMD shell. It has nothing to do
> with Python. If Python exits due to an unhandled exception, the
> process exit code will be 1. If CMD waits on the process, it will set
> the ERRORLEVEL based on the exit code. But CMD doesn't always wait. By
> default its START command doesn't wait. Also, at the interactive
> command prompt it doesn't wait for non-console applications such as
> "pythonw.exe"; it only waits for console applications such as
> "python.exe".
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Exceptions versus Windows ERRORLEVEL

[Stephen Tucker]

Hi,

I have found that raising an exception in a Python 2.7.10 program running
under Windows does not set ERRORLEVEL.

I realise that Python 2.x is no longer supported.

Does an exception raised by a Python 3.x program on a Windows machine set
ERRORLEVEL?

If not, are there plans for it to do so?

Stephen.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Multiple comparisons in a single statement

[Stephen Tucker]

OK. You are right. I was misleading.

I am sorry for wasting your time.

Thanks you for helping me to express myself.

Stephen.



<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Fri, Mar 13, 2020 at 7:05 PM Dennis Lee Bieber 
wrote:

> On Fri, 13 Mar 2020 09:34:35 +, Stephen Tucker  >
> declaimed the following:
>
> >
> >*All:  *For the record, I meant that the tuples are all the same. The
>
>
> "all the same" is still ambiguous...
>
> >>> str1 = "A String"
> >>> str2 = "Another String"
> >>> str3 = "A" + " " + "String"
> >>> id(str1), id(str2), id(str3)
> (105461360, 105461296, 105461232)
> >>> tpl1 = (str1, str2)
> >>> tpl2 = (str1, str2)
> >>> tpl3 = (str3, str2)
> >>> id(tpl1), id(tpl2), id(tpl3)
> (90094216, 90094152, 90093576)
> >>> tpl1 == tpl2
> True
> >>> tpl1 == tpl3
> True
> >>> tpl4 = tpl3
> >>> tpl4 == tpl3
> True
> >>> tpl1 is tpl2
> False
> >>> tpl1 is tpl3
> False
> >>> tpl1 is tpl4
> False
> >>> tpl3 is tpl4
> True
> >>>
>
> tpl3 IS THE SAME as tpl4
> tpl1 is NOT THE SAME as tpl2, tpl3, or tpl4
>
> However, the CONTENTS of tpl4 compare as equivalent to tpl1, tpl2, and tpl3
>
>
> --
> Wulfraed Dennis Lee Bieber AF6VN
> wlfr...@ix.netcom.com
> http://wlfraed.microdiversity.freeddns.org/
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Multiple comparisons in a single statement

[Stephen Tucker]

*Chris:*   Thank you for your confirmation.

*All:  *For the record, I meant that the tuples are all the same. The
tuples I have in mind contain strings, so the issue regarding the
"equality" (or otherwise) of 0 and 0.0 does not arise in my case.

Stephen.




To answer the question

On Thu, Mar 12, 2020 at 11:26 PM John Pote 
wrote:

>
> On 12/03/2020 18:08, Chris Angelico wrote:
> > On Fri, Mar 13, 2020 at 4:55 AM Stephen Tucker 
> wrote:
> >> A quickie (I hope!).
> >>
> >> I am running Python 2.7.10 (and, yes, I know, support for it has been
> >> withdrawn.)
> > This is the same in Python 3.
> >
> >> I have three tuples that have been generated separately and I want to
> check
> >> that they are identical. all I want to do is to terminate the program
> and
> >> report an error if all three are not identical.
> >>
> >> My initial attempt to do this is to use logic of the form
> >>
> >> if not (mytup1 == mytup2 == mytup3):
> >> raise Exception ("Tuples are not identical")
> >>
> >> I have tried this logic form in IDLE, and it seems to do what I want.
> >>
> >> Is this a reasonable way to do this, or is there a better way?
> >>
> > Yes absolutely! (Although, as a minor quibble, I would say "equal"
> > rather than "identical" here - when you talk about identity, you're
> > usually using the 'is' operator.) The meaning of chained comparisons
> > is broadly equivalent to comparing the middle one against the others
> > ("a==b==c" is "a==b and b==c"), which does the right thing here.
> >
> > It's slightly unusual to negate a query rather than using "!=", but it
> > makes good sense here.
>
> In case anyone thinks the original expr
>   not (mytup1 == mytup2 == mytup3)
> could be changed to
>   (mytup1 != mytup2!= mytup3)
> remember that applying De Morgan's theorem to the original expression
> would require the 'and' operation used in chained comparisons to change
> to an 'or' operation (Python always does the 'and' operation in chained
> comparisions). EG for simple integers instead of tuples,
>
>  >>> not (1==1==1)
> False
>  >>> not (2==1==1)
> True
>  >>> (1!=1!=1)
> False  correct as before
>  >>> (2!=1!=1)
> False  oops!
>
> John
>
> >
> > ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Multiple comparisons in a single statement

[Stephen Tucker]

A quickie (I hope!).

I am running Python 2.7.10 (and, yes, I know, support for it has been
withdrawn.)

I have three tuples that have been generated separately and I want to check
that they are identical. all I want to do is to terminate the program and
report an error if all three are not identical.

My initial attempt to do this is to use logic of the form

if not (mytup1 == mytup2 == mytup3):
   raise Exception ("Tuples are not identical")

I have tried this logic form in IDLE, and it seems to do what I want.

Is this a reasonable way to do this, or is there a better way?

Thanks in anticipation.

Stephen Tucker.


<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Clarification on Immutability please

[Stephen Tucker]

Oh dear, I am sorry. I have created quite a storm.

Moreover, I am sorry because I misremembered what I had typed into Idle. My
original tuple only had two elements, not three, so the slicing [:2] didn't
affect the tuple at all - and so the second id() gave the same address as
the first one.

So, yes, I am stupid - or just having senior moments (I am 65, after all).

Stephen.




On Wed, Jan 22, 2020 at 4:30 AM Michael Torrie  wrote:

> On 1/21/20 6:52 PM, Ethan Furman wrote:
> > On 01/21/2020 10:55 AM, Michael Torrie wrote:
> >
> >> Slicing
> >> returns a new object whether one is slicing a tuple, list, or a string,
> >> the latter two are mutable objects.
> >
> > Strings are not mutable.
>
> Yup I got my items in the wrong order. I meant to say list, tuple, or a
> string, the latter two are immutable objects.  Thanks for correcting me.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Clarification on Immutability please

[Stephen Tucker]

Hi Python-list folk,

I am sure this has been answered many times before, but I need to ask it
again.

Given that the following is valid Python 2.7.10 (when keyboarded into Idle):

mytup = ("q", "w", "e")
id(mytup)
mytup = mytup [:2]
id(mytup)

and even that the first id(mytup) returns the same address as the second
one, I am left wondering exactly what immutability is.

I am left concluding that mytup is not actually a tuple (even though type
(mytup) tells me that it is).

My only explanation is that mytup is, actually, a pointer to a tuple; the
pointer can't change, but the contents of that pointer (or the data to
which the pointer points) can change.

That said, if I then type

mytup = mytup + ("r", "t")
id(mytup)

I find that this third id call returns a different address from the one
returned by the first two.

Somehow, it seems, tuples can be reduced in length (from the far end)
(which is not what I was expecting), but they cannot be extended (which I
can understand).

I am probably confused, ignorant, stupid, or any combination of the above.

Can you please help me become less so?

Many thanks.

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The file method read does not "see" CR

[Stephen Tucker]

Chris,

Many thanks. The first part of your answer was spot on. I have modified the
program, opening its input file in "rb" mode, and now the program is
working as I intend it to.

I am puzzled by the second part, where you say that the read call will
return one character, if the file is in text mode.

I was expecting this statement to read

   The read call will return one byte if the file is in binary mode.

Given that I *was *opening the file in "r" mode (which, I am assuming, is
"text" mode), then, if your statement is true, I have two more questions:

   1. If CR is a character and I open the file in text mode, then why does
the program not "see" the CR characters?

   2. Does the program not see CR characters under these circumstances
because they do not "count" as characters or for some other reason?

(You have probably spotted that this question 1 is virtually the same as my
original question.)

Stephen.














On Tue, Dec 10, 2019 at 10:39 AM Chris Angelico  wrote:

> On Tue, Dec 10, 2019 at 9:17 PM Stephen Tucker 
> wrote:
> >
> > I am running Python 2.7.10 on a Windows 10 machine. I have written a
> Python
> > program that counts occurrences of bytes in a text file and have run it
> on
> > a small test file. This file has CR-LF sequences at the ends of its
> lines.
> > The Python program counted the LF bytes, but didn't count any CR bytes.
> > Given that the documentation for the read method claims that exactly size
> > bytes are read by each call to the method (in this case, size is 1) does
> > this behaviour constitute a bug?
> >
>
> Are you trying to treat the file as text, or bytes? Are you looking
> for occurrences of bytes, or characters? Make a decision on that, then
> code accordingly. If you want to look for bytes, then open the file in
> binary mode and read it as bytes. If you want to look for characters,
> then you probably shouldn't care about which sort of line ending it
> is, but if you really do, then disable newline processing and examine
> the end of each logical line.
>
> The read call will return one character, if the file is in text mode.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

The file method read does not "see" CR

[Stephen Tucker]

I am running Python 2.7.10 on a Windows 10 machine. I have written a Python
program that counts occurrences of bytes in a text file and have run it on
a small test file. This file has CR-LF sequences at the ends of its lines.
The Python program counted the LF bytes, but didn't count any CR bytes.
Given that the documentation for the read method claims that exactly size
bytes are read by each call to the method (in this case, size is 1) does
this behaviour constitute a bug?

Stephen Tucker.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Syntax Suggestion: Pass Function Definition as Argument

[Stephen Waldron]

Ok firstly, this idea was inspired specifically by a project I'm working on for 
school concerning linked lists, in which I was trying to create a method that 
performed a function on elements iteratively without having to navigate the 
list from the head each time (of course taking the function as a parameter). 
This project is intended to be a personal module for future use. 

In the process of this discourse here, however, I think I've found an 
alternative for this specific issue, (so thanks guys) but generally speaking, 
there were other times I've wanted to use functions as parameters, typically 
while making personal modules, like a purely Js button module I had made a few 
years back, and it was apparent that the only way to do so in Python was to 
define first, then pass, which I simply didn't like the idea of in the context 
of a module, short of defining each function before the module instances with 
`def temp(*args):` or some other generic name I won't use past the module 
function instance.

Now yes, as many of you have pointed out, there may exist other workarounds for 
various instances in which one may think to use a function as an argument. 
Multi-line lambdas are a very intriguing method for the broad topic, (thanks, 
they will be useful in the future ^ ^b ++), but as @Chris Angelico mentioned, 
they can't really be used as replacements for def, and from what I've 
experienced trying them out, it wasn't always clear what could be done for some 
of the lines one would want to write.

Finally, some clarification for @David Raymond
> Isn't this basically just defining a function... but without an argument 
> list? How would that know what "str" is?

In my original post, I said:
> If they are not, they won't be assigned to, which would cause an error unless 
> a default value is given. These parameters could be available for passing in 
> the annexed function if mentioned when the parameter is called.

Nyeh, it's written poorly, but I guess I should clarify by saying that, per the 
proposal, arguments are inherent from the parent function to the annexed 
function if and only if that argument is assigned to when the annexed function 
is called as the parameter it fills in the parent function's definition. Of 
course it can also use all other variables within it's scope.

In the example you're referring to, this is done on line 3:
foo (str = str1)
making foo inherit "str" from myFoo, which was given the default value " ".

(I would suppose another viable alternative would be using nonlocal as I'll 
example below)

Also you ask if it's just defining a function. In all of my examples, I use 
only one instance of the parent function (which is actually pretty useless), 
but the power in doing this actually lies in calling it in various locations 
and passing different arguments and annexing different functions (kinda what 
functions are for, haha). I might be hard-pressed to think of an example short 
of writing out a more complex program, but here is.. something?.. I guess 
simply assume runOpt(prompt, foo) is a previously defined function, and 
variables used in the annex are within the annexed function's scope, but prompt 
is not inherited:


runOpt ("Play Game"):
getUserData()
chr[0].id = 0
time = 0
score = 0

scene = 1


runOpt ("How To Play"):
nonlocal prompt

loadBg()
display(prompt)

scene = 2


runOpt ("Settings"):
checkDefaults()
chr[0].id = 5

scene = 3





N.B. on everything (also some TLs you could skip): 
This is just a suggestion, I think it would be nice for previously stated 
reasons (ease, aesthetics, function, yadda-yadda). I would still love to hear 
ways I could currently achieve things similar, as that is one of the things I 
look for from suggestions since "I don't know how to do x right now and have 
not previously found a solution" is implied, but one trigger I do have is a 
reply of "that is not the way it is" to a "this is how I think it should be" 
statement; doesn't give me any hint as to the merits or demerits of my thought, 
as flawed as it may indeed be, in logistics, implementation, practice etc. 
Can't really say anyone has done that quite yet, but just making it known, so 
"What you can try" and "The suggestion is impractical because..." responses are 
welcome. 

Also, one of the reasons for my suggestions is ease. The structure used is 
fairly common in Python, and thus fairly intuitive (I hypothesize at least). I 
know from experience that once you learn something, it becomes fairly easy and 
intuitive for you, but not necessarily for others and you sometimes forget the 
struggle you endured to get there. Of course I'm not saying others shouldn't 
have to learn to, but if there's a way to make it easier for others, I'd push 
it. Now I am still going to learn all about async, await, yield (which I think 
deserves more cred from what I can see so far), and lambda (cuz I wanna) but if 
I think learning and utili

Re: Syntax Suggestion: Pass Function Definition as Argument

[Stephen Waldron]

Thanks Antoon. I do suppose that it is kind of wrong to say the only way is to 
"reference its [the function's] name" as an argument, however the point I was 
trying to make was that it isn't possible to pass a function that is either not 
in some way previously defined or a reference to something previously defined.

In your example, you still make reference to the existing function `truediv` 
from the `operator` library, even though the `partial` function does create a 
new function.

What I'm aiming for is the ability to, within a function call, pass a suite 
that would be there automatically defined by the compiler/interpreter. Another 
comment did mention lambda functions, which does to some degree provide that 
capability, but is restricted to well, lambda functions (only expressions, not 
statements).

Your example does however suggest the possibilities of a function or expression 
that creates functions. For example, the `function` expression in JavaScript.

```
//Definition
function myFoo (str1, str2, foo){
console.log( foo(str1), foo(str2) );
}


//Call
myFoo ("hello", "world!", function(str){

str[0] = str[0].toUpper();
return str;

});


Output:
Hello World!
```

However it does not seem that there would be any such expression in Python as 
is. In python, this code would be written:


```
#Definition
def myFoo (str1, str2, foo):
print( foo(str1), foo(str2) )


#Call
def other_foo(str):
str = list(str)[0].upper() + str[1:]
return str

myFoo ("hello", "world!", other_foo)

```


Here is it rewritten using the proposal:
```
#Definition
def myFoo (str1, str2, foo, str = " "):
print( foo(str = str1), foo(str = str2) )


#Call
myFoo ("hello", "world!"):
str = list(str)[0].upper() + str[1:]
return str

```

Of course this example presents only a minor, though you can see the difference 
in the call, in which no new function needs be referenced.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Syntax Suggestion: Pass Function Definition as Argument

[Stephen Waldron]

Thanks Antoon. I do suppose that it is kind of wrong to say the only way is to 
"reference its [the function's] name" as an argument, however the point I was 
trying to make was that you cannot pass a function that is either not in some 
way previously defined or a reference to something previously defined.

In your example, you still make reference to the existing function `truediv` 
from the `operator` library, even though the `partial` function does create a 
new function.

What I'm aiming for is the ability to, within a function call, pass a suite 
that would be there automatically defined by the compiler/interpreter. Another 
comment did mention lambda functions, which does to some degree provide that 
capability, but is restricted to well, lambda functions (only expressions, not 
statements).

Your example does however suggest the possibilities of a function or expression 
that creates functions. For example, the `function` expression in JavaScript.

```
//Definition
function myFoo (str1, str2, foo){
console.log( foo(str1), foo(str2) );
}


//Call
myFoo ("hello", "world!", function(str){

str[0] = str[0].toUpper();
return str;

});


Output:
Hello World!
```

However it does not seem that there would be any such expression in Python as 
is. In python, this code would be written:


```
#Definition
def myFoo (str1, str2, foo):
print( foo(str1), foo(str2) )


#Call
def other_foo(str):
str = list(str)[0].upper() + str[1:]
return str

myFoo ("hello", "world!", other_foo)

```


Here is it rewritten using the proposal:
```
#Definition
def myFoo (str1, str2, foo, str = " "):
print( foo(str = str1), foo(str = str2) )


#Call
myFoo ("hello", "world!"):
str = list(str)[0].upper() + str[1:]
return str

```

Of course this example presents only a minor, though you can see the difference 
in the call, in which no new function needs be referenced.
-- 
https://mail.python.org/mailman/listinfo/python-list

Syntax Suggestion: Pass Function Definition as Argument

[Stephen Waldron]

Hi, I'm new to the group and to Python, so forgive me if I make any faux-pas 
here. As I can tell, the only way to pass a function as an argument is to 
reference its name as follows:

def foo1(message):
print(message)

def foo2(foo, message):
print("Your function says:")
foo(message)


>>foo2(foo1, "Hello World!")
Your function says:
Hello World!


This is how it is at the moment, however it may be more agreeable, especially 
if that is the only purpose of the function, for python users to be able to 
define new functions inside of function calls. The way I propose this may occur 
is using similar syntax as control structures, as exampled below:

def foo2(foo):
print("Message to print:", end = " ")
foo()


>>foo2():
print("Hello World!")

Message to print: Hello World!


In this proposal, it should be noted, that unlike control structures, the 
brackets are required for the function call. The "suite" or "annex" is defined 
as a function object by the compiler and passed inside the call. Here are some 
other examples of the proposal.



*Non-Function Arguments with Annexed Functions*
Non-Function Arguments should be placed in the brackets before annexed 
functions, and this should be reflected in the parameter order. If they are 
not, they won't be assigned to, which would cause an error unless a default 
value is given. These parameters could be available for passing in the annexed 
function if mentioned when the paramaeter is called.

def myFoo (num1, num2, foo):
return foo (num1, num2)


>>myFoo(20, 30):
return num1 + num2

50


*Annexing Multiple Functions or Out of Order Annex*
To specify the order in which functions are annexed, or annex multiple 
functions, one could use the parameter name before the specific suite.

class Book:
def __init__(self, name, author):
self.name = name
self.author = author


def doToBooks (bookList, first, then, book = None):

for book in bookList
first(book = book)
then(book = book)

myBooks = [
Book("The Way of the World", "Lucy Cole"),
Book("To Live or To Love", "Georgio Dunham"),
Book("Twelve Days of Success", "Anita Duvette")
]

doToBooks (myBooks):
print(book.name)
then:
print(" - " + book.author + "\n")



>
The Way of The World
 - Lucy Cole

To Live or To Die
 - Georgio Dunham

Twelve Days of Success
 - Anita Duvette



This proposal does not detract from the core appearance and appeal of Python 
while creating an option for users and module makers who want themselves or the 
users of their modules not to have to previously define functions in order to 
use the code. This would make such code more aesthetic and compact, and take 
away the need for defining a new function, name and all, specifically for such 
which ultimately would be used only once as an argument. 
   
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Difference between array( [1,0,1] ) and array( [ [1,0,1] ] )

[Stephen Tucker]

Markos,

I can explain the difference from a non-numpy point of view - I hope you
will be able to see how this difference affects what you are trying to do
in numpy.

vector_1 is an np.array consisting of a three-element list, with the three
elements being 1, 0 and 1.

vector_2 is an np.array consisting (at the top level) of a one-element
list, with that element (at this top level) being a three-element list,
with these three elements (at the lower level) being 1, 0 and 1.

Stephen.




On Fri, Jun 21, 2019 at 7:29 AM Markos  wrote:

>
> Hi,
>
> I'm studying Numpy and I don't understand the difference between
>
> >>> vector_1 = np.array( [ 1,0,1 ] )
>
> with 1 bracket and
>
> >>> vector_2 = np.array( [ [ 1,0,1 ] ] )
>
> with 2 brackets
>
> The shape of vector_1 is:
>
> >>> vector_1.shape
> (3,)
>
> But the shape of vector_2 is:
>
> >>> vector_2.shape
> (1, 3)
>
> The transpose on vector_1 don't work:
>
> >>> vector_1.T
> array([1, 0, 1])
>
> But the transpose method in vector_2 works fine:
>
> >>> vector_2.T
> array([[1],
> [0],
> [1]])
>
>
> I thought that both vectors would be treated as an matrix of 1 row and 3
> columns.
>
> Why this difference?
>
> Any tip?
>
> Thank you,
> Markos
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question about floating point

[Stephen Tucker]

Hi Frank,

One difference is that, in order to carry out the instructions embodied in
the first example, the computer has to perform arithmetic. And it adds the
binary approximation of 1.1 (which is very slightly wrong) to the binary
approximation of 2.2 (which, again, is very slightly wrong). It then prints
the denary equivalent to the sum of those two which happens to be very
slightly more than 3.3.

When you type 3.3, it stores that as the binary approximation of 3.3 (which
is y at that stage, to answer your question, and is, again, no surprise,
also very slightly wrong) and then prints the denary equivalent of that
binary approximation which happens to end up sufficiently close to 3.3 so
as to cause it to print 3.3.

I hope that helps.

The experts around here (of which I am not one) may well point you to the
decimal  package which allows better handling of such arithmetic, if you
are want the computer to behave more like you would want it to behave.

Regards,

Stephen Tucker.








On Tue, Aug 28, 2018 at 3:11 PM, Frank Millman  wrote:

> Hi all
>
> I know about this gotcha -
>
> x = 1.1 + 2.2
>>>> x
>>>>
>>> 3.3003
>
> According to the docs, the reason is that "numbers like 1.1 and 2.2 do not
> have exact representations in binary floating point."
>
> So when I do this -
>
> y = 3.3
>>>> y
>>>>
>>> 3.3
>
> what exactly is happening? What is 'y' at this point?
>
> Or if I do this -
>
> z = (1.1 + 2.2) * 10 / 10
>>>> z
>>>>
>>> 3.3
>
> What makes it different from the first example?
>
> Thanks
>
> Frank Millman
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Learning Python

[Stephen Tucker]

I have found Learning Python by Mark Lutz helpful. I have the 4th edition
(2009). Its ISBN is 978-0-596-15806-4.

A lot will depend on your learning style. This book reads more like a set
of course notes than a reference book, but it does contain tables and
summaries, too.

On Tue, Jun 5, 2018 at 5:51 PM, T Berger  wrote:

> Can someone learn Python through a book such as Head Start Python? Would
> an online course from MIT or google be better?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Ruby parens-free function calls [was Re: Accessing parent objects]

[Stephen Hansen]

On Mon, Mar 26, 2018, at 2:19 PM, Rick Johnson wrote:
>Sure, the behavior that Steven
> uncovered is odd, but it could be that Maz harbors a strong
> disliking for undisciplined pupils, and thus, he designed
> and placed this little trap in the hopes the pain it induced
> might encourage the petulant little meat-heads to follow
> some sensible styling rules.

My god, I've been away from this list for quite awhile, but we're still 
entertaining this fool? 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: integer copy

[Stephen Tucker]

ast,

For what it's worth,

After

a = 5
b = 5
afloat = float(a)
bfloat = float(b)

afloat is bfloat

returns False.

Stephen Tucker.


On Fri, Oct 20, 2017 at 9:58 AM, ast  wrote:

>
> "ast"  a écrit dans le message de
> news:59e9b419$0$3602$426a7...@news.free.fr...
>
> Neither works for large integers which is
> even more disturbing
>
> a = 6555443
>
> b = copy.copy(a)
> a is b
>
> True
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python programming language vulnerabilities

[Stephen Michell]

CORRECTION.
My sincere apologies to anyone that tried the link that I posted. The actual 
link is

www.open-std.org/jtc1/sc22/wg23

follow the link to documents, or go directly there via

www.open-std.org/jtc1/sc22/wg23/docs/documents.html

I was informed that there are some broken links to documents. I believe that 
they are all fixed now.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python programming language vulnerabilities

[Stephen Michell]

My apologies. I maintain that website.

There should have been no broken links. I will fix that.

The previous version of TR 24772 had annexes for language-specific material. We 
have split those out, so the main document (Tr 24772-1) only has language 
independent material. The last Python document is N0702 at 
open-std.org/sc22/wg23//docs/documents.html. This document was one that Serihy 
could not access. That link is fixed, so it can be accessed now.


-- 
https://mail.python.org/mailman/listinfo/python-list

Python programming language vulnerabilities

[Stephen Michell]

I chair ISO/IEC/JTC1/SC22/WG23 Programming Language Vulnerabilities. We publish 
an international technical report, ISO IEC TR 24772 Guide to avoiding 
programming language vulnerabilities through language selection use. Annex D in 
this document addresses vulnerabilities in Python. This document is freely 
available from ISO and IEC.

We are updating this technical report, adding a few vulnerabilities and 
updating language applicability as programming languages evolve. We are also 
subdividing the document by making the language-specific annexes each their own 
technical report. For the Python Part, the major portions are written, but we 
have about 6 potential vulnerabilities left to complete.

We need help in finishing the Python TR. We are looking for a few Python 
experts that have experience in implementing Python language systems, or 
experts in implementing significant systems in Python (for technical level, 
persons that provide technical supervision to implementers, or that write and 
maintain organizational Python coding standards.

If you are interested in helping, please reply to this posting.

Thank you
Stephen Michell
Convenor, ISO/IEC/JTC 1/SC 22/WG 23 Programming Language Vulnerabilities
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Stephen Tucker]

Sam,

You use

f.write(r'hello\tworld')

The r in front of the string stands for raw and is intended to switch off
the escape function of the backslash in the string. It works fine so long
as the string doesn't end with a backslash, as in

f.write('hello\tworld\')

If you try this, you get an error message.

Stephen.



<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Jul 5, 2017 at 3:37 PM, Sam Chats  wrote:

> I want to write, say, 'hello\tworld' as-is to a file, but doing
> f.write('hello\tworld') makes the file
> look like:
> hello   world
>
> How can I fix this? Thanks in advance.
>
> Sam
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Stephen Tucker]

Sam,

You use

r'hello\tworld'

The r in front of the string stands for raw and it is intended to switch
off the normal escape function of a backslash. It works fine so long as the
string doesn't end with a backslash. If you end the string with a
backslash, as in

r'hello\tworld\'

you get an error message.

Stephen.


<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Jul 5, 2017 at 3:37 PM, Sam Chats  wrote:

> I want to write, say, 'hello\tworld' as-is to a file, but doing
> f.write('hello\tworld') makes the file
> look like:
> hello   world
>
> How can I fix this? Thanks in advance.
>
> Sam
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to add new tuple as key in dictionary?

[Stephen Tucker]

Hello there, Ho Yeung Lee,

The use of groupkey[tuple([0,3])] to the right of the = is attempting to
access an entry in your dictionary that you have not already created. That
is why you are getting the error message.
You need to create an entry whose key is tuple([0,3]) before you can access
it. The following code does not provoke an error message:

groupkey = {(0,0): []}
groupkey = {tuple([0,3]): []}
groupkey[tuple([0,3])] = groupkey[tuple([0,3])] + [[0,1]]

I hope this helps you.

Stephen.




<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Fri, Jun 30, 2017 at 11:35 AM, Ho Yeung Lee  wrote:

> I find that list can not be key in dictionary
> then find tuple can be as key
>
> but when I add new tuple as key , got error in python 2.7
>
> groupkey = {(0,0): []}
> groupkey[tuple([0,3])] = groupkey[tuple([0,3])] + [[0,1]]
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Unhelpful error message

[Stephen Tucker]

Hi,

I have just been thrown through an unecessary loop because of an unhelpful
error message.

I am running Python 2.7.10 on a Windows 10 machine.

I incorporate sample output from IDLE:

~~~

>>> print float ("123.456")
123.456
>>> print float ("")

Traceback (most recent call last):
  File "", line 1, in 
print float ("")
ValueError: could not convert string to float:
>>>

~~~
The unhelpful message is the response that the system gives when you try to
convert a null string to a floating point number. Now, in the example I
have given here, it is obvious that the string is null, but in the example
that threw me into a loop, it was not obvious.

It would be more helpful is the system replied:

ValueError: could not convert null string to float

Any chance of that becoming the case?

Stephen Tucker.


<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Help Please ! Undocumented ERROR message so dont know how to fix the problem

[Stephen Tucker]

To murdock: What Rhodri wrote is correct. I sense that it might be helpful
for you if I were to tell you that there is a difference between a function
and a function call. If your function were named

MyFunction

then

print (MyFunction)

would print a user-friendly-ish message about the function. This message
tells you (a) that the object you are asking Python to print is a function,
and (b) the address (in hexadecimal) in memory where the code for
MyFunction is stored. This is what your code is doing - hence the message
from Python.

If, on the other hand you have

print (MyFunction(any-parameters-to-MyFunction-with-commas-between-them))

then Python would recognise that (a) you are wanting it to call MyFunction
with the specified parameters, if any, (b) you are expecting that the
function will return a printable result, and (c) you are also expecting it
to print that result.

So, in order to get Python to do that, all you need to do is:

(1) to add the parentheses after the function name in the print statement [
so print (MyFunction) becomes print (MyFunction()) ];

(2) If your function has any parameters, add them, separated with commas,
between the parentheses [ so print (MyFunction()) becomes print
(MyFunction(parameters-separated-with-commas)) ];

(3) and rerun the new code - which should then do what you are wanting it
to do.

I hope this helps.

Stephen Tucker.









<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Tue, May 2, 2017 at 2:11 PM, Rhodri James  wrote:

> On 02/05/17 03:57, murdock wrote:
>
>> I am having a problem that seems to persist. I have written a program
>> that makes a mathematical calculation and uses a uses library that I have
>> written. It had been working but somehow in playing around with it, it
>> stoppedgo figure!  But here is the thing, when I run the program it
>> gives me a very ambiguous message with only a string as in:
>>
>> "The Receiver Noise Figure =  > 0x063E5A60>  dBm"
>>
>
> This is not an error message.  This is Python faithfully printing out what
> you asked it to print out.  Here's the line that does the printing:
>
> print ("The Receiver Noise Figure = ",Hamath._Noise_Figure," dBm" )
>
> The weird-looking bit of the message is the representation of a function,
> and lo! in the parameters to print() there is the function
> Hamath._Noise_Figure.  Note that this is the function itself, *not* the
> result of calling it, since you *don't* call it.
>
> --
> Rhodri James *-* Kynesim Ltd
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: calling a program from Python batch file

[Stephen Tucker]

This might be totally irrelevant, but, if (a) the data to be read by the
program during a given run is known when the program is run/launched, (b)
that data is purely textual and (c) that data can be read by the program
from the stdin channel, then my idea is (1) before the launch, put that
data into a file named, say, inputfile.dat, and then (2)  launch the
program with a command like

call program.bat outputfile.dat

Finally, if the program can read its data from a given file and possibly
needs to output its data, again, to a named file, and those files' names
can be read from the parameters to the call, then

call program.bat inputfile.dat outputfile.dat

can be used.








On Wed, Dec 7, 2016 at 6:23 PM, Michael Torrie  wrote:

> On 12/07/2016 10:59 AM, John Gordon wrote:
> > In  Karim
> Farokhnia  writes:
> >
> >> Hi there,
> >
> >> I am writing a batch file in Python. The batch file, in part, calls a
> >> program named "oq-console.bat" to run. Then once the program comes up
> >> (it looks like windows CMD), I need the batch file to type some commands
> >> to make it run (just like I would do it by myself).
> >
> > If you need to start up a program, provide interactive input to it, and
> > perhaps examine its interactive output, then you want the "pexpect"
> module:
> >
> > Pexpect is a pure Python module for spawning child applications;
> > controlling them; and responding to expected patterns in their
> output.
> > Pexpect allows your script to spawn a child application and control
> it
> > as if a human were typing commands.
> >
> > https://pexpect.readthedocs.io/en/stable/
>
> Does Pexpect work on Windows?
>
> In the OP's case it looks like the standard in pipe is sufficient.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Quick way to calculate lines of code/comments in a collection of Python scripts?

[Stephen Tucker]

Tomasz,

How about using the command prompt command FIND /C  on each of your source
files as follows:

FIND/C "#" >NumbersOfLinesContainingPythonComments.dat
FIND/C /V "#" >NumbersOfLinesNotContainingPythonComments.dat

You would end up with two files each with a column of line counts;

Import these lines into an Excel Spreadsheet and calculate whatever you
like with them.

Stephen.



On Sun, Oct 23, 2016 at 9:51 PM, Tomasz Rola  wrote:

> On Wed, Oct 05, 2016 at 01:56:59PM -0400, Malcolm Greene wrote:
> > Looking for a quick way to calculate lines of code/comments in a
> > collection of Python scripts. This isn't a LOC per day per developer
> > type analysis - I'm looking for a metric to quickly judge the complexity
> > of a set of scripts I'm inheriting.
> >
> > Thank you,
> > Malcolm
>
> A bit more than what you asked for (and sorry for being late) but I
> find sloccount quite good. Or at least interesting (computes sloc and
> some stats about project, given project dir or a single file with
> code):
>
> http://www.dwheeler.com/sloccount/
>
> --
> Regards,
> Tomasz Rola
>
> --
> ** A C programmer asked whether computer had Buddha's nature.  **
> ** As the answer, master did "rm -rif" on the programmer's home**
> ** directory. And then the C programmer became enlightened...  **
> ** **
> ** Tomasz Rola  mailto:tomasz_r...@bigfoot.com **
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Doubled backslashes in Windows paths

[Stephen Tucker]

Hi Oz,

This might only be tangential to your actual issue, but, there again, it
might be the tiny clue that you actually need.

In Python, I use raw strings and single backslashes in folder hierarchy
strings to save the problem of the backslash in ordinary strings. Even with
this policy, however, there is a slight "gotcha": Although it is claimed
that r" ... " suspends the escape interpretation of the backslash in the
string, a raw string cannot end with a backslash:

   myraw = "\a\b\"

provokes the error message:

   SyntaxError: EOL while scanning string literal

To see how well this approach deals with folder hierarchies with spaces in
their names, I created the following file:

c:\Python27\ArcGIS10.4\Lib\idlelib\sjt\sp in\test.txt

Note the space in the folder name  sp in .

In IDLE, I then issued the following statement:

infile= open (r"c:\Python27\ArcGIS10.4\Lib\idlelib\sjt\sp in\test.txt", "r")

Note that I didn't need to get enclosing quotes into my folder hierarchy
string. It begins with  c  and ends with  t .

The statement worked as you might expect, and granted me access to the
lines in the file.

It seems that it is only necessary to enclose a folder hierarchy string in
quotes when defining the string in a situation in which spaces would
otherwise be interpreted as terminators. The classis case of this is in a
command with parameters in a batch file. If the string has been defined
before it is presented to the Windows Command interpreter, the spaces are
accepted as part of it without the need then of enclosing quotes.

Hope this helps.

Yours,

Stephen Tucker.


On Fri, Oct 7, 2016 at 6:30 AM, Oz-in-DFW  wrote:

> I'm using Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC
> v.1900 32 bit (Intel)] on Windows 7
>
> I'm trying to write some file processing that looks at file size,
> extensions, and several other things and I'm having trouble getting a
> reliably usable path to files.
>
> The problem *seems* to be doubled backslashes in the path, but I've read
> elsewhere that this is just an artifact of the way the interpreter
> displays the strings.
>
> I'm getting an error message on an os.path.getsize call;
>
> Path: -
> "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg" -
> Traceback (most recent call last):
>   File "C:\Users\Rich\workspace\PyTest\test.py", line 19, in 
> if os.path.getsize(path)>1:
>   File "C:\Python32\lib\genericpath.py", line 49, in getsize
> return os.stat(filename).st_size
> WindowsError: [Error 123] The filename, directory name, or volume
> label syntax is incorrect:
> '"C:\\Users\\Rich\\Desktop\\2B_Proc\\2307e60da6451986dd8d23635b8453
> 86.jpg"'
>
> From (snippet)
>
> path = '"'+dirpath+name+'"'
> path = os.path.normpath(path)
> print("Path: -",path,"-")
> if os.path.getsize(path)>1:
> print("Path: ",path," Size:
> ",os.path.getsize(dirpath+name))
>
> but if I manually use a console window and cut and paste the path I
> print, it works;
>
> C:\>dir
> "C:\Users\Rich\Desktop\2B_Proc\2307e60da6451986dd8d23635b845386.jpg"
>  Volume in drive C is Windows7_OS
>
>  Directory of C:\Users\Rich\Desktop\2B_Proc
>
> 10/03/2016  08:35 AM59,200
> 2307e60da6451986dd8d23635b845386.jpg
>1 File(s) 59,200 bytes
>0 Dir(s)  115,857,260,544 bytes free
>
> So the file is there and the path is correct. I'm adding quotes to the
> path to deal with directories and filenames that have spaces in them.
> If I drop the quotes, everything works as I expect *until* I encounter
> the first file/path with spaces.
>
> I'll happily RTFM, but I need some hints as to which FM to R
>
> --
> mailto:o...@ozindfw.net
> Oz
> POB 93167
> Southlake, TX 76092 (Near DFW Airport)
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: degrees and radians.

[Stephen Tucker]

For what it's worth, mathematicians naturally work with angles in radians.

The mathematics of the trignonmetric functions works naturally when the
angle is expressed in radians.

For the older among us, logarithms also have a "natural" base, and that is
the number e. Back in those days, however, even the mathematicians would
use logarithm tables where the values were shown to base 10.


On Wed, Aug 24, 2016 at 6:26 AM, Steven D'Aprano <
steve+comp.lang.pyt...@pearwood.info> wrote:

> On Wednesday 24 August 2016 14:26, Gary Herron wrote:
>
> > Do you really need anything more complex than this?
> >
> >  >>> toRadians = math.pi/180.0
> >
> >  >>> math.sin(90*toRadians)
> > 1.0
> >
> > Perhaps I'm not understanding what you mean by "clunky",  but this seems
> > pretty clean and simple to me.
>
> The math module has two conversion functions, math.radians() and
> math.degrees().
>
>
> Some other languages (Julia, by memory, and perhaps others) have dedicated
> sind(), cosd(), tand() or possibly dsin(), dcos(), dtan() functions which
> take
> their argument in degrees and are more accurate than doing a conversion to
> radians first. I'd like to see that.
>
> I've also seen languages with sinp() etc to calculate the sine of x*pi
> without
> the intermediate calculation.
>
> But if I were designing Python from scratch, I'd make sin(), cos() and
> tan()
> call dunder methods __sin__ etc:
>
>
> def sin(obj):
> if hasattr(type(obj), '__sin__'):
> y = type(obj).__sin__()
> if y is not NotImplemented:
> return y
> elif isinstance(obj, numbers.Number):
> return float.__sin__(float(obj))
> raise TypeError
>
> Likewise for asin() etc.
>
> Then you could define your own numeric types, such as a Degrees type, a
> PiRadians type, etc, with their own dedicated trig function
> implementations,
> without the caller needing to care about which sin* function they call.
>
>
>
>
> --
> Steve
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Self Learning Fortran Programming

[Stephen Tucker]

Hi, there, Muhammad,

I have found Learning Python by Mark Lutz helpful. The fourth edition
covers both Python 2.6 and 3.x. Although it is a text book for a course
that Mark delivers, there are useful summaries of the various functions and
methods for strings, integers, etc at various spots in the book.

Stephen Tucker.

<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avast.com
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Jun 1, 2016 at 6:47 AM, Joel Goldstick 
wrote:

> start here:
>
>
> https://www.python.org/about/gettingstarted/
>
> On Wed, Jun 1, 2016 at 12:50 AM, Muhammad Ali
>  wrote:
> >
> > Hello,
> >
> > I am interested in Python programming, however, it will be my first
> serious attempt towards coding/simulation/programming. My back ground is
> Physics, no practical experience with programming languages.
> >
> > So, this post is for the valuable suggestions from the experts that how
> can I start self learning Python from scratch to advanced level in minimum
> time. For this, please recommend Python version, literature, text books,
> websites, video lectures, your personnel tips, etc. In addition, you may
> also add some extra suggestions for shell script writing as well. You may
> recommend for both Linux and Windows operating systems.
> >
> > Looking for your posts.
> >
> > Thank you.
> > --
> > https://mail.python.org/mailman/listinfo/python-list
>
>
>
> --
> Joel Goldstick
> http://joelgoldstick.com/blog
> http://cc-baseballstats.info/stats/birthdays
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for / while else doesn't make sense

[Stephen Hansen]

On Thu, May 19, 2016, at 07:02 PM, gst wrote:
> Python 4.0 ? My son will thank us !

No, he won't, because while Python 4.0 will happen, likely after Python
3.9, it will not be a major backwards compatible breaking point.

Some people infer that because 3.0 was, 4.0, 5.0, 6.0 are open to it.
They aren't.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Steve D'Aprano, you're the "master". What's wrong with this concatenation statement?

[Stephen Hansen]

On Tue, May 10, 2016, at 09:16 AM, DFS wrote: 
> But no, "master", your answers are incorrect.  What's wrong with that 
> concat statement is that += concatenation is frowned upon by python's 
> creator, and is not recommended (in PEP8):
> 
> 
> Code should be written in a way that does not disadvantage other 
> implementations of Python (PyPy, Jython, IronPython, Cython, Psyco, and 
> such).
> 
> For example, do not rely on CPython's efficient implementation of 
> in-place string concatenation for statements in the form a += b or a = a 
> + b . This optimization is fragile even in CPython (it only works for 
> some types) and isn't present at all in implementations that don't use 
> refcounting. In performance sensitive parts of the library, the 
> ''.join() form should be used instead. This will ensure that 
> concatenation occurs in linear time across various implementations.
> 

You once again misread PEP8. 

Not one of Steven's answers used string concatenation, except for the
implicit literal concatenation, which all other implementations support.

Note:

> sSQL = """line 1
> line 2
> line 3"""

No concatenation. One string literal. Works in all implementations.

> sSQL = "line 1\nline 2\nline 3"

No concatenation. One string literal. Works in all implementations.

>
> sSQL = ("line 1\n"
> "line 2\n"
> "line 3")

Concatenation occurs at compile time, implicitly. Works in all
implementations.

The PEP says when building a string dynamically (that is, adding
together two strings that exist and are separate), use "".join() in
performance sensitive places. It says don't combine two strings with the
addition operator -- because while that's sometimes efficient in
CPython, the language doesn't guarantee it.

You'll notice that the one time Steven combined two strings, it was
implicitly at compile time.

In every Python implementation, it is guaranteed that:

>>> a = "a" "a"
>>> b = "aa"

Are the same. Two+ string literals are implicitly combined into one at
compile time.

> 'master craftswoman' my ass...

Yes, you're being that. Please stop.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What should a decorator do if an attribute already exists?

[Stephen Hansen]

On Tue, May 10, 2016, at 08:45 AM, Steven D'Aprano wrote:
> I have a decorator that adds an attribute to the decorated function:
> [...]
> My question is, what should I do if the decorated function already has an
> instrument attribute?
> 
> 1. raise an exception?

This. Your decorator should, IMHO, treat the attribute as private data,
and if something else is using the same thing, something has clearly
gone wrong and raising the error early and clearly is right.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sun, May 8, 2016, at 02:46 PM, DFS wrote:
> On 5/8/2016 5:38 PM, Stephen Hansen wrote:
> > On Sun, May 8, 2016, at 02:16 PM, DFS wrote:
> >> I was surprised to see the PEP8 guide approve of:
> >>
> >> "Yes: if x == 4: print x, y; x, y = y, x"
> >>
> >> https://www.python.org/dev/peps/pep-0008/#pet-peeves
> >
> > That is not approving of that line of code as something to mimic, its
> > speaking *only* about *whitespace*.
> >
> > ALL its saying is, "don't put spaces before commas, colons or
> > semicolons". You can infer nothing else about it.
> 
> I can infer that it's 100% approved of, since he used it as an example.

Not if you don't want to be a fool.

And at this point I'm signing out from helping you.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sun, May 8, 2016, at 02:16 PM, DFS wrote:
> I was surprised to see the PEP8 guide approve of:
> 
> "Yes: if x == 4: print x, y; x, y = y, x"
> 
> https://www.python.org/dev/peps/pep-0008/#pet-peeves

That is not approving of that line of code as something to mimic, its
speaking *only* about *whitespace*.

ALL its saying is, "don't put spaces before commas, colons or
semicolons". You can infer nothing else about it.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sun, May 8, 2016, at 08:06 AM, DFS wrote:
> On 5/8/2016 10:36 AM, Chris Angelico wrote:
> > ... and then you just commit???!?
> >
> 
> That's what commit() does.
> 

I assure you, he knows what commit does :)

The point is, you don't usually commit after an error happens. You
rollback. Or correct the data. Since the data didn't go in, there should
(in theory) be nothing TO commit if an error happens. Or, there should
be partial data in that needs a rollback before you decide to do
something else.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sun, May 8, 2016, at 07:25 AM, DFS wrote:
> for nm,street,city,state,zipcd in zip(nms,street,city,state,zipcd):

> > for vals in zip(nms,street,city,state,zipcd):
> > nm,street,city,state,zipcd = vals
> > cSQL = "INSERT INTO ADDRESSES VALUES (?,?,?,?,?)"
> 
> 
> I like the first one better.  python is awesome, but too many options 
> for doing the same thing also makes it difficult.  For me, anyway.

Eeh, Now you're just making trouble for yourself.

for name, street, city, state, zipcd in zip(names, streets, cities,
states, zipcds):


may be sorta vaguely long, but its not that long. Just do it and move
on. Get over whatever makes you not like it. 


-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sat, May 7, 2016, at 08:28 PM, DFS wrote:
> >> +-++
> >> |superfluous-parens   |3   | I like to surround 'or'
> >>  statments with parens
> >
> > I would need examples to comment
> 
> 
> if ("Please choose a state" in str(matches)):
> if (var == "val" or var2 == "val2"):

Gah, don't do that. You're adding meaningless noise. 

Especially in the first case.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sat, May 7, 2016, at 08:04 PM, DFS wrote:
> The lists I actually use are:
> 
> for j in range(len(nms)):
>   cSQL = "INSERT INTO ADDRESSES VALUES (?,?,?,?,?)"
>   vals = nms[j],street[j],city[j],state[j],zipcd[j]
> 
> 
> The enumerated version would be:
> 
> ziplists = zip(nms,street,city,state,zipcd)
> for nm,street,city,state,zipcd in ziplists:
>   cSQL = "INSERT INTO ADDRESSES VALUES (?,?,?,?,?)"
>   vals = nm,street,city,state,zipcd
> 
> 
> I guess the enumeration() is a little nicer to look at.  Why do you 
> think it's more maintainable?

Code is read more then its written.

That which is nicer to look at, therefore, is easier to read.

That which is easier to read is easier to maintain.

Beyond that, its simpler, and more clearly articulates in the local
space what's going on. 

> Aside: I haven't tried, but is 'names' a bad idea or illegal for the 
> name of a python list or variable?

Nothing wrong with names. Or 'name', for that matter. Try to avoid
abbreviations.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sat, May 7, 2016, at 06:16 PM, DFS wrote:

> Why is it better to zip() them up and use:
> 
> for item1, item2, item3 in zip(list1, list2, list3):
>   do something with the items
> 
> than
> 
> for j in range(len(list1)):
> do something with list1[j], list2[j], list3[j], etc.

Although Chris has a perfectly good and valid answer why conceptually
the zip is better, let me put forth: the zip is simply clearer, more
readable and more maintainable.

This is a question of style and to a certain degree aesthetics, so is
somewhat subjective, but range(len(list1)) and list1[j] are all
indirection, when item1 is clearly (if given a better name then 'item1')
something distinct you're working on.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pylint prefers list comprehension over filter...

[Stephen Hansen]

On Sat, May 7, 2016, at 12:17 PM, Christopher Reimer wrote:
> On 5/5/2016 6:37 PM, Stephen Hansen wrote:
> > On Thu, May 5, 2016, at 06:26 PM, Christopher Reimer wrote:
> >> Which is one is correct (Pythonic)? Or does it matter?
> > First, pylint is somewhat opinionated, and its default options shouldn't
> > be taken as gospel. There's no correct: filter is fine.
> 
> Since the code I'm working on is resume fodder (i.e., "Yes, I code in 
> Python! Check out my chess engine code on GitHub!"), I want it to be as 
> Pythonic and PEP8-compliant as possible. That includes scoring 10/10 
> with pylint. Never know when an asshat hiring manager would reject my 
> resume out of hand because my code fell short with pylint.
> 
> For my purposes, I'm using the list comprehension over filter to keep 
> pylint happy.

Bear in mind, when I say, "Pylint is opinionated", I mean the tool --
especially in its default configuration -- has its own opinion of what
is good style, and *I think its wrong on a number of points*. 

Its fine to use, but I'd read over PEP8 (the document, not the tool) and
apply style guide recommendations thoughtfully, not mechanically. 

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

On Sat, May 7, 2016, at 11:52 AM, Christopher Reimer wrote:
> You can do better.  You should strive for 10/10 whenever possible, 
> figure out why you fall short and ask for help on the parts that don't 
> make sense.

I think this is giving far too much weight to pylint's opinion on what
is "good" or "bad" programming habits.

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pylint woes

[Stephen Hansen]

Pylint is very opinionated. Feel free to adjust its configuration to
suit your opinions of style.

In particular, several of these might be related to PEP8 style issues.

On Sat, May 7, 2016, at 09:51 AM, DFS wrote:
>DFS comments
> +-++ ---
> |message id   |occurrences |
> +=++
> |mixed-indentation|186 | I always use tab

And yet, it appears there's some space indentation in there. In
Notepad++ enable View->Show Symbol->Show White Space and Tab and Show
Indent Guide.

> +-++
> |invalid-name |82  | every single variable name?!

It probably defaults to PEP8 names, which are variables_like_this, not
variablesLikeThis.

> +-++
> |bad-whitespace   |65  | mostly because I line up =
>   signs:
>   var1  = value
>   var10 = value

Yeah and PEP8 says don't do that. Adjust the configuration of pylint if
you want.

> +-++
> |multiple-statements  |23  | do this to save lines.
>   Will continue doing it.

This you really shouldn't do, imho. Saving lines is not a virtue,
readability is -- dense code is by definition less readable.

> +-++
> |no-member|5   |
> 
> "Module 'pyodbc' has no 'connect' member"   Yes it does.
> "Module 'pyodbc' has no 'Error' member" Yes it does.
> 
> Issue with pylint, or pyodbc?

Pylint. 

> +-++
> |line-too-long|5   | meh

I'm largely meh on this too. But again its a PEP8 thing.

> +-++
> |wrong-import-order   |4   | does it matter?

Its useful to have a standard so you can glance and tell what's what and
from where, but what that standard is, is debatable.

> +-++
> |missing-docstring|4   | what's the difference between
>   a docstring and a # comment?

A docstring is a docstring, a comment is a comment. Google python
docstrings :) Python prefers files to have a docstring on top, and
functions beneath their definition. Comments should be used as little as
possible, as they must be maintained: an incorrect comment is worse then
no comment.

Go for clear code that doesn't *need* commenting.

> +-++
> |superfluous-parens   |3   | I like to surround 'or'
>   statments with parens

Why?

> +-++
> |multiple-imports |2   | doesn't everyone?

I don't actually know what its complaining at.

> +-++
> |bad-builtin  |2   | warning because I used filter?

Don't know what its complaining at about here either.

> +-++
> |missing-final-newline|1   | I'm using Notepad++, with
>   EOL Conversion set to
>   'Windows Format'.  How
>   or should I fix this?

Doesn't have anything to do with it. Just scroll to the bottom and press
enter. It wants to end on a newline, not code.

> Global evaluation
> -
> Your code has been rated at -7.64/10
> 
> I assume -7.64 is really bad?
> 
> Has anyone ever in history gotten 10/10 from pylint for a non-trivial 
> program?

No clue, I don't use pylint at all.

> [1]
> pylint says "Consider using enumerate instead of iterating with range 
> and len"
> 
> the offending code is:
> for j in range(len(list1)):
>do something with list1[j], list2[j], list3[j], etc.
> 
> enumeration would be:
> for j,item in enumerate(list1):
>do something with list1[j], list2[j], list3[j], etc.
> 
> Is there an advantage to using enumerate() here?

Its cleaner, easier to read. In Python 2 where range() returns a list,
its faster. (In python2, xrange returns a lazy evaluating range)

Use the tools Python gives you. Why reinvent enumerate when its built
in?

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python is an Equal Opportunity Programming Language

[Stephen Hansen]

On Fri, May 6, 2016, at 11:43 PM, Gregory Ewing wrote:
> Steven D'Aprano wrote:
> > Who is setting and enforcing this quota, and given that only about 1 in 20
> > Python programmers is a woman, do you think men are seriously missing out
> > on any opportunities?
> 
> Suppose there are 100 people wanting to ask questions, and
> there is only time to answer 10 questions. If the 1 in 20
> ratio holds, then 5 of those people are women and the other
> 95 are men.
> 
> Alternating between men and women means that all of the
> women get their questions answered, and only 5/95 of the
> men. So in this example, if you're a woman you have a 100%
> chance of getting answered, and if you're a man you only
> have a 5.26% chance.
> 
> Whether you think this is a good strategy or not,
> beliavsky is right that it's not "equal".

This is a pedantically and nonsensical definition of "equal", that
ignores the many, many reasons why there are 1 in 20 women in that
conference. Its looking at the end effect and ignoring everything that
leads up to it, and deciding its instead special rights -- this is the
great argument against minorities getting a voice, that their requests
for equal *opportunity* are instead *special rights* that diminish the
established majority's entrenched power. 

Those women are dealing with suppression, discrimination and dismissal
on multiple levels that leave them in a disenfranchised position. 

Recognizing those faults and taking corrective action is fundamentally
an act in the name of equality. 

Correcting for inequalities can not, itself, be a purely "equal" task
done in pure blindness of the contextual reality of what is going on in
the world. 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why do these statements evaluate the way they do?

[Stephen Hansen]

On Fri, May 6, 2016, at 11:36 PM, Anthony Papillion wrote:
> I'm trying to figure out why the following statements evaluate the way
> they do and I'm not grasping it for some reason. I'm hoping someone can
> help me.
> 
> 40+2 is 42 #evaluates to True
> But
> 2**32 is 2**32 #evaluates to False
> 
> This is an example taken from a Microsoft blog on the topic. They say the
> reason is because the return is based on identity and not value but, to
> me, these statements are fairly equal.

Yes, those statements are fairly *equal*. But you are not testing
*equality*. The "is" operator is testing *object identity*.

>>> a = 123456
>>> b = 123456
>>> a is b
False
>>> a == b
True

Equality means, "do these objects equal the same", identity means, "are
these specific objects the exact same objects".

In the above, a and b are different objects. There's two objects that
contain a value of 12345. They are equal (see the == test), but they are
different objects. 

If you write on a piece of paper "12345", and I write on a piece of
paper "12345", are those two pieces of paper identical? Of course not.
Your paper is in your hand, mine is in mine. These are different pieces
of paper. Just look at them: they're clearly not the same thing. That
said, if you look at the number they contain, they're the same value.

Our two pieces of paper are equivalent, but they're different objects.

This might get confusing for you because:

>>> a = 123
>>> b = 123
>>> a is b
True
>>> a == b 
True

This happens because Python caches small integers, so everytime you
write '123', it re-uses that same '123' object everyone else is using.
It shares those objects as an optimization.

The long and short of it is: you should almost never use 'is' for
comparing integers (or strings). It doesn't mean what you think it does
and isn't useful to you. Compare equality.

In general, the only things you should use 'is' for is when comparing to
singletons like None, True or False (And consider strongly not comparing
against False/True with is, but instead just 'if thing' and if its True,
it passes).

Otherwise, 'is' should only be used when you're comparing *object
identity*. You don't need to do that usually. Only do it when it matters
to you that an object with value A might be equal to an object of value
B, but you care that they're really different objects.


-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A fun python CLI program for all to enjoy!

[Stephen Hansen]

On Fri, May 6, 2016, at 04:58 PM, DFS wrote:
> Improper f.close didn't seem to affect any of the files my program wrote 
> - and I checked a lot of them when I was writing the code.

To be clear, its not an "improper" f.close. That command is simply not
closing the file. Period. "f.close" is how you get the 'close' function
from the 'f' object, and then... you do nothing with it.

If you removed "f.close" entirely, you'd get the exact same behavior as
you have now. The "f.close" does nothing.

That said, in CPython semantics, closing a file explicitly is often not
required. CPython is reference-counted. Once the references to an object
reaches 0, CPython deletes the object. This is an implementation detail
of the CPython and not a guarantee of the Python language itself, which
is why explicit close calls are preferred.

So while 'f.close' does nothing, CPython might be closing the file
*anyways*, and it might work... but that 'might' is hard to reason about
without a deeper understanding, so using explicit closing mechanics
(either via f.close() or with or something else) is strongly
recommended. 

For example, if you were to do:

for item in sequence:
f = open(item, 'wb')
f.write("blah")

It probably works fine. The first time through, 'f' is bound to a file
object, and you write to it. The second time through, 'f' is bound to a
*new file object*, and the original file object now has 0 references, so
is automatically deleted. 

The last sequence through, f is not closed: the 'for loop' is not a
scope which deletes its internal name bindings when its done. So that
'f' will likely remain open until the very end of the current function,
which may be an issue for you.

Implicit closing actually works in a large number of situations in
CPython, but it isn't a good thing to rely on. It only works in simple
operations where you aren't accidentally storing a reference somewhere
else. You have to keep track of the references in your head to make sure
things will get closed at proper times.

The 'with' statement clearly defines when resources should be closed, so
its preferred (As I see you've adopted from other responses). But its
also needed in other Python implementations which might not follow
CPython's reference counting scheme.

I'm not giving further feedback because MRAB caught everything I thought
was an issue.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pylint prefers list comprehension over filter...

[Stephen Hansen]

On Thu, May 5, 2016, at 07:46 PM, Dan Sommers wrote:
> On Thu, 05 May 2016 18:37:11 -0700, Stephen Hansen wrote:
> 
> > ''.join(x for x in string if x.isupper())
> 
> > The difference is, both filter and your list comprehension *build a
> > list* which is not needed, and wasteful. The above skips building a
> > list, instead returning a generator ...
> 
> filter used to build a list, but now it doesn't (where "used to" means
> Python 2.7 and "now" means Python 3.5; I'm too lazy to track down the
> exact point(s) at which it changed):

Oh, didn't know that. Then again the OP was converting the output of
filter *into* a list, which wasted a list either way.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pylint prefers list comprehension over filter...

[Stephen Hansen]

On Thu, May 5, 2016, at 06:26 PM, Christopher Reimer wrote:
> Which is one is correct (Pythonic)? Or does it matter?

First, pylint is somewhat opinionated, and its default options shouldn't
be taken as gospel. There's no correct: filter is fine.

That said, the general consensus is, I believe, that list comprehensions
are good, and using them is great.

In your case, though, I would not use a list comprehension. I'd use a
generator comprehension. It looks almost identical:

''.join(x for x in string if x.isupper())

The difference is, both filter and your list comprehension *build a
list* which is not needed, and wasteful. The above skips building a
list, instead returning a generator, and join pulls items out of it one
at a time as it uses them. No needlessly creating a list only to use it
and discard it.

-- 
Stephen Hansen
  m e @ i x o k a i  . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 11:03 AM, Steven D'Aprano wrote:
> - Nobody could possibly want to support non-ASCII text. (Apart from the
> approximately 6.5 billion people in the world that don't speak English of
> course, an utterly insignificant majority.)

Oh, I'd absolutely want to support non-ASCII text. If I have unicode
input, though, I unfortunately have to rely on
https://pypi.python.org/pypi/regex as 're' doesn't support matching on
character properties. 

I keep hoping it'll replace "re", then we could do:

pattern = regex.compile(ru"^\p{Lu}\s&]+$")

where \p{property} matches against character properties in the unicode
database.

> - Data validity doesn't matter, because there's no possible way that you
> might accidentally scrape data from the wrong part of a HTML file and end
> up with junk input.

Um, no one said that. I was arguing that the *regular expression*
doesn't need to be responsible for validation.

> - Even if you do somehow end up with junk, there couldn't possibly be any
> real consequences to that.

No one said that either...

> - It doesn't matter if you match too much, or to little, that just means
> the
> specs are too pedantic.

Or that...

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 05:31 AM, DFS wrote:
> You are out of your mind.

Whoa, now. I might disagree with Steven D'Aprano about how to approach
this problem, but there's no need to be rude. Everyone's trying to help
you, after all.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 10:43 AM, Steven D'Aprano wrote:
> On Thu, 5 May 2016 11:32 pm, Stephen Hansen wrote:
> 
> > On Thu, May 5, 2016, at 12:36 AM, Steven D'Aprano wrote:
> >> Oh, a further thought...
> >> 
> >> On Thursday 05 May 2016 16:46, Stephen Hansen wrote:
> >> > I don't even care about faster: Its overly complicated. Sometimes a
> >> > regular expression really is the clearest way to solve a problem.
> >> 
> >> Putting non-ASCII letters aside for the moment, how would you match these
> >> specs as a regular expression?
> > 
> > I don't know, but mostly because I wouldn't even try. 
> 
> Really? Peter Otten seems to have found a solution, and Random832 almost
> found it too.
> 
> 
> > The requirements 
> > are over-specified. If you look at the OP's data (and based on previous
> > conversation), he's doing web scraping and trying to pull out good data.
> 
> I'm not talking about the OP's data. I'm talking about *my* requirements.
> 
> I thought that this was a friendly discussion about regexes, but perhaps
> I
> was mistaken. Because I sure am feeling a lot of hostility to the ideas
> that regexes are not necessarily the only way to solve this, and that
> data
> validation is a good thing.

Umm, what? Hostility? I have no idea where you're getting that.

I didn't say that regexs are the only way to solve problems; in fact
they're something I avoid using in most cases. In the OP's case, though,
I did say I thought was a natural fit. Usually, I'd go for
startswith/endswith, "in", slicing and such string primitives before I
go for a regular expression.

"Find all upper cased phrases that may have &'s in them" is something
just specific enough that the built in string primitives are awkward
tools.

In my experience, most of the problems with regexes is people think
they're the hammer and every problem is a nail: and then they get into
ever more convoluted expressions that become brittle.  More specific in
a regular expression is not, necessarily, a virtue. In fact its exactly
the opposite a lot of times.

> > There's no absolutely perfect way to do that because the system he's
> > scraping isn't meant for data processing. The data isn't cleanly
> > articulated.
> 
> Right. Which makes it *more*, not less, important to be sure that your
> regex
> doesn't match too much, because your data is likely to be contaminated by
> junk strings that don't belong in the data and shouldn't be accepted.
> I've
> done enough web scraping to realise just how easy it is to start grabbing
> data from the wrong part of the file.

I have nothing against data validation: I don't think it belongs in
regular expressions, though. That can be a step done afterwards.

> > Instead, he wants a heuristic to pull out what look like section titles.
> 
> Good for him. I asked a different question. Does my question not count?

Sure it counts, but I don't want to engage in your theoretical exercise.
That's not being hostile, that's me not wanting to think about a complex
set of constraints for a regular expression for purely intellectual
reasons.

> I was trying to teach DFS a generic programming technique, not solve his
> stupid web scraping problem for him. What happens next time when he's
> trying to filter a list of floats, or Widgets? Should he convert them to
> strings so he can use a regex to match them, or should he learn about
> general filtering techniques?

Come on. This is a bit presumptuous, don't you think?

> > This translates naturally into a simple regular expression: an uppercase
> > string with spaces and &'s. Now, that expression doesn't 100% encode
> > every detail of that rule-- it allows both Q&A and Q & A-- but on my own
> > looking at the data, I suspect its good enough. The titles are clearly
> > separate from the other data scraped by their being upper cased. We just
> > need to expand our allowed character range into spaces and &'s.
> > 
> > Nothing in the OP's request demands the kind of rigorous matching that
> > your scenario does. Its a practical problem with a simple, practical
> > answer.
> 
> Yes, and that practical answer needs to reject:
> 
> - the empty string, because it is easy to mistakenly get empty strings
> when
> scraping data, especially if you post-process the data;
> 
> - strings that are all spaces, because "   " cannot possibly be a
> title;
> 
> - strings that are all ampersands, because "&&&&&" is not 

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 12:36 AM, Steven D'Aprano wrote:
> Oh, a further thought...
> 
> On Thursday 05 May 2016 16:46, Stephen Hansen wrote:
> > I don't even care about faster: Its overly complicated. Sometimes a
> > regular expression really is the clearest way to solve a problem.
> 
> Putting non-ASCII letters aside for the moment, how would you match these 
> specs as a regular expression?

I don't know, but mostly because I wouldn't even try. The requirements
are over-specified. If you look at the OP's data (and based on previous
conversation), he's doing web scraping and trying to pull out good data.
There's no absolutely perfect way to do that because the system he's
scraping isn't meant for data processing. The data isn't cleanly
articulated.

Instead, he wants a heuristic to pull out what look like section titles. 

The OP looked at the data and came up with a simple set of rules that
identify these section titles:

>> Want to keep all elements containing only upper case letters or upper 
case letters and ampersand (where ampersand is surrounded by spaces)

This translates naturally into a simple regular expression: an uppercase
string with spaces and &'s. Now, that expression doesn't 100% encode
every detail of that rule-- it allows both Q&A and Q & A-- but on my own
looking at the data, I suspect its good enough. The titles are clearly
separate from the other data scraped by their being upper cased. We just
need to expand our allowed character range into spaces and &'s.

Nothing in the OP's request demands the kind of rigorous matching that
your scenario does. Its a practical problem with a simple, practical
answer.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Thu, May 5, 2016, at 12:04 AM, Steven D'Aprano wrote:
> On Thursday 05 May 2016 16:46, Stephen Hansen wrote:
> > > On Wed, May 4, 2016, at 11:04 PM, Steven D'Aprano wrote:
> >> Start by writing a function or a regex that will distinguish strings that
> >> match your conditions from those that don't. A regex might be faster, but
> >> here's a function version.
> >> ... snip ...
> > 
> > Yikes. I'm all for the idea that one shouldn't go to regex when Python's
> > powerful string type can answer the problem more clearly, but this seems
> > to go out of its way to do otherwise.
> > 
> > I don't even care about faster: Its overly complicated. Sometimes a
> > regular expression really is the clearest way to solve a problem.
> 
> You're probably right, but I find it easier to reason about matching in 
> Python rather than the overly terse, cryptic regular expression mini-
> language.
> 
> I haven't tested my function version, but I'm 95% sure that it is
> correct. 
> It trickiest part of it is the logic about splitting around ampersands.
> And 
> I'll cheerfully admit that it isn't easy to extend to (say) "ampersand,
> or 
> at signs". But your regex solution:
> 
> r"^[A-Z\s&]+$"
> 
> is much smaller and more compact, but *wrong*. For instance, your regex 
> wrongly accepts both "&&&&&" and "  " as valid strings, and wrongly 
> rejects "ΔΣΘΛ". Your Greek customers will be sad...

Meh. You have a pedantic definition of wrong. Given the inputs, it
produced right output. Very often that's enough. Perfect is the enemy of
good, it's said. 

There's no situation where "&&&&&" and " " will exist in the given
dataset, and recognizing that is important. You don't have to account
for every bit of nonsense. 

If the OP needs a unicode-aware solution that redefines "A-Z" as perhaps
"\w" with an isupper call. Its still far simpler then you're suggesting.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Wed, May 4, 2016, at 11:04 PM, Steven D'Aprano wrote:
> Start by writing a function or a regex that will distinguish strings that 
> match your conditions from those that don't. A regex might be faster, but 
> here's a function version.
> ... snip ...

Yikes. I'm all for the idea that one shouldn't go to regex when Python's
powerful string type can answer the problem more clearly, but this seems
to go out of its way to do otherwise.

I don't even care about faster: Its overly complicated. Sometimes a
regular expression really is the clearest way to solve a problem.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Whittle it on down

[Stephen Hansen]

On Wed, May 4, 2016, at 09:58 PM, DFS wrote:
> Want to whittle a list like this:
> 
> [u'Espa\xf1ol', 'Health & Fitness Clubs (36)', 'Health Clubs & 
> Gymnasiums (42)', 'Health Fitness Clubs', 'Name', 'Atlanta city guide', 
> 'edit address', 'Tweet', 'PHYSICAL FITNESS CONSULTANTS & TRAINERS', 
> 'HEALTH CLUBS & GYMNASIUMS', 'HEALTH CLUBS & GYMNASIUMS', 
> 'www.custombuiltpt.com/', 'RACQUETBALL COURTS PRIVATE', 
> 'www.lafitness.com', 'GYMNASIUMS', 'HEALTH & FITNESS CLUBS', 
> 'www.lafitness.com', 'HEALTH & FITNESS CLUBS', 'www.lafitness.com', 
> 'PERSONAL FITNESS TRAINERS', 'HEALTH CLUBS & GYMNASIUMS', 'EXERCISE & 
> PHYSICAL FITNESS PROGRAMS', 'FITNESS CENTERS', 'HEALTH CLUBS & 
> GYMNASIUMS', 'HEALTH CLUBS & GYMNASIUMS', 'PERSONAL FITNESS TRAINERS', 
> '5', '4', '3', '2', '1', 'Yellow Pages', 'About Us', 'Contact Us', 
> 'Support', 'Terms of Use', 'Privacy Policy', 'Advertise With Us', 
> 'Add/Update Listing', 'Business Profile Login', 'F.A.Q.']
> 
> down to
> 
> ['PHYSICAL FITNESS CONSULTANTS & TRAINERS', 'HEALTH CLUBS & GYMNASIUMS', 
> 'HEALTH CLUBS & GYMNASIUMS', 'RACQUETBALL COURTS PRIVATE', 'GYMNASIUMS', 
> 'HEALTH & FITNESS CLUBS', 'HEALTH & FITNESS CLUBS',  'PERSONAL FITNESS 
> TRAINERS', 'HEALTH CLUBS & GYMNASIUMS', 'EXERCISE & PHYSICAL FITNESS 
> PROGRAMS', 'FITNESS CENTERS', 'HEALTH CLUBS & GYMNASIUMS', 'HEALTH CLUBS 
> & GYMNASIUMS', 'PERSONAL FITNESS TRAINERS']

Sometimes regular expressions are the tool to do the job:

Given:

>>> input = [u'Espa\xf1ol', 'Health & Fitness Clubs (36)', 'Health Clubs & 
>>> Gymnasiums (42)', 'Health Fitness Clubs', 'Name', 'Atlanta city guide', 
>>> 'edit address', 'Tweet', 'PHYSICAL FITNESS CONSULTANTS & TRAINERS', 'HEALTH 
>>> CLUBS & GYMNASIUMS', 'HEALTH CLUBS & GYMNASIUMS', 'www.custombuiltpt.com/', 
>>> 'RACQUETBALL COURTS PRIVATE', 'www.lafitness.com', 'GYMNASIUMS', 'HEALTH & 
>>> FITNESS CLUBS', 'www.lafitness.com', 'HEALTH & FITNESS CLUBS', 
>>> 'www.lafitness.com', 'PERSONAL FITNESS TRAINERS', 'HEALTH CLUBS & 
>>> GYMNASIUMS', 'EXERCISE & PHYSICAL FITNESS PROGRAMS', 'FITNESS CENTERS', 
>>> 'HEALTH CLUBS & GYMNASIUMS', 'HEALTH CLUBS & GYMNASIUMS', 'PERSONAL FITNESS 
>>> TRAINERS', '5', '4', '3', '2', '1', 'Yellow Pages', 'About Us', 'Contact 
>>> Us', 'Support', 'Terms of Use', 'Privacy Policy', 'Advertise With Us', 
>>> 'Add/Update Listing', 'Business Profile Login', 'F.A.Q.']

Then:

>>> pattern = re.compile(r"^[A-Z\s&]+$")
>>> output = [x for x in list if pattern.match(x)]
>>> output
['PHYSICAL FITNESS CONSULTANTS & TRAINERS', 'HEALTH CLUBS & GYMNASIUMS',
'HEALTH CLUBS & GYMNASIUMS', 'RACQUETBALL COURTS PRIVATE', 'GYMNASIUMS',
'HEALTH & FITNESS CLUBS', 'HEALTH & FITNESS CLUBS', 'PERSONAL FITNESS
TRAINERS', 'HEALTH CLUBS & GYMNASIUMS', 'EXERCISE & PHYSICAL FITNESS
PROGRAMS', 'FITNESS CENTERS', 'HEALTH CLUBS & GYMNASIUMS', 'HEALTH CLUBS
& GYMNASIUMS', 'PERSONAL FITNESS TRAINERS']

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: No SQLite newsgroup, so I'll ask here about SQLite, python and MS Access

[Stephen Hansen]

On Wed, May 4, 2016, at 03:46 PM, DFS wrote:
> I can't find anything on the web.

Have you tried: 
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
If you really must access it over a newsgroup, you can use the Gmane
mirror:
http://gmane.org/info.php?group=gmane.comp.db.sqlite.general

> Any ideas?

Sorry, I don't use Access.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Conditionals And Control Flows

[Stephen Hansen]

On Wed, May 4, 2016, at 07:41 AM, Cai Gengyang wrote:
> I am trying to understand the boolean operator "and" in Python. It is
> supposed to return "True" when the expression on both sides of "and" are
> true 

The thing is, its kinda dubious to think of 'and' as a 'boolean
operator', because once you go down that road, some people start wanting
it to be a *pure* boolean operator. Something that always returns True
or False. Instead, 'and' and 'or' return something that is true, or
something that is false. Notice the lower case. (I know the docs call
them Boolean Operations, but I still think saying 'boolean' is
unhelpful)

Python defines false things as False, None, 0 (of any numeric type), an
empty container (lists, tuples, mappings, something else that defines
__len__ and it returns 0), and instances of classes that define
__nonzero__ that return 0 or False.

Everything else is a true thing.

If you see "x and y", the rule is: if x is a false thing, it'll return
something false. As it happens, it has x handy, and since its decided x
is false, it'll return that. Therefore, "x and y" is false. If x is
true, though, it'll return y. In this case, "x and y" will be a true
thing if y is a true thing, and a false thing if y is a false thing.

As you can see, all of this logic happens without ever using True or
False. 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Mon, May 2, 2016, at 08:57 PM, jf...@ms4.hinet.net wrote:
> Stephen Hansen at 2016/5/3 11:49:22AM wrote:
> > On Mon, May 2, 2016, at 08:27 PM, jf...@ms4.hinet.net wrote:
> > > But when I try to get this forum page, it does get a html file but can't
> > > be viewed normally.
> > 
> > What does that mean?
> > 
> > -- 
> > Stephen Hansen
> >   m e @ i x o k a i . i o
> 
> The page we are looking at:-)
> https://groups.google.com/forum/#!topic/comp.lang.python/jFl3GJbmR7A

Try scraping gmane. Google Groups is one big javascript application.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Saving Consol outputs in a python script

[Stephen Hansen]

On Tue, May 3, 2016, at 05:14 AM, drewes@gmail.com wrote:
> What I need are the 2 values for the 2 classes saved in a variable in the
> .py script, so that I can write them into a text file.
> 
> Would be super nice if someone could help me!

You shouldn't use the call() convienence function, but instead create a
process using the Popen constructor, passing PIPE to stdout. Then use
communicate() to get the output.

This should get you started:

process = subprocess.Popen(["commandline"], stdout=subprocess.PIPE)
output, error = process.communicate()

Output will be a string, string has a splitlines method, etc.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Mon, May 2, 2016, at 08:27 PM, jf...@ms4.hinet.net wrote:
> But when I try to get this forum page, it does get a html file but can't
> be viewed normally.

What does that mean?

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to clean up list items?

[Stephen Hansen]

On Mon, May 2, 2016, at 11:09 AM, DFS wrote:
> I'd prefer to get clean data in the first place, but I don't know a 
> better way to extract it from the HTML.

Ah, right. I didn't know you were scraping HTML. Scraping HTML is rarely
clean so you have to do a lot of cleanup.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python3 html scraper that supports javascript

[Stephen Hansen]

On Mon, May 2, 2016, at 08:33 AM, zljubi...@gmail.com wrote:
> I tried to use the following code:
> 
> from bs4 import BeautifulSoup
> from selenium import webdriver
> 
> PHANTOMJS_PATH =
> 'C:\\Users\\Zoran\\Downloads\\Obrisi\\phantomjs-2.1.1-windows\\bin\\phantomjs.exe'
> 
> url =
> 'https://hrti.hrt.hr/#/video/show/2203605/trebizat-prica-o-jednoj-vodi-i-jednom-narodu-dokumentarni-film'
> 
> browser = webdriver.PhantomJS(PHANTOMJS_PATH)
> browser.get(url)
> 
> soup = BeautifulSoup(browser.page_source, "html.parser")
> 
> x = soup.prettify()
> 
> print(x)
> 
> When I print x variable, I would expect to see something like this:
>  src="mediasource:https://hrti.hrt.hr/2e9e9c45-aa23-4d08-9055-cd2d7f2c4d58";
> id="vjs_video_3_html5_api" class="vjs-tech" preload="none"> type="application/x-mpegURL"
> src="https://prd-hrt.spectar.tv/player/get_smil/id/2203605/video_id/2203605/token/Cny6ga5VEQSJ2uZaD2G8pg/token_expiration/1462043309/asset_type/Movie/playlist_template/nginx/channel_name/trebiat__pria_o_jednoj_vodi_i_jednom_narodu_dokumentarni_film/playlist.m3u8?foo=bar";>
> 
> 
> but I can't come to that point.

Why? As important as it is to show code, you need to show what actually
happens and what error message is produced.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to clean up list items?

[Stephen Hansen]

On Mon, May 2, 2016, at 09:33 AM, DFS wrote:
> Have: list1 = ['\r\n   Item 1  ','  Item 2  ','\r\n  ']

I'm curious how you got to this point, it seems like you can solve the
problem in how this is generated.

> Want: list1 = ['Item 1','Item 2']

That said:

list1 = [t.strip() for t in list1 if t and not t.isspace()]

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Mon, May 2, 2016, at 12:37 AM, DFS wrote:
> On 5/2/2016 2:27 AM, Stephen Hansen wrote:
> > I'm again going back to the point of: its fast enough. When comparing
> > two small numbers, "twice as slow" is meaningless.
> 
> Speed is always meaningful.
> 
> I know python is relatively slow, but it's a cool, concise, powerful 
> language.  I'm extremely impressed by how tight the code can get.

I'm sorry, but no. Speed is not always meaningful. 

It's not even usually meaningful, because you can't quantify what "speed
is". In context, you're claiming this is twice as slow (even though my
tests show dramatically better performance), but what details are
different?

You're ignoring the fact that Python might have a constant overhead --
meaning, for a 1k download, it might have X speed cost. For a 1meg
download, it might still have the exact same X cost.

Looking narrowly, that overhead looks like "twice as slow", but that's
not meaningful at all. Looking larger, that overhead is a pittance.

You aren't measuring that.

> > You have an assumption you haven't answered, that downloading a 10 meg
> > file will be twice as slow as downloading this tiny file. You haven't
> > proven that at all.
> 
> True.  And it has been my assumption - tho not with 10MB file.

And that assumption is completely invalid.

> I noticed urllib and curl returned the html as is, but urllib2 and 
> requests added enhancements that should make the data easier to parse. 
> Based on speed and functionality and documentation, I believe I'll be 
> using the requests HTTP library (I will actually be doing a small amount 
> of web scraping).

The requests library's added-value is ease-of-use, and its overhead is
likely tiny: so using it means you spend less effort making a thing
happen. I recommend you embrace this. 

> VBScript
> 1st run: 7.70 seconds
> 2nd run: 5.38
> 3rd run: 7.71
> 
> So python matches or beats VBScript at this much larger file.  Kewl.

This is what I'm talking about: Python might have a constant overhead,
but looking at larger operations, its probably comparable. Not fast,
mind you. Python isn't the fastest language out there. But in real world
work, its usually fast enough.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 10:59 PM, DFS wrote:
> startTime = time.clock()
> for i in range(loops):
>   r = urllib2.urlopen(webpage)
>   f = open(webfile,"w")
>   f.write(r.read())
>   f.close
> endTime = time.clock()  
> print "Finished urllib2 in %.2g seconds" %(endTime-startTime)

Yeah on my system I get 1.8 out of this, amounting to 0.18s. 

I'm again going back to the point of: its fast enough. When comparing
two small numbers, "twice as slow" is meaningless.

You have an assumption you haven't answered, that downloading a 10 meg
file will be twice as slow as downloading this tiny file. You haven't
proven that at all. 

I suspect you have a constant overhead of X, and in this toy example,
that makes it seem twice as slow. But when downloading a file of size,
you'll have the same constant factor, at which point the difference is
irrelevant. 

If you believe otherwise, demonstrate it.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Sun, May 1, 2016, at 10:23 PM, DFS wrote:
> Trying the rawstring thing (say it fast 3x):
> 
> webpage = "http://econpy.pythonanywhere.com/ex/001.html";
> 
> 
> webfile = "D:\\econpy001.html"
> urllib.urlretrieve(webpage,webfile) WORKS
> 
> webfile = "rD:\econpy001.html"

The r is *outside* the string.

Its: r"D:\econpy001.html"

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Code Opinion - Enumerate

[Stephen Hansen]

On Sun, May 1, 2016, at 08:17 PM, Sayth Renshaw wrote:
> Just looking for your opinion on style would you write it like this
> continually calling range or would you use enumerate instead, or neither
> (something far better) ?

I can't comment on your specific code because there's too much noise to
it, but in general:

Using enumerate increases readability, and I use it whenever the idiom:

for index, item in enumerate(thing):
...

is used.

Enumerate is your friend. Hug it.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Sun, May 1, 2016, at 10:08 PM, DFS wrote:
> On 5/2/2016 1:02 AM, Stephen Hansen wrote:
> >> I actually use "D:\\file.html" in my code.
> >
> > Or you can do that. But the whole point of raw strings is not having to
> > escape slashes :)
> 
> 
> Nice.  Where/how else is 'r' used?

Raw strings are primarily used A) for windows paths, and more
universally, B) for regular expressions. 

But in theory they're useful anywhere you have static/literal data that
might include backslashes where you don't actually intend to use any
escape characters.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 10:04 PM, DFS wrote:
> And two small numbers turn into bigger numbers when the webpage is big, 
> and soon the download time differences are measured in minutes, not half 
> a second.

Are you sure of that? Have you determined that the time is not a
constant overhead verses that the time is directly relational to the
size of the page? If so, how have you determined that?

You aren't showing how you're testing. 0.4s difference is meaningless to
me, if its a constant overhead. If its twice as slow for a 1 meg file,
then you might have an issue. Maybe. You haven't shown that.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 10:00 PM, DFS wrote:
> I tried the 10-loop test several times with all versions.

Also how, _exactly_, are you testing this?

C:\Python27>python -m timeit "filename='C:\\test.txt';
webpage='http://econpy.pythonanywhere.com/ex/001.html'; import urllib2;
r = urllib2.urlopen(webpage); f = open(filename, 'w');
f.write(r.read()); f.close();"
10 loops, best of 3: 175 msec per loop

That's a whole lot less the 0.88secs.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 09:50 PM, DFS wrote:
> On 5/2/2016 12:40 AM, Chris Angelico wrote:
> > On Mon, May 2, 2016 at 2:34 PM, Stephen Hansen  wrote:
> >> On Sun, May 1, 2016, at 09:06 PM, DFS wrote:
> >>> Then I tested them in loops - the VBScript is MUCH faster: 0.44 for 10
> >>> iterations, vs 0.88 for python.
> >> ...
> >>> I know it's asking a lot, but is there a really fast AND really short
> >>> python solution for this simple thing?
> >>
> >> 0.88 is not fast enough for you? That's less then a second.
> >
> > Also, this is timings of network and disk operations. Unless something
> > pathological is happening, the language used won't make any
> > difference.
> >
> > ChrisA
> 
> 
> Unfortunately, the VBScript is twice as fast as any python method.

And 0.2 is twice as fast as 0.1. When you have two small numbers, 'twice
as fast' isn't particularly meaningful as a metric. 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You gotta love a 2-line python solution

[Stephen Hansen]

On Sun, May 1, 2016, at 09:51 PM, DFS wrote:
> On 5/2/2016 12:31 AM, Stephen Hansen wrote:
> > On Sun, May 1, 2016, at 08:39 PM, DFS wrote:
> >> To save a webpage to a file:
> >> -
> >> 1. import urllib
> >> 2. urllib.urlretrieve("http://econpy.pythonanywhere.com
> >>  /ex/001.html","D:\file.html")
> >> -
> >
> > Note, for paths on windows you really want to use a rawstring. Ie,
> > r"D:\file.html".
> > 
> Thanks.
> 
> I actually use "D:\\file.html" in my code.

Or you can do that. But the whole point of raw strings is not having to
escape slashes :) 

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fastest way to retrieve and write html contents to file

[Stephen Hansen]

On Sun, May 1, 2016, at 09:06 PM, DFS wrote:
> Then I tested them in loops - the VBScript is MUCH faster: 0.44 for 10 
> iterations, vs 0.88 for python.
...
> I know it's asking a lot, but is there a really fast AND really short 
> python solution for this simple thing?

0.88 is not fast enough for you? That's less then a second.

-- 
Stephen Hansen
  m e @ i x o k a i . i o
-- 
https://mail.python.org/mailman/listinfo/python-list