Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?
> >Are you sure you want `str()`? > > > str(b'aaa') > >"b'aaa'" > > > >Probably you want: > > > >map(lambda x: x.decode(), bbb) > > _And_ you need to know the encoding of the text in the bytes. The above > _assumes_ UTF-8 because that is the default for bytes.decode, and if > that is _not_ what is in the bytes objects you will get mojibake. > > Because a lot of stuff is "mostly ASCII", this is the kind of bug which > can lurk until much later when you have less usual data. As I said, crawl the delivery chain looking for where things come from (and what they are). Karsten -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
Stefan Ram wrote: > Chris Angelico writes: > >But this is a really good job for a list comprehension: > >sss = [str(word) for word in bbb] > > Are you all sure that "str" is really what you all want? > Not absolutely, you no doubt have been following other threads related to this one. :-) > |>>> b = b"b" > |>>> str( b ) > |"b'b'" > > Maybe try to /decode/ the bytes? > > |>>> b.decode( "ASCII" ) > |'b' > > Therein lies the problem, the incoming byte stream *isn't* ASCII, it's an E-Mail message which may, for example, have UTF-8 or other encoded characters in it. Hopefully it will have an encoding given in the header but that's only if the sender is 'well behaved', one needs to be able to handle almost anything and it must be done without 'manual' interaction. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
Chris Angelico wrote: > On Fri, Aug 28, 2020 at 6:36 AM Chris Green wrote: > > > > This sounds quite an easy thing to do but I can't find how to do it > > elegantly. > > > > I have a list of bytes class objects (i.e. a list containing sequences > > of bytes, which are basically text) and I want to convert it to a list > > of string objects. > > > > One of the difficulties of finding out how to do this is that 'list of > > bytes' tends to mean a bytes object with a sequence of bytes in it > > which is *not* what I'm after converting. :-) > > > > Obviously I can do:- > > > > bbb = [b'aaa', b'bbb', b'ccc'] > > sss = [] > > for i in range(0, 2): > > sss.append(str(bbb[i]) > > > > but that does seem a bit clumsy. Is there a better way? > > > > Firstly, you shouldn't iterate over the range, but over the items themselves: > > for word in bbb: > Yes, of course, it was just a 'quick and dirty' to make the point. > But this is a really good job for a list comprehension: > > sss = [str(word) for word in bbb] > That seems the way to do it, thanks. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: How do I do this in Python 3 (string.join())?
Cameron Simpson wrote: [snip] > > >The POP3 processing is solely to collect E-Mail that ends up in the > >'catchall' mailbox on my hosting provider. It empties the POP3 > >catchall mailbox, checks for anything that *might* be for me or other > >family members then just deletes the rest. > > Very strong email policy, that one. Personally I fear data loss, and > process everything; anything which doesn't match a rule lands in my > "UNKNOWN" mail folder for manual consideration when I'm bored. It is > largely spam, but sometimes has a message wanting a new filing rule. > It's not *that* strong, the catchall is for *anything* that is addressed to either of the two domains hosted there. I.e. mail for xhghj...@isbd.net will arrive in the catchall mailbox. So I just search the To: address for anything that might be a typo for one of our names or anything else that might be of interest. I have an associated configuration file that specifies the patterns to look for so I can change things on the fly as it were. One of the scripts that I'm having trouble converting to Python 3 is the one that does this catchall management. > >> >E.g. in this case the only (well the only ready made) way to get a > >> >POP3 message is using poplib and this just gives you a list of lines > >> >made up of "bytes as text" :- > >> > > >> >popmsg = pop3.retr(i+1) > >> > >> Ok, so you have bytes? You need to know. > >> > >The documentation says (and it's exactly the same for Python 2 and > >Python 3):- > > > >POP3.retr(which) > >Retrieve whole message number which, and set its seen flag. Result > >is in form (response, ['line', ...], octets). > > > >Which isn't amazingly explicit unless 'line' implies a string. > > Aye. But "print(repr(a_pop_line))" will tell you. Almost certainly a > string-of-bytes, so I would expect bytes. The docs are probably > unchanged during the Python2->3 move. > Yes, I added some print statments to my catchall script to find out and, yes, the returned value is a list of 'byte strings'. It's a pity there isn't a less ambiguous name for 'string-of-bytes'! :-) > >> >I join the lines to feed them into mailbox.mbox() to create a mbox I > >> >can analyse and also a message which can be sent using SMTP. > > Ah. I like Maildirs for analysis; every message has its own file, which > makes adding and removing messages easy, and avoids contention with > other things using the Maildir. > > My mailfiler can process Maildirs (scan, add, remove) and add to > Maildirs and mboxes. > I've switched to maildir several times in the past and have always switched back because they have so many 'standards'. I use mutt as my MUA and that does handle maildir as well as anything but still doesn't do it for me. :-) -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
Cameron Simpson wrote: > On 27Aug2020 23:54, Marco Sulla wrote: > >Are you sure you want `str()`? > > > str(b'aaa') > >"b'aaa'" > > > >Probably you want: > > > >map(lambda x: x.decode(), bbb) > > _And_ you need to know the encoding of the text in the bytes. The above > _assumes_ UTF-8 because that is the default for bytes.decode, and if > that is _not_ what is in the bytes objects you will get mojibake. > > Because a lot of stuff is "mostly ASCII", this is the kind of bug which > can lurk until much later when you have less usual data. > If there's an encoding given in the header of the incoming E-Mail then one (hopefully) knows what the encoding is. However you have to be able to handle the more general case where either the encoding isn't given or it's wrong. In the real world E-Mail survives having an incorrect encoding in the header, what you see is either missing or garbled characters with the remainder being OK. Garbling the whole lot isn't a good approach. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
How do I place a preset into the text box?
The following program compiles but does not quite do what I would like it to
do. Line 19 is the preset information but I do not seem to be able to get it
into the form by code. My purpose is to let the user make changes without
having to re-enter the entire code.
Suggestions welcome.
Steve
#===
import tkinter as tk
from tkinter import ttk
import sys
window = tk.Tk()
window.title("Python Tkinter Text Box")
window.minsize(600,400)
def Submit():
label.configure(text= 'The new code is: ' + NewCode.get())
def ClickExit():
#This exit closes the program but the form remains and is still active.
# I want only to close the form.
print("Exiting")
sys.exit()
OldCode = ("1234-abcd")
label = ttk.Label(window, text = "Enter the new code")
label.grid(column = 1, row = 1)
NewCode = tk.StringVar()
CodeEntered = ttk.Entry(window, width = 15, textvariable = NewCode)
CodeEntered.grid(column = 2, row = 3)
button = ttk.Button(window, text = "Submit", command = Submit)
button.grid(column= 2, row = 5)
button = ttk.Button(window, text = "Quit", command = ClickExit)
button.grid(column= 2, row = 7)
window.mainloop()
x = (NewCode.get())
print("The new code entered is: " + x)
#=
Footnote:
Mars is the only known planet in our solar system solely inhabited by
functioning robots.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
On 28Aug2020 08:56, Chris Green wrote:
>Stefan Ram wrote:
>> Chris Angelico writes:
>> >But this is a really good job for a list comprehension:
>> >sss = [str(word) for word in bbb]
>>
>> Are you all sure that "str" is really what you all want?
>>
>Not absolutely, you no doubt have been following other threads related
>to this one. :-)
It is almost certainly not what you want. You want some flavour of
bytes.decode. If the BytesParser doesn't cope, you may need to parse the
headers as some kind of text (eg ISO8859-1) until you find a
content-transfer-encoding header (which still applies only to the body,
not the headers).
>> |>>> b = b"b"
>> |>>> str( b )
>> |"b'b'"
>>
>> Maybe try to /decode/ the bytes?
>>
>> |>>> b.decode( "ASCII" )
>> |'b'
>>
>>
>Therein lies the problem, the incoming byte stream *isn't* ASCII, it's
>an E-Mail message which may, for example, have UTF-8 or other encoded
>characters in it. Hopefully it will have an encoding given in the
>header but that's only if the sender is 'well behaved', one needs to
>be able to handle almost anything and it must be done without 'manual'
>interaction.
POP3 is presumably handing you bytes containing a message. If the Python
email.BytesParser doesn't handle it, stash the raw bytes _elsewhere_ in
a distinct file in some directory.
with open('evil_msg_bytes', 'wb') as f:
for bs in bbb:
f.write(bs)
No interpreation requires, since parsing failed. Then you can start
dealing with these exceptions. _Do not_ write unparsable messages into
an mbox!
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
Cameron Simpson wrote:
> On 28Aug2020 08:56, Chris Green wrote:
> >Stefan Ram wrote:
> >> Chris Angelico writes:
> >> >But this is a really good job for a list comprehension:
> >> >sss = [str(word) for word in bbb]
> >>
> >> Are you all sure that "str" is really what you all want?
> >>
> >Not absolutely, you no doubt have been following other threads related
> >to this one. :-)
>
> It is almost certainly not what you want. You want some flavour of
> bytes.decode. If the BytesParser doesn't cope, you may need to parse the
> headers as some kind of text (eg ISO8859-1) until you find a
> content-transfer-encoding header (which still applies only to the body,
> not the headers).
>
> >> |>>> b = b"b"
> >> |>>> str( b )
> >> |"b'b'"
> >>
> >> Maybe try to /decode/ the bytes?
> >>
> >> |>>> b.decode( "ASCII" )
> >> |'b'
> >>
> >>
> >Therein lies the problem, the incoming byte stream *isn't* ASCII, it's
> >an E-Mail message which may, for example, have UTF-8 or other encoded
> >characters in it. Hopefully it will have an encoding given in the
> >header but that's only if the sender is 'well behaved', one needs to
> >be able to handle almost anything and it must be done without 'manual'
> >interaction.
>
> POP3 is presumably handing you bytes containing a message. If the Python
> email.BytesParser doesn't handle it, stash the raw bytes _elsewhere_ in
> a distinct file in some directory.
>
> with open('evil_msg_bytes', 'wb') as f:
> for bs in bbb:
> f.write(bs)
>
> No interpreation requires, since parsing failed. Then you can start
> dealing with these exceptions. _Do not_ write unparsable messages into
> an mbox!
>
Maybe I shouldn't but Python 2 has been managing to do so for several
years without any issues. I know I *could* put the exceptions in a
bucket somewhere and deal with them separately but I'd really rather
not.
At prsent (with the Python 2 code still installed) it all 'just works'
and the absolute worst corruption I ever see in an E-Mail is things
like accented characters missing altogether or £ signs coming out as a
funny looking string. Either of these don't really make the message
unintelligible.
Are we saying that Python 3 really can't be made to handle things
'tolerantly' like Python 2 used to?
--
Chris Green
·
--
https://mail.python.org/mailman/listinfo/python-list
Re: How do I place a preset into the text box?
Hi Steve,
> On 28 Aug 2020, at 11:03, Steve wrote:
>
>
> The following program compiles but does not quite do what I would like it to
> do. Line 19 is the preset information but I do not seem to be able to get it
> into the form by code. My purpose is to let the user make changes without
> having to re-enter the entire code.
I'm no Tk expert but does the following do what you want? (Strictly speaking,
the parentheses in ("1234-abcd") are not wrong just unnecessary.)
#===
import tkinter as tk
from tkinter import ttk
import sys
window = tk.Tk()
window.title("Python Tkinter Text Box")
window.minsize(600,400)
def Submit():
label.configure(text= 'The new code is: ' + NewCode.get())
def ClickExit():
#This exit closes the program but the form remains and is still active.
# I want only to close the form.
print("Exiting")
#sys.exit()
window.destroy()
#OldCode = ("1234-abcd")
OldCode = "1234-abcd"
label = ttk.Label(window, text = "Enter the new code")
label.grid(column = 1, row = 1)
#NewCode = tk.StringVar()
NewCode = tk.StringVar(value=OldCode)
CodeEntered = ttk.Entry(window, width = 15, textvariable = NewCode)
CodeEntered.grid(column = 2, row = 3)
button = ttk.Button(window, text = "Submit", command = Submit)
button.grid(column= 2, row = 5)
button = ttk.Button(window, text = "Quit", command = ClickExit)
button.grid(column= 2, row = 7)
window.mainloop()
x = (NewCode.get())
print("The new code entered is: " + x)
#=
Regards,
Colin
--
https://mail.python.org/mailman/listinfo/python-list
Re: How do I place a preset into the text box?
Steve wrote: > The following program compiles but does not quite do what I would like it to > do. Line 19 is the preset information but I do not seem to be able to get it > into the form by code. My purpose is to let the user make changes without > having to re-enter the entire code. > You might consider moving the entry get() function into the Submit() call back def Submit() : label.configure( text = 'The new code is : ' + NewCode.get() ) x = ( NewCode.get() ) print( "\n The new code entered is : " + x ) The following insert function will show the OldCode in the entry box OldCode = ( "1234-abcd" ) # CodeEntered = ttk.Entry( window , width = 15 , textvariable = NewCode ) CodeEntered.grid( column = 2 , row = 3 , pady = 10 ) CodeEntered.insert( 0 , OldCode ) -- Stanley C. Kitching Human Being Phoenix, Arizona -- https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?
> > No interpreation requires, since parsing failed. Then you can start > > dealing with these exceptions. _Do not_ write unparsable messages into > > an mbox! > > > Maybe I shouldn't but Python 2 has been managing to do so for several > years without any issues. I am inclined to congratulate you on that sheer amount of luck. I don't believe there were no issues because everything worked just right under py2 but rather because py2 cared less than py3 does now. > Are we saying that Python 3 really can't be made to handle things > 'tolerantly' like Python 2 used to? It sure should be possible but it will require *explicit* en/decode()s in more places than before because AFAICT there's less impliciteness as to which encoding to apply (regardless of whether it applies). Karsten -- https://mail.python.org/mailman/listinfo/python-list
Re: Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?
On 8/28/20 7:50 AM, Karsten Hilbert wrote: >>> No interpreation requires, since parsing failed. Then you can start >>> dealing with these exceptions. _Do not_ write unparsable messages into >>> an mbox! >>> >> Maybe I shouldn't but Python 2 has been managing to do so for several >> years without any issues. > I am inclined to congratulate you on that sheer amount of luck. I don't > believe there were no issues because everything worked just right under > py2 but rather because py2 cared less than py3 does now. > >> Are we saying that Python 3 really can't be made to handle things >> 'tolerantly' like Python 2 used to? > It sure should be possible but it will require *explicit* en/decode()s in > more places than before because AFAICT there's less impliciteness as to > which encoding to apply (regardless of whether it applies). > > Karsten > > > -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Re: Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?
On 8/28/20 7:50 AM, Karsten Hilbert wrote: >>> No interpreation requires, since parsing failed. Then you can start >>> dealing with these exceptions. _Do not_ write unparsable messages into >>> an mbox! >>> >> Maybe I shouldn't but Python 2 has been managing to do so for several >> years without any issues. > I am inclined to congratulate you on that sheer amount of luck. I don't > believe there were no issues because everything worked just right under > py2 but rather because py2 cared less than py3 does now. > >> Are we saying that Python 3 really can't be made to handle things >> 'tolerantly' like Python 2 used to? > It sure should be possible but it will require *explicit* en/decode()s in > more places than before because AFAICT there's less impliciteness as to > which encoding to apply (regardless of whether it applies). > > Karsten > > > This might be one of the cases where Python 2's lack handling of string vs bytes was an advantage. If he was just scanning the message for specific ASCII strings, then not getting the full message decoded write is unlikely to have been causing problems. Python2 handled that sort of case quite easily. Python 3 on the other hand, will have issue converting the byte message to a string, since there isn't a single encoding that you could use for all of it all the time. This being 'fussier' does make sure that the program is handling all the text 'properly', and would be helpful if some of the patterns being checked for contained 'extended' (non-ASCII) characters. One possible solution in Python3 is to decode the byte string using an encoding that allows all 256 byte values, so it won't raise any encoding errors, just give your possibly non-sense characters for non-ASCII text. -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Re: Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?
On Fri, Aug 28, 2020 at 10:32 PM Richard Damon wrote: > > This might be one of the cases where Python 2's lack handling of string > vs bytes was an advantage. > > If he was just scanning the message for specific ASCII strings, then not > getting the full message decoded write is unlikely to have been causing > problems. > > Python2 handled that sort of case quite easily. Python 3 on the other > hand, will have issue converting the byte message to a string, since > there isn't a single encoding that you could use for all of it all the > time. This being 'fussier' does make sure that the program is handling > all the text 'properly', and would be helpful if some of the patterns > being checked for contained 'extended' (non-ASCII) characters. > > One possible solution in Python3 is to decode the byte string using an > encoding that allows all 256 byte values, so it won't raise any encoding > errors, just give your possibly non-sense characters for non-ASCII text. Why? If you want to work with bytes, work with bytes. There's no reason to decode in a meaningless way. Python 3 can handle the job of searching a bytestring for ASCII text just fine. Also, if you're parsing an email message, you can and should be doing so with respect to the encoding(s) stipulated in the headers, after which you will have valid Unicode text. Please don't spread misinformation like this. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
Richard Damon wrote: > On 8/28/20 7:50 AM, Karsten Hilbert wrote: > >>> No interpreation requires, since parsing failed. Then you can start > >>> dealing with these exceptions. _Do not_ write unparsable messages into > >>> an mbox! > >>> > >> Maybe I shouldn't but Python 2 has been managing to do so for several > >> years without any issues. > > I am inclined to congratulate you on that sheer amount of luck. I don't > > believe there were no issues because everything worked just right under > > py2 but rather because py2 cared less than py3 does now. > > > >> Are we saying that Python 3 really can't be made to handle things > >> 'tolerantly' like Python 2 used to? > > It sure should be possible but it will require *explicit* en/decode()s in > > more places than before because AFAICT there's less impliciteness as to > > which encoding to apply (regardless of whether it applies). > > > > Karsten > > > > > > > This might be one of the cases where Python 2's lack handling of string > vs bytes was an advantage. > > If he was just scanning the message for specific ASCII strings, then not > getting the full message decoded write is unlikely to have been causing > problems. > > Python2 handled that sort of case quite easily. Python 3 on the other > hand, will have issue converting the byte message to a string, since > there isn't a single encoding that you could use for all of it all the > time. This being 'fussier' does make sure that the program is handling > all the text 'properly', and would be helpful if some of the patterns > being checked for contained 'extended' (non-ASCII) characters. > > One possible solution in Python3 is to decode the byte string using an > encoding that allows all 256 byte values, so it won't raise any encoding > errors, just give your possibly non-sense characters for non-ASCII text. > But this will simply get some things quite wrong and produce garbage won't it? Whereas Python 2 would simply scramble the odd character. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
Stefan Ram wrote: > Chris Green writes: > >Therein lies the problem, the incoming byte stream *isn't* ASCII, it's > >an E-Mail message which may, for example, have UTF-8 or other encoded > >characters in it. Hopefully it will have an encoding given in the > >header but that's only if the sender is 'well behaved', one needs to > >be able to handle almost anything and it must be done without 'manual' > >interaction. > > I would make a difference between "scoring" and "transport": > > To transfer a message into an mbox it can be transferred as it is. > Just the bytes from the POP3 server. Let mutt deal with them. > That's what I do at present in Python 2, the problem is that Python 3 complains when I use the standard library to put the message into the mbox. I want to transport the message into my mbox and Python 3 won't do it without knowing how it's encoded whereas Python 2 just stuffed it in there 'as is'. I want Python 3's mailbox class to juyst put what I tell it (even if mis-formatted or mis-encoded) into the mbox. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
On Fri, Aug 28, 2020 at 10:51 PM Chris Green wrote: > > > One possible solution in Python3 is to decode the byte string using an > > encoding that allows all 256 byte values, so it won't raise any encoding > > errors, just give your possibly non-sense characters for non-ASCII text. > > > But this will simply get some things quite wrong and produce garbage > won't it? Whereas Python 2 would simply scramble the odd character. > It would. Don't do that. Either keep it as bytes, or decode it using the correct encoding. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
On 8/28/20 8:39 AM, Chris Green wrote: > Richard Damon wrote: >> On 8/28/20 7:50 AM, Karsten Hilbert wrote: > No interpreation requires, since parsing failed. Then you can start > dealing with these exceptions. _Do not_ write unparsable messages into > an mbox! > Maybe I shouldn't but Python 2 has been managing to do so for several years without any issues. >>> I am inclined to congratulate you on that sheer amount of luck. I don't >>> believe there were no issues because everything worked just right under >>> py2 but rather because py2 cared less than py3 does now. >>> Are we saying that Python 3 really can't be made to handle things 'tolerantly' like Python 2 used to? >>> It sure should be possible but it will require *explicit* en/decode()s in >>> more places than before because AFAICT there's less impliciteness as to >>> which encoding to apply (regardless of whether it applies). >>> >>> Karsten >>> >>> >>> >> This might be one of the cases where Python 2's lack handling of string >> vs bytes was an advantage. >> >> If he was just scanning the message for specific ASCII strings, then not >> getting the full message decoded write is unlikely to have been causing >> problems. >> >> Python2 handled that sort of case quite easily. Python 3 on the other >> hand, will have issue converting the byte message to a string, since >> there isn't a single encoding that you could use for all of it all the >> time. This being 'fussier' does make sure that the program is handling >> all the text 'properly', and would be helpful if some of the patterns >> being checked for contained 'extended' (non-ASCII) characters. >> >> One possible solution in Python3 is to decode the byte string using an >> encoding that allows all 256 byte values, so it won't raise any encoding >> errors, just give your possibly non-sense characters for non-ASCII text. >> > But this will simply get some things quite wrong and produce garbage > won't it? Whereas Python 2 would simply scramble the odd character. > Yes, when the message has extended characters, it will put the 'wrong' characters into the message, but if you are only looking for a fixed set of ASCII strings, especially in the headers of the message, that doesn't matter, those will still be there. It is a pragmatic short cut, something that is 'good enough' to get the job done, even if not 100% correct. As was elsewhere mentioned, you could also do at least most of the processing as bytes (this may need converting some of the strings being uses to bytes), but I don't know exactly what they are doing, so don't know if there is something that really needs a string. Basically, mail messages are complicated, and to 'properly' convert a message into a format for proper analysis would be significant work (and the result would NOT be a simple string), but it sounds like they didn't need that level of work with the messages. Particually if they only need to process the headers, and are working to try to 'whitelist' files to get them, being close and simple is likely good enough, -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?
> I want to transport the message into my mbox and Python 3 won't do it > without knowing how it's encoded whereas Python 2 just stuffed it in > there 'as is'. > > I want Python 3's mailbox class to juyst put what I tell it (even if > mis-formatted or mis-encoded) into the mbox. I guess using the mailbox class already implies that you do _not_ want to "simply put the msgs into an mbox file" but rather want the class to do what it was designed for: "put proper msgs properly into an mbox file". We can't have it both ways I fear. If we simply want to stuff a file with bytes and call that mbox we should do so: drop the bytes into a file and call it an mbox file. Karsten -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
Chris Angelico wrote: > > Also, if you're parsing an email message, you can and should be doing > so with respect to the encoding(s) stipulated in the headers, after > which you will have valid Unicode text. > But not all E-Mail messages are 'well behaved', the above works fine if the headers specify the correct text encoding but quite often one will get messages with no encoding specified and also one gets messages with the wrong encoding specified. One needs a way to handle these 'rogue' messages such that most of the characters come out right. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?
On 2020-08-28 08:30, Richard Damon wrote: > This might be one of the cases where Python 2's lack handling of string > vs bytes was an advantage. For English speaking Americans. > Python2 handled that sort of case quite easily. Python 3 on the other > hand, will have issue converting the byte message to a string, since > there isn't a single encoding that you could use for all of it all the > time. This being 'fussier' does make sure that the program is handling > all the text 'properly', and would be helpful if some of the patterns > being checked for contained 'extended' (non-ASCII) characters. > > One possible solution in Python3 is to decode the byte string using an > encoding that allows all 256 byte values, so it won't raise any encoding > errors, just give your possibly non-sense characters for non-ASCII text. Or simply handle the errors in the way that makes sense for your requirements. Check the section on error handling here: https://www.askpython.com/python/string/python-encode-and-decode-functions -- D'Arcy J.M. Cain Vybe Networks Inc. A unit of Excelsior Solutions Corporation - Propelling Business Forward http://www.VybeNetworks.com/ IM:da...@vybenetworks.com VoIP: sip:da...@vybenetworks.com signature.asc Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
On Fri, Aug 28, 2020 at 11:24 PM Chris Green wrote: > > Chris Angelico wrote: > > > > Also, if you're parsing an email message, you can and should be doing > > so with respect to the encoding(s) stipulated in the headers, after > > which you will have valid Unicode text. > > > But not all E-Mail messages are 'well behaved', the above works fine > if the headers specify the correct text encoding but quite often one > will get messages with no encoding specified and also one gets > messages with the wrong encoding specified. One needs a way to handle > these 'rogue' messages such that most of the characters come out right. > As D'Arcy posted, this is what the error handling is for. If you want to decode in a "sloppy" way such that mis-encoded text can be partially decoded, then that's what you want to do - decode with a sloppy error handler. Don't abuse arbitrary eight bit encodings in the hope that it'll do a better job just because it doesn't spit out any exceptions. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
On 8/28/20 8:44 AM, Chris Green wrote: > Stefan Ram wrote: >> Chris Green writes: >>> Therein lies the problem, the incoming byte stream *isn't* ASCII, it's >>> an E-Mail message which may, for example, have UTF-8 or other encoded >>> characters in it. Hopefully it will have an encoding given in the >>> header but that's only if the sender is 'well behaved', one needs to >>> be able to handle almost anything and it must be done without 'manual' >>> interaction. >> I would make a difference between "scoring" and "transport": >> >> To transfer a message into an mbox it can be transferred as it is. >> Just the bytes from the POP3 server. Let mutt deal with them. >> > That's what I do at present in Python 2, the problem is that Python 3 > complains when I use the standard library to put the message into the > mbox. > > I want to transport the message into my mbox and Python 3 won't do it > without knowing how it's encoded whereas Python 2 just stuffed it in > there 'as is'. > > I want Python 3's mailbox class to juyst put what I tell it (even if > mis-formatted or mis-encoded) into the mbox. > It looks like the mailbox class has gotten 'pickier' in Python 3, and won't accept a message as a byte string, just as either a email message or a real string. My guess would be that 'simplest' path would be to convert your message into a parsed Message class, and add that. -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Didn't understand the output of the following Python 3 code with reduce function?
I have seen this code on one of competative programming site but I didn't get it, Why output is 9? from functools import * def ADDS(a,b): return a+1 nums = [1, 2, 3, 4, 5, 6, 7, 8, 9] add = reduce(ADDS, nums) print(add) output: 9 -- https://mail.python.org/mailman/listinfo/python-list
RE: Didn't understand the output of the following Python 3 code with reduce function?
All the numbers in the nums list don't matter and aren't used. Only the first number, and how many there are. https://docs.python.org/3.8/library/functools.html#functools.reduce Basically it's doing ADDS(1, 2) which returns 2 that 2 gets fed back into ADDS(2, 3) which returns 3 that 3 gets fed back into ADDS(3, 4) which returns 4 ... ADDS(8, 9) which returns 9 > I have seen this code on one of competative programming site but I didn't get > it, Why output is 9? > > from functools import * > > def ADDS(a,b): > return a+1 > nums = [1, 2, 3, 4, 5, 6, 7, 8, 9] > add = reduce(ADDS, nums) > print(add) > > output: 9 -- https://mail.python.org/mailman/listinfo/python-list
Re: Didn't understand the output of the following Python 3 code with reduce function?
Shivlal Sharma wrote:
> I have seen this code on one of competative programming site but I didn't
> get it, Why output is 9?
>
> from functools import *
>
> def ADDS(a,b):
> return a+1
> nums = [1, 2, 3, 4, 5, 6, 7, 8, 9]
> add = reduce(ADDS, nums)
> print(add)
>
> output: 9
Rewrite the ADDS() function like so
>>> def ADDS(a, b):
... result = a + 1
... print(f"ADDS({a}, {b}) --> {result}")
... return result
...
to see what's going on:
>>> reduce(ADDS, [1,2,3,4,5,6,7,8,9])
ADDS(1, 2) --> 2
ADDS(2, 3) --> 3
ADDS(3, 4) --> 4
ADDS(4, 5) --> 5
ADDS(5, 6) --> 6
ADDS(6, 7) --> 7
ADDS(7, 8) --> 8
ADDS(8, 9) --> 9
9
ADDS is called with the first two values,
then with the first result (first value + 1) and the third value
then the second result ((first value + 1) + 1) and the fourth value...
This can be written recursively as
ADDS(ADDS(ADDS(ADDS(ADDS(ADDS(ADDS(ADDS(1, 2), 3), 4), 5), 6), 7), 8), 9)
As the second argument is always discarded we get the first a=1 eight times
incremented by one, i.e 9.
To drive the point home that we always get
first item + (len(items) -1):
>>> reduce(ADDS, [42,"a","b","c","d"])
ADDS(42, a) --> 43
ADDS(43, b) --> 44
ADDS(44, c) --> 45
ADDS(45, d) --> 46
46
--
https://mail.python.org/mailman/listinfo/python-list
Re: Video file to subtitles file
On Friday, August 28, 2020 at 12:27:25 AM UTC+5:30, Barry Scott wrote: > > On 27 Aug 2020, at 18:00, Muskan Sanghai wrote: > > > > I would be really thankful if someone can suggest me how can I generate > > subtitles file (srt format) from a video or audio without using Google > > cloud and AWS. > What do you know about how subtitles work with video? Do you mean you want to > extract the bitmap subtitle data from a MPEG video? > > Barry > > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > Thank you Barry for your reply, I just know the basics and I want to extract subtitles from a MPEG video and then put the subtitles in that same video. Subtitles can be of any format but it should be convenient for the entire procedure. -- https://mail.python.org/mailman/listinfo/python-list
Re: Video file to subtitles file
> On 28 Aug 2020, at 17:37, Muskan Sanghai wrote: > > On Friday, August 28, 2020 at 12:27:25 AM UTC+5:30, Barry Scott wrote: On 27 Aug 2020, at 18:00, Muskan Sanghai wrote: >>> >>> I would be really thankful if someone can suggest me how can I generate >>> subtitles file (srt format) from a video or audio without using Google >>> cloud and AWS. >> What do you know about how subtitles work with video? Do you mean you want >> to extract the bitmap subtitle data from a MPEG video? >> >> Barry >> >> >> >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> > > Thank you Barry for your reply, > I just know the basics and I want to extract subtitles from a MPEG video and > then put the subtitles in that same video. Subtitles can be of any format but > it should be convenient for the entire procedure. It seems you are looking for an App to do this work? I searched the web and saw this. https://www.openshot.org/ I have not used this app, maybe it’s a starting point for you. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Video file to subtitles file
On Sat, Aug 29, 2020 at 3:24 AM Barry wrote: > > > > > On 28 Aug 2020, at 17:37, Muskan Sanghai wrote: > > > > On Friday, August 28, 2020 at 12:27:25 AM UTC+5:30, Barry Scott wrote: > On 27 Aug 2020, at 18:00, Muskan Sanghai wrote: > >>> > >>> I would be really thankful if someone can suggest me how can I generate > >>> subtitles file (srt format) from a video or audio without using Google > >>> cloud and AWS. > >> What do you know about how subtitles work with video? Do you mean you want > >> to extract the bitmap subtitle data from a MPEG video? > >> > >> Barry > >> > >> > >> > >>> -- > >>> https://mail.python.org/mailman/listinfo/python-list > >>> > > > > Thank you Barry for your reply, > > I just know the basics and I want to extract subtitles from a MPEG video > > and then put the subtitles in that same video. Subtitles can be of any > > format but it should be convenient for the entire procedure. > > It seems you are looking for an App to do this work? > I searched the web and saw this. > > https://www.openshot.org/ > > I have not used this app, maybe it’s a starting point for you. > > Barry > Not familiar with Openshot, but it's worth looking into. Alternatively, I'd definitely recommend ffmpeg for anything like this sort of job. But if you actually need to OCR something, then you may need to do some scripting work. I don't have code to offer you, but it would involve FFMPEG to lift the images, something like Tesseract to do the actual OCRing, and then you'd write the rest of it yourself in Python. Other than that, this probably is something best done with a dedicated movie editing tool, not Python. Use what exists. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
RE: How do I place a preset into the text box?
Works like a charm, special thanks. Steve = Footnote: He called his wife to see if he should pick up Fish and Chips on the way home. She hung up on him. She is still fuming over letting him name the kids. -Original Message- From: Python-list On Behalf Of Cousin Stanley Sent: Friday, August 28, 2020 7:47 AM To: python-list@python.org Subject: Re: How do I place a preset into the text box? Steve wrote: > The following program compiles but does not quite do what I would like > it to do. Line 19 is the preset information but I do not seem to be > able to get it into the form by code. My purpose is to let the user > make changes without having to re-enter the entire code. > You might consider moving the entry get() function into the Submit() call back def Submit() : label.configure( text = 'The new code is : ' + NewCode.get() ) x = ( NewCode.get() ) print( "\n The new code entered is : " + x ) The following insert function will show the OldCode in the entry box OldCode = ( "1234-abcd" ) # CodeEntered = ttk.Entry( window , width = 15 , textvariable = NewCode ) CodeEntered.grid( column = 2 , row = 3 , pady = 10 ) CodeEntered.insert( 0 , OldCode ) -- Stanley C. Kitching Human Being Phoenix, Arizona -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: How do I place a preset into the text box?
Yes, the form/window now closes properly.
Removal of sys.exit() and inserting window.destroy() cleared up the exiting
problem.
Thanks.
I saw several variations on that but I guess I just never twerked it enough.
Footnote:
"What rhymes with orange?"
"No it doesn't.."
From: Colin McPhail
Sent: Friday, August 28, 2020 7:46 AM
To: Chris Narkiewicz via Python-list
Cc: Steve
Subject: Re: How do I place a preset into the text box?
Hi Steve,
On 28 Aug 2020, at 11:03, Steve mailto:Gronicus@SGA.Ninja> > wrote:
The following program compiles but does not quite do what I would like it to
do. Line 19 is the preset information but I do not seem to be able to get it
into the form by code. My purpose is to let the user make changes without
having to re-enter the entire code.
I'm no Tk expert but does the following do what you want? (Strictly
speaking, the parentheses in ("1234-abcd") are not wrong just unnecessary.)
#===
import tkinter as tk
from tkinter import ttk
import sys
window = tk.Tk()
window.title("Python Tkinter Text Box")
window.minsize(600,400)
def Submit():
label.configure(text= 'The new code is: ' + NewCode.get())
def ClickExit():
#This exit closes the program but the form remains and is still active.
# I want only to close the form.
print("Exiting")
#sys.exit()
window.destroy()
#OldCode = ("1234-abcd")
OldCode = "1234-abcd"
label = ttk.Label(window, text = "Enter the new code")
label.grid(column = 1, row = 1)
#NewCode = tk.StringVar()
NewCode = tk.StringVar(value=OldCode)
CodeEntered = ttk.Entry(window, width = 15, textvariable = NewCode)
CodeEntered.grid(column = 2, row = 3)
button = ttk.Button(window, text = "Submit", command = Submit)
button.grid(column= 2, row = 5)
button = ttk.Button(window, text = "Quit", command = ClickExit)
button.grid(column= 2, row = 7)
window.mainloop()
x = (NewCode.get())
print("The new code entered is: " + x)
#=
Regards,
Colin
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 how to convert a list of bytes objects to a list of strings?
On 28Aug2020 12:26, Chris Green wrote:
>Cameron Simpson wrote:
>> POP3 is presumably handing you bytes containing a message. If the
>> Python
>> email.BytesParser doesn't handle it, stash the raw bytes _elsewhere_ in
>> a distinct file in some directory.
>>
>> with open('evil_msg_bytes', 'wb') as f:
>> for bs in bbb:
>> f.write(bs)
>>
>> No interpreation requires, since parsing failed. Then you can start
>> dealing with these exceptions. _Do not_ write unparsable messages into
>> an mbox!
>>
>Maybe I shouldn't but Python 2 has been managing to do so for several
>years without any issues. I know I *could* put the exceptions in a
>bucket somewhere and deal with them separately but I'd really rather
>not.
>
>At prsent (with the Python 2 code still installed) it all 'just works'
>and the absolute worst corruption I ever see in an E-Mail is things
>like accented characters missing altogether or £ signs coming out as a
>funny looking string. Either of these don't really make the message
>unintelligible.
>
>Are we saying that Python 3 really can't be made to handle things
>'tolerantly' like Python 2 used to?
It can, but if you're decoding bytes to strings without the correct
encoding then rubbish will be happening. In Python 2 also, it just isn't
being flagged.
One approach would be to break your parser process up:
- collect bytes from POP3
- parse headers for filtering purposes
- those you keep, append to the mbox _as bytes_
It sounds like your filter is uninterested in the message body, so you
don't need to decode it at all. Just ensure the bod has no embedded
lines with b'From ' at the start, and ensure the last line ends in a
newline b'\n' or that you append one, so that the b'From ' of the next
message is recognised.
So: collect bytes, decode ehaders and parse/filter, save _bytes_.
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
Re: Didn't understand the output of the following Python 3 code with reduce function?
Shivlal Sharma writes: > I have seen this code on one of competative programming site but I > didn't get it, Why output is 9? > > from functools import * > > def ADDS(a,b): > return a+1 > nums = [1, 2, 3, 4, 5, 6, 7, 8, 9] > add = reduce(ADDS, nums) > print(add) > > output: 9 Hint: reduce(f, [e1, e2]) is f(e1, e2) reduce(f, [e1, e2, e3]) is f(f(e1, e2), e3) reduce(f, [e1, e2, e3, e4]) is f(f(f(e1, e2), e3), e4) Replace f with a function that adds one to its first argument. Does that help? -- Ben. -- https://mail.python.org/mailman/listinfo/python-list
Fwd: Issue in installing Python (Windows 10)
-- Forwarded message - From: Debasis Chatterjee Date: Fri, Aug 28, 2020 at 3:41 AM Subject: Re: Issue in installing Python (Windows 10) To: Terry Reedy Hi Terry, OK. I think the issue is now solved. I used "pip uninstall" and then "pip install". With that I can run Python programs that use the installed packages. By the way, is there a site where I can login to see such mails and also manage mail notification options? The same as https://python-forum.io/member.php ? Or another one? Thank you Debasis On Wed, Aug 26, 2020 at 5:03 PM Debasis Chatterjee wrote: > Hi Terry, > > I still have the issue. I took help from my company's IT Admin and > deinstalled everything and then made fresh installation. > > Still have issues the packages. > > Such as we need SEGYIO in the program. Program fails to find that. > > When I try to add using "pip" then I get this message. > > C:\Users\debasisc>pip install segyio > Defaulting to user installation because normal site-packages is not > writeable > Requirement already satisfied: segyio in > c:\users\debasisc\appdata\roaming\python\python38\site-packages (1.9.1) > Requirement already satisfied: numpy>=1.10 in > c:\users\debasisc\appdata\roaming\python\python38\site-packages (from > segyio) (1.19.1) > WARNING: You are using pip version 20.1.1; however, version 20.2.2 is > available. > You should consider upgrading via the 'c:\program files > (x86)\python38-32\python.exe -m pip install --upgrade pip' command. > > What do you suggest for troubleshooting? > > Thank you > > Debasis > > On Mon, Aug 24, 2020 at 3:19 PM Terry Reedy wrote: > >> On 8/23/2020 12:39 PM, Debasis Chatterjee wrote: >> >> > I started off by using "python-3.8.5.exe". >> >> 32-bit Windows installer? Windows version might be relevant. >> >> > I use "Run as Administrator" option to click this (provide my >> local-admin >> > username/pwd). >> >> > After this, I see python shell available. But no IDLE. Not sure why? >> >> You mean no IDLE entry on Python 3.8 directory on Start menu? >> >> Installer has a checkbox for tkinter and IDLE. It is on by default for >> a fresh install but I expect it defaults to previous install for >> reinstalls. >> >> On a command line, run 'python -c "import tkinter' to see if you have >> tkinter. 'python -m idlelib' starts IDLE if present. >> >> > From CMD, I can check "pip list". Surprisingly, it shows me packages >> that I >> > installed earlier, before deinstalling and reinstalling. >> >> De-installing does not remove pythonxy directory if you add anything to >> it. >> >> > Something is not right. Hence I am trying to make a fresh restart. >> > >> > One thing odd is that even after removing installed program (Python) >> from >> > control panel, I still find that from "Start". >> >> Have you installed more than one version of Python? Or both 32 and 64 >> bit variation of same Python version? >> >> >> -- >> Terry Jan Reedy >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Issue in installing Python (Windows 10)
On 2020-08-28 at 18:38:03 -0500, Debasis Chatterjee wrote: > By the way, is there a site where I can login to see such mails and > also manage mail notification options? Better than that, you can have the emails themselves delivered directly to your inbox: https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
