Re: WedWonder: Scripts and Modules
On 12/09/19 10:37 AM, Alan Bawden wrote: DL Neil writes: ... However, reversing the question in my mind led me to ask (myself): how many scripts do I have (in "production use") which are ever used (also) as a module by some other script? I think the answer is/was: "none"! Accordingly, (spoiler alert: this statement may be heresy) I stopped using the "if". I found that the problem with doing this is that `pydoc' now _runs_ my scripts, when all I wanted was a quick look at the constants and procedures defined in them. I haven't experienced this. However, I'll make a note to test... -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: Email messages from grouped email using IMAPClient in Python.
On 12/09/19 5:06 PM, Srinivas Pullabhotla wrote:
Hello,
I am trying to fetch email messages from a gmail inbox. So, there will be 1000s
of messages sent to Inbox and since they are 1000s, the emails are grouped 100
per each email item.
When I tried this method, the program only fetches some but not all and
especially it happens with grouped email messages. How can I get all the
messages from that grouped email.
import email, time, sys
from imapclient import IMAPClient
with IMAPClient(HOST) as server:
server.login(USERNAME, PASSWORD)
server.select_folder('INBOX', readonly=True)
messages = server.search(['ALL', 'UNSEEN'])
for uid, message_data in server.fetch(messages, 'RFC822').items():
email_message = email.message_from_bytes(message_data[b'RFC822'])
print(email_message.get_payload(None, True))
The program should fetch all the email messages from the grouped email and
output to the file (in my case I am grabbing the href links). How best I can
achieve this ? Appreciate thoughts and suggestions.
First debug is to assign the server.select_folder() return values, and
subsequently print them for inspection. (know what you are dealing
with/that you have been given something to work with)
The server.search() doesn't request ALL of the msgs. Are you sure those
missing from the 'group' are not marked as 'read'? So, second debug
would be to remove the 'UNSEEN' criteria and observe any difference.
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list
Email messages from grouped email using IMAPClient in Python.
Hello,
I am trying to fetch email messages from a gmail inbox. So, there will be 1000s
of messages sent to Inbox and since they are 1000s, the emails are grouped 100
per each email item.
When I tried this method, the program only fetches some but not all and
especially it happens with grouped email messages. How can I get all the
messages from that grouped email.
import email, time, sys
from imapclient import IMAPClient
with IMAPClient(HOST) as server:
server.login(USERNAME, PASSWORD)
server.select_folder('INBOX', readonly=True)
messages = server.search(['ALL', 'UNSEEN'])
for uid, message_data in server.fetch(messages, 'RFC822').items():
email_message = email.message_from_bytes(message_data[b'RFC822'])
print(email_message.get_payload(None, True))
The program should fetch all the email messages from the grouped email and
output to the file (in my case I am grabbing the href links). How best I can
achieve this ? Appreciate thoughts and suggestions.
--
https://mail.python.org/mailman/listinfo/python-list
Re: WedWonder: Scripts and Modules
On 12Sep2019 08:24, DL Neil wrote:
In this day-and-age do you have a script in live/production-use,
which is also a module? What is the justification/use case?
Many. Many many.
1: Many of my modules run their unit tests if invoked as the main
programme.
2: Several modules are their own utility programme. I've got a heap of
these - anything that provides useful access to something can often be
usefully used from the command line.
A quick grep from my bin directory shows stuff like this:
[~/hg/css/bin(hg:default)]fleet*> grep ' -m cs.' *
beyonwiz:exec python3 -m cs.app.beyonwiz ${1+"$@"}
calibre:exec py3 -m cs.app.calibre ${1+"$@"}
haproxy-tool:exec python2 -m cs.app.haproxy ${1+"$@"}
iphoto:exec python3 -m cs.app.osx.iphoto ${1+"$@"}
maildb:exec python3 -m cs.app.maildb ${1+"$@"}
mailfiler:exec python3 -m cs.app.mailfiler ${1+"$@"}
mklinks:exec python -m cs.app.mklinks ${1+"$@"}
myke:exec python3 -m cs.app.myke ${1+"$@"}
nodedb:exec python -m cs.nodedb.__init__ "$CS_NODEDB_URL" "$op" ${1+"$@"}
pilfer:exec python3 -m cs.app.pilfer ${1+"$@"}
portfwd:exec python3 -m cs.app.portfwd ${1+"$@"}
s3-clone-website: set -- python3 -m cs.app.aws s3 "$s3bucket" sync-up -D -% .
svcd:exec python3 -m cs.app.svcd ${1+"$@"}
vbox:exec python -m cs.app.virtualbox ${1+"$@"}
vt:exec python3 -m cs.vt ${1+"$@"}
wol:exec python -m cs.wol ${1+"$@"}
Consider: if you write a package, would it have a __main__.py?
Well, if the answer is ever "yes" then the same applies to ordinary
modules, simple enough to not be worth splitting onto a package.
So, yes, for me this is really really common.
Even for my current client project, which is largely a package,
several of the individual modules within the package have their own
main programmes for testing and for various utility tasks dealing
solely with that particular subsystem. There's an overarching shell
script to set up the environment and then do various things from the
command line, and it directly invokes particular modules for some
operations that act only on one subsystem.
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
Re: WedWonder: Scripts and Modules
DL Neil writes: > ... However, reversing the question in my mind led me to ask (myself): > how many scripts do I have (in "production use") which are ever used > (also) as a module by some other script? I think the answer is/was: > "none"! Accordingly, (spoiler alert: this statement may be heresy) I > stopped using the "if". I found that the problem with doing this is that `pydoc' now _runs_ my scripts, when all I wanted was a quick look at the constants and procedures defined in them. It's pretty handy sometimes to fire up `pydoc' in http server mode in a directory full of random Python code (some scripts and some modules), and use it to browse around to figure out what everything does. Scripts that aren't protected with the usual `if __name__' are more likely to crash in that situation, often crashing `pydoc' in the process. In fact, I have been known to add `if __name__' to my colleagues' Python scripts, just so that I can safely `pydoc' them. -- Alan Bawden -- https://mail.python.org/mailman/listinfo/python-list
Re: WedWonder: Scripts and Modules
On Thu, Sep 12, 2019 at 7:34 AM DL Neil via Python-list wrote: > > On 12/09/19 8:43 AM, Chris Angelico wrote: > > Yes, absolutely. It's the easiest way to share code between two > > scripts. Here's an example that I created recently: > > > > https://github.com/Rosuav/shed/blob/master/BL1_find_items.py > > https://github.com/Rosuav/shed/blob/master/BL2_find_items.py > > Interestingly, we appear to have opposite approaches to this situation - > as soon as I think the word "common" it morphs immediately into "module". I think we actually both agree here. The only difference is that in my view, a module that's needed by only one other script can remain as the script it is, whereas you instantly make it *exclusively* an importable module. BL1_find_items *is* a module as well as a script, and it has to be implemented with that in mind. > I have also needed to scan a directory/tree recently, but took the > course of refactoring a previous implementation's 'directory-walk' code > into its own (generator) function, and thus the mainline of the newer > application calls the utility function and chooses whether or not to > deal with each 'found' file/dir in-turn. The previous script using the > code was similarly refactored. > (and there are no doubt ?many more scripts 'lurking' in my code-base > which could be similarly 'improved' - such frequent commonality leading > to my preference). > > My bias (I'm not criticising/complaining about/decrying the choices you > have illustrated) probably comes out of "separation of concerns". An > issue which has 'bitten' me, more than once... > > For example both BL1 and BL2 feature: > > def money(savefile): savefile.money[0] += 500 # Add more dollars > > - a minor issue, agreed (and 'picking on it' purely to make a point). Actually no they don't, and that's part of the subtlety of working with the two distinctly different file formats. In BL2, money is an array (dollars, Eridium, Seraph tokens, etc), but in BL1, it's a single value. So the BL2 version is as you posted, but the BL1 one is slightly different: def money(savefile): savefile.money += 500 Both files follow the same structure of synthesizers (hence the common code in the FunctionArg class), so they will look similar, but they're not identical enough to actually share. :( > The "s-o-c" is, that one day it will be possible to decide that the unit > of addition should change, but only (remember to) amend the code in one > of the two scripts! If I cared that the amount added be the same, then I could put that into a constant, I guess; if anything, what I might do is parameterize it, but it's a fairly arbitrary figure and doesn't really matter much. > Which brings me back to the preference for 'encapsulation' (just to > prove I can speak "OOP"), and why I would have written quite separate > 'main-lines' and extracted money() (and presumably a lot more) into a > module which could then be called by both. When I started the BL1 project, I thought about sharing heaps of code with BL2, but it just didn't work out. > Aside2: > Meantime, thanks for the opportunity to review your code. I was thinking > of considering "data classes" (new in v3.7, IIRC) in relation to an SQL > interface on today's ToDo list - constructing the application's queries > and the data transfers 'in' and 'out', without the 'weight' of > SQLAlchemy. Perhaps? > Data classes are awesome. Also, this is a great demonstration of why annotations shouldn't be restricted to any particular definition of "type" - it's wonderfully elegant to define a data structure with constants for signatures, types for most values, and then use range(256) or range(65536) to mean 1-byte or 2-byte integers. Within the context of the parser/encoder here, these things ARE types, but in a general Python context, they aren't :) Incidentally, data classes can be used pre-3.7 if you pip install the backport module. So it doesn't have to be a hard requirement. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: WedWonder: Scripts and Modules
On 12/09/19 8:43 AM, Chris Angelico wrote: On Thu, Sep 12, 2019 at 6:34 AM DL Neil via Python-list wrote: In this day-and-age do you have a script in live/production-use, which is also a module? What is the justification/use case? Yes, absolutely. It's the easiest way to share code between two scripts. Here's an example that I created recently: https://github.com/Rosuav/shed/blob/master/BL1_find_items.py https://github.com/Rosuav/shed/blob/master/BL2_find_items.py These programs do similar jobs on very different formats of file, so there's a small amount of common code and a large amount that isn't common. One of the common sections is the FunctionArg class, which ties in with argparse; it's implemented in BL1_find_items, and then imported into BL2_find_items. Of course I could break this out into its own dedicated file... but why bother? It's not significant enough to warrant its own module, and I don't see any value in an importable file of "all the stuff that I might feel like importing"; it's just these two files that will need this. Basically, the script/module distinction is a convenient way to simplify a common situation that doesn't need the overhead of anything else. If the code starts getting used in lots more places, it'll eventually get promoted to actual module. Thanks for such a rapid response! Interestingly, we appear to have opposite approaches to this situation - as soon as I think the word "common" it morphs immediately into "module". I have also needed to scan a directory/tree recently, but took the course of refactoring a previous implementation's 'directory-walk' code into its own (generator) function, and thus the mainline of the newer application calls the utility function and chooses whether or not to deal with each 'found' file/dir in-turn. The previous script using the code was similarly refactored. (and there are no doubt ?many more scripts 'lurking' in my code-base which could be similarly 'improved' - such frequent commonality leading to my preference). My bias (I'm not criticising/complaining about/decrying the choices you have illustrated) probably comes out of "separation of concerns". An issue which has 'bitten' me, more than once... For example both BL1 and BL2 feature: def money(savefile): savefile.money[0] += 500 # Add more dollars - a minor issue, agreed (and 'picking on it' purely to make a point). The "s-o-c" is, that one day it will be possible to decide that the unit of addition should change, but only (remember to) amend the code in one of the two scripts! (when it comes to trusting my memory, I'm an 'old git' with history, but not an old git repo with "history"!) Which brings me back to the preference for 'encapsulation' (just to prove I can speak "OOP"), and why I would have written quite separate 'main-lines' and extracted money() (and presumably a lot more) into a module which could then be called by both. Aside1: However, you've started me thinking about a related consideration/philosophy (Danger Will Robinson!) - but please let me cogitate on that for a day or so... Aside2: Meantime, thanks for the opportunity to review your code. I was thinking of considering "data classes" (new in v3.7, IIRC) in relation to an SQL interface on today's ToDo list - constructing the application's queries and the data transfers 'in' and 'out', without the 'weight' of SQLAlchemy. Perhaps? -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: WedWonder: Scripts and Modules
On Thu, Sep 12, 2019 at 6:34 AM DL Neil via Python-list wrote: > > In this day-and-age do you have a script in live/production-use, which > is also a module? What is the justification/use case? > Yes, absolutely. It's the easiest way to share code between two scripts. Here's an example that I created recently: https://github.com/Rosuav/shed/blob/master/BL1_find_items.py https://github.com/Rosuav/shed/blob/master/BL2_find_items.py These programs do similar jobs on very different formats of file, so there's a small amount of common code and a large amount that isn't common. One of the common sections is the FunctionArg class, which ties in with argparse; it's implemented in BL1_find_items, and then imported into BL2_find_items. Of course I could break this out into its own dedicated file... but why bother? It's not significant enough to warrant its own module, and I don't see any value in an importable file of "all the stuff that I might feel like importing"; it's just these two files that will need this. Basically, the script/module distinction is a convenient way to simplify a common situation that doesn't need the overhead of anything else. If the code starts getting used in lots more places, it'll eventually get promoted to actual module. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
WedWonder: Scripts and Modules
In this day-and-age do you have a script in live/production-use, which is also a module? What is the justification/use case? (discounting distutils and similar installation tools, or unit testing methodology) There are over 500 questions on StackOverflow which refer to Python's if __name__ == __main__: construct. Even more if you include the idea of a main() multiple entry-point. This construct enables code to distinguish between being "run" as a "script", and being imported as a "module". (which is not in question) It intrigues me how often (per SO, quoted above) this causes anything from a pause to complete consternation amongst Python neophytes. In my own case, I accepted it as a "Pythonic" idiom, adopted its use in my scripts, and moved on. Until I adopted unittest/pytest and took-on the closer definitions of TDD, I used to use the script/module switch as a means of testing modules (see also 'good, old days', below). These days, pytest requires a different approach and splits 'test code' from (production) "delivered code". Oh yeah! However, I can't recall ever gaining similar benefit from using the 'switch' within code designed to be a "script"! Ages ago some beginner asked me the 'script/module switch' question, and I gave the standard/formulaic/Pythonic answer. However, reversing the question in my mind led me to ask (myself): how many scripts do I have (in "production use") which are ever used (also) as a module by some other script? I think the answer is/was: "none"! Accordingly, (spoiler alert: this statement may be heresy) I stopped using the "if". Back in the ?good, old days of mainframes we wrote "monolithic" programs. Then we developed "modular programming" and started to split code into functional units, ie functions, subroutines, paragraphs, procedures. Thus "code re-use" was born. (see what a new idea it is! Subroutine libraries on mag-tape, anyone?) We distinguished a subroutine or "called code" (importable module in Python) from the calling-code by calling the latter the "main-line" (nothing to do with/say no to drugs). We even developed little "stub" programs; which would exercise or test specific subroutines to ensure that they worked (wow, how new is (much of) Test-Driven Development?) So (putting nostalgia aside), these days my Python *scripts* are pure "main-line" code. Faced with the perennial "main/name" question again yesterday, led me to review the above policy/laziness, and thus this "Wednesday Wondering": - modules aside, how often do we write multiple-entry code these days, as opposed to having multiple scripts/main-lines which call re-usable functional modules, as-and-when? I still don't have a single file containing a combination script/module amongst my applications. Do you? Why/who/how/when? -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: Issue with Python installation
On 11/09/2019 17:48, Manasiz Paul wrote: Dear Sir/Madam, I have installed the latest version of Python but while running it, I am facing this issue continuously. [image: Untitled.png] I'm afraid the mailing list stripped off your attachment. Please copy and paste the error messages into the text of your email, that way we will get to see them! (The temptation to say "Use your words" is oh so strong :-) -- Rhodri James *-* Kynesim Ltd -- https://mail.python.org/mailman/listinfo/python-list
Re: Get Count of function arguments passed in
Sayth Renshaw schreef op 11/09/2019 om 12:11: I want to allow as many lists as needed to be passed into a function. But how can I determine how many lists have been passed in? I expected this to return 3 but it only returned 1. matrix1 = [[1, -2], [-3, 4],] matrix2 = [[2, -1], [0, -1]] matrix3 = [[2, -1], [0, -1]] # print(add(matrix1, matrix2)) def add(*matrix): print(len(locals())) May I suggest renaming matrix to matrices or matrix_list or something? I find that much clearer to read and understand: singular for one object, plural for a collection of objects. def add(*matrices): print(len(matrices)) -- "Honest criticism is hard to take, particularly from a relative, a friend, an acquaintance, or a stranger." -- Franklin P. Jones Roel Schroeven -- https://mail.python.org/mailman/listinfo/python-list
Issue with Python installation
Dear Sir/Madam, I have installed the latest version of Python but while running it, I am facing this issue continuously. [image: Untitled.png] Kindly let me know how to resolve this issue. -- https://mail.python.org/mailman/listinfo/python-list
Re: Get Count of function arguments passed in
> I expected this to return 3 but it only returned 1. > > matrix1 = [[1, -2], [-3, 4],] > matrix2 = [[2, -1], [0, -1]] > matrix3 = [[2, -1], [0, -1]] > > def add(*matrix): > print(len(locals())) > > print(add(matrix1, matrix2)) In this case, locals will be a dictionary with exactly one key. The key will be "matrix", and its value will be a tuple of the two matrices matrix1 and matrix2. One solution (the wrong solution) to your problem would be to instead have # Don't do this --- this is to illustrate what locals is doing def add(*matrix): print(len(locals()['matrix'])) Now let's get to the right way to do this. Python stores the arguments specified by *matrix as a tuple of the name `matrix`. Of course, this is what locals what telling us, but there is no need to go through locals. Instead, you can do def add(*matrix): print(len(matrix)) and this will tell you how many arguments were passed in. Good luck! David Lowry-Duda -- https://mail.python.org/mailman/listinfo/python-list
Re: Get Count of function arguments passed in
Le 11/09/2019 à 12:11, Sayth Renshaw a écrit : Hi I want to allow as many lists as needed to be passed into a function. But how can I determine how many lists have been passed in? I expected this to return 3 but it only returned 1. matrix1 = [[1, -2], [-3, 4],] matrix2 = [[2, -1], [0, -1]] matrix3 = [[2, -1], [0, -1]] # print(add(matrix1, matrix2)) def add(*matrix): print(len(locals())) add(matrix1,matrix2,matrix3) Cheers Sayth It returns 1 because there is only 1 local variable inside function add. It's a list matrix which contains the 3 matrix If you want the number of arguments passed, then just call: len(matrix) def add(*matrix): print(len(matrix)) -- https://mail.python.org/mailman/listinfo/python-list
Re: Get Count of function arguments passed in
On Wednesday, 11 September 2019 20:25:32 UTC+10, Sayth Renshaw wrote: > On Wednesday, 11 September 2019 20:11:21 UTC+10, Sayth Renshaw wrote: > > Hi > > > > I want to allow as many lists as needed to be passed into a function. > > But how can I determine how many lists have been passed in? > > > > I expected this to return 3 but it only returned 1. > > > > matrix1 = [[1, -2], [-3, 4],] > > matrix2 = [[2, -1], [0, -1]] > > matrix3 = [[2, -1], [0, -1]] > > # print(add(matrix1, matrix2)) > > > > def add(*matrix): > > print(len(locals())) > > > > add(matrix1,matrix2,matrix3) > > > > Cheers > > > > Sayth > > Tried creating a list of the arguments, however I am creating too many > positional arguments. > > matrix1 = [[1, -2], [-3, 4],] > matrix2 = [[2, -1], [0, -1]] > matrix3 = [[2, -1], [0, -1]] > > matrix = [] > def add_many(a = list(*matrix)): > for num in a: > for i in range(len(matrix[num])): > for j in range(len(matrix[num])): > print(matrix[num][i][j] + matrix[num][i][j]) > > add_many(matrix1,matrix2) Last failure for the moment matrix1 = [[1, -2], [-3, 4],] matrix2 = [[2, -1], [0, -1]] matrix3 = [[2, -1], [0, -1]] matrix = [] def add_many(*matrix): for num in range(add_many().func_code.co_argcount): for i in range(len(matrix[num])): for j in range(len(matrix[num])): print(matrix[i][j] + matrix[i][j]) add_many(matrix1,matrix2) Sayth -- https://mail.python.org/mailman/listinfo/python-list
Re: Get Count of function arguments passed in
On Wednesday, 11 September 2019 20:11:21 UTC+10, Sayth Renshaw wrote: > Hi > > I want to allow as many lists as needed to be passed into a function. > But how can I determine how many lists have been passed in? > > I expected this to return 3 but it only returned 1. > > matrix1 = [[1, -2], [-3, 4],] > matrix2 = [[2, -1], [0, -1]] > matrix3 = [[2, -1], [0, -1]] > # print(add(matrix1, matrix2)) > > def add(*matrix): > print(len(locals())) > > add(matrix1,matrix2,matrix3) > > Cheers > > Sayth Tried creating a list of the arguments, however I am creating too many positional arguments. matrix1 = [[1, -2], [-3, 4],] matrix2 = [[2, -1], [0, -1]] matrix3 = [[2, -1], [0, -1]] matrix = [] def add_many(a = list(*matrix)): for num in a: for i in range(len(matrix[num])): for j in range(len(matrix[num])): print(matrix[num][i][j] + matrix[num][i][j]) add_many(matrix1,matrix2) -- https://mail.python.org/mailman/listinfo/python-list
Get Count of function arguments passed in
Hi I want to allow as many lists as needed to be passed into a function. But how can I determine how many lists have been passed in? I expected this to return 3 but it only returned 1. matrix1 = [[1, -2], [-3, 4],] matrix2 = [[2, -1], [0, -1]] matrix3 = [[2, -1], [0, -1]] # print(add(matrix1, matrix2)) def add(*matrix): print(len(locals())) add(matrix1,matrix2,matrix3) Cheers Sayth -- https://mail.python.org/mailman/listinfo/python-list
