Re: [Tutor] directory structure with tests?
On 06Mar2019 23:41, Alan Gauld wrote: On 06/03/2019 22:38, Mats Wichmann wrote: "It depends". ... How's that for a definitive answer? :) Pretty good! and I agree with all of it :-) Although personally I always use a test subdirectory and then, for building a distributable, use a file filter (eg find in *nix) to prune those out. All version controlled of course! There are those who argue that it is better to include the test files in the distribution. Aside from transparency and completeness, it may also aid bug reports as a test might only fail in the end user's environment and not that of the developer. Being able to get the error output from the end user running the tests is a win there. Q: How many user support people does it take to change a light bulb? A: We have an exact copy of the light bulb here and it seems to be working fine. Can you tell me what kind of system you have? For variety, in my own code I keep the tests for foo.py in foo_tests.py, the better to be seen next to foo.py in listings and file completion. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] directory structure with tests?
On 06/03/2019 22:38, Mats Wichmann wrote: > "It depends". ... > How's that for a definitive answer? :) Pretty good! and I agree with all of it :-) Although personally I always use a test subdirectory and then, for building a distributable, use a file filter (eg find in *nix) to prune those out. All version controlled of course! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] directory structure with tests?
On 3/6/19 3:17 PM, James Hartley wrote: > It is preferable to sprinkle tests files throughout the directories of a > project, or coalesce all tests in a test directory? "It depends". There are people who have strong opinions. If you're going to use conventions like naming the test for the functionality in foo.py as test_foo.py, it makes sense for the two to be nearby each other. That can be the same directory, or a subdirectory (a harness like Py.Test will find files named test_* and assume they're tests whichever place they're in). Very large projects may well split tests into more subdirectories based on their purpose or usage. On the other hand, if your project is going to be turned into a package, you might want to keep the tests in a separate directory, as it may be somewhat easier to select what goes into the package if they're not sprinkled together. You may also end up handling unit tests and integration tests differently. How's that for a definitive answer? :) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] directory structure with tests?
It is preferable to sprinkle tests files throughout the directories of a project, or coalesce all tests in a test directory? Thanks! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On 30Sep2017 07:35, Chris wrote:
On Sat, 30 Sep 2017 00:23:45 +0100
Alan Gauld via Tutor wrote:
If you really need it in memory there are some generic modules
for building tree structures in memory - you will need to
define the mail objects of course the modules are data neutral,
they just allow you to create and navigate the tree, populating
it with whatever objects you want.
Ok, can you recommend some specific?
Maybe not, but for your needs wouldn't nested dicts do the trick? Or nested
objects?
class MaildirInfo(object):
def __init__(self, path):
self.path = path
self.subdirs = [] # array of more Maildirinfo instances
self.other stuff here as needed ...
top = MaildirInfo("top-maildir-path")
then do the os.walk thing and fill out the data structure.
.subdirs could also be a dict mapping named to MaildirInfo instance, depending
on your needs.
Rather than Alan's take that nobody would want to do this, as someone who has
spent much time mucking with my mail tree using many scripts I'd say: everyone
wants to do something different. The task itself is fairly simple, so roll your
own. That makes is easy to implement your own policy instead of trying to do
some kind of feature survey of stuff out there.
Cheers,
Cameron Simpson (formerly c...@zip.com.au)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
Thank you for your detailed reply! I've tried to explain in my mails to
Alan and Mats what I'm trying to achieve.
On Sat, 30 Sep 2017 11:32:57 +1000
Steven D'Aprano wrote:
> On Fri, Sep 29, 2017 at 07:02:07PM +0200, Chris wrote:
>
> > Background: Maildirs with mails older than five years should be
> > archived. The folder structure should be kept in the target.
>
> Archived to what?
A separate Maildir on tape.
> > I was very surprised, that there seems no readily usable module
> > available. (In Perl neither).
>
> Reusable module to do what *precisely*? If you cannot explain what
> you need, how do you expect somebody to have anticipated your
> requirements and written a module to do it?
Represent the structure in memory.
> > What's the best way to save them?
>
> Depends on what you are doing. But coding the paths in your source
> code is almost certainly not what you want to do. Surely you want to
> read the paths from the maildir itself, as it *actually* exists,
> rather than try to hard-code what you expect it to be in your source
> code?
Well, if I had the structure in memory, I could save additional
information and could print different lists, e.g. sorted by attachment
size, sorted by project. A project can appear in different places
in the tree.
> Have you looked at the contents of a maildir? Its actually an almost
> flat structure. Nested mail folders are not nested on the disk: a
> user's mail folder structure that looks like:
>
> inbox
> sent
> trash
> personal
> ├── family
> └── friends
> work
> ├── critical
> ├── important
> └── low
>
>
> is stored on disk as:
>
> Maildir/
> ├── .sent/
> ├── .trash/
> ├── .personal/
> ├── .personal.family/
> ├── .person.friends/
> ├── .work
> ├── .work.critical
> ├── .work.important
> └── .work.low
Good objection. You can make dovecot use the first layout on disk.
Probably not a gain in disguise.
> So all you really need is to record the path to the top level maildir
> directories (the original, and the place where you are archiving
> them). The subdirectories, you read from the disk as you go.
Ok, I could even do this with the first structure.
> Actually, *you* don't read them at all. Have you looked at the
> mailbox module in the standard library? It supports Maildir. I expect
> that what you would do is something like:
>
> source = Maildir('path/to/source')
> archive = Maildir('path/to/archive')
> for each directory in source:
> for each mail in directory:
> if mail older than five years:
> copy mail to archive
> delete mail from source
I've used os.walk. I'll have a look at the Maildir library.
- Chris
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On Fri, Sep 29, 2017 at 07:02:07PM +0200, Chris wrote:
> Background: Maildirs with mails older than five years should be
> archived. The folder structure should be kept in the target.
Archived to what?
> I was very surprised, that there seems no readily usable module
> available. (In Perl neither).
Reusable module to do what *precisely*? If you cannot explain what you
need, how do you expect somebody to have anticipated your requirements
and written a module to do it?
> What's the best way to save them?
Depends on what you are doing. But coding the paths in your source code
is almost certainly not what you want to do. Surely you want to read the
paths from the maildir itself, as it *actually* exists, rather than try
to hard-code what you expect it to be in your source code?
Have you looked at the contents of a maildir? Its actually an almost
flat structure. Nested mail folders are not nested on the disk: a
user's mail folder structure that looks like:
inbox
sent
trash
personal
├── family
└── friends
work
├── critical
├── important
└── low
is stored on disk as:
Maildir/
├── .sent/
├── .trash/
├── .personal/
├── .personal.family/
├── .person.friends/
├── .work
├── .work.critical
├── .work.important
└── .work.low
(Not shown: the mails themselves, some dovecot specific data files
in the top level, and the special cur/ new/ tmp/ subdirectories found
in each mail directory.)
So all you really need is to record the path to the top level maildir
directories (the original, and the place where you are archiving them).
The subdirectories, you read from the disk as you go.
Actually, *you* don't read them at all. Have you looked at the mailbox
module in the standard library? It supports Maildir. I expect that what
you would do is something like:
source = Maildir('path/to/source')
archive = Maildir('path/to/archive')
for each directory in source:
for each mail in directory:
if mail older than five years:
copy mail to archive
delete mail from source
and let the mailbox module do the hard work. (Especially the part about
copying the mail, since the module will ensure that the flags and dates
are copied correctly.)
--
Steve
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On 29/09/17 20:34, Chris wrote: > I want to store some kind of representation of the tree in memory when > the code runs. Then I could not only move the mails but also create > lists, e.g. a table with mail headers. Thanks for the extra detail but... Probably the reason you can't find any modules out there is this is not something anyone would normally want to do. It would consume quite a lot of memory and provide little advantage (a little bit of speed) compared to building the lists (or copying the files) dynamically as you traverse the real directories. > Directory tree, it's called Maildir. Server is dovecot. > > Create an archive elsewhere. Copything the mails isn't the problem. > Finding them is easy. I just don't know howto represent the tree in the > script. Are you sure you really need to? The oly reason I can see for trying to do that would be if you were trying to build some kind of dynamic query engine that processed a lot of different queries in a single session. If you know before you run the code what you need to extract you are usually better just scanning the folders and processing the data as you find it. That way you only traverse the tree once rather than once to build the tree then search the tree (in memory) and go back again to process the files. If you really need it in memory there are some generic modules for building tree structures in memory - you will need to define the mail objects of course the modules are data neutral, they just allow you to create and navigate the tree, populating it with whatever objects you want. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On Fri, 29 Sep 2017 12:16:11 -0600 Mats Wichmann wrote: > It's not clear what you're really looking for... Sorry. Tried to ask more precisely in my reply to Alan. > File/directory usage is really an OS-specific thing, and most of the > functionality you want seems like it would be in the os module. > Have you looked at functions like os.renames() and os.makedirs()? Sure, but I'd like to know how to represent the directory structure in my script. Finding (os.walk) and copying the mails is not the problem. > Do you need something much more complicated than that? Well, I just like to know how to do that. Of course, I can find the mails with os.walk and move them. Create the necessary directory structure with something like mkdir -p. That's no problem. But when I want to print a list for example, I have to keep the structure, because folders have different meanings depending where they're in the tree. - Chris ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On 29/09/17 18:02, Chris wrote: > I'd like to store a directory tree in a python script. That doesn't make much sense. A directory tree is stored on the hard disk. A python script is the source code to a program you execute What do you mean by "store the directory tree in the script"? Do you mean you want the script to copy an existing directory tree? Or to create a new one? Or to store some kind of representation of the tree in memory when your code runs? Or create a static model of a directory tree in your source code? Or something else? - if so, what? > Background: Maildirs with mails older than five years should be > archived. The folder structure should be kept in the target. Again that's not very clear. Which mail system? Which OS? Some use a single file? Others use a directory tree (so I'm guess you mean one of those) and others use a database. (the representation as folders within a mail client bears little resemblance to how the physical data may be stored on disk) And what are you trying to do? remove old folders from the existing mail system - that could screw up its internal integrity checking (checksums for example)~ - or create an archive elsewhere? > I was very surprised, that there seems no readily usable module > available. (In Perl neither). Once we understand what you actually want to do we might find something suitable. But for now I have no clue what exactly you need. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On 09/29/2017 11:02 AM, Chris wrote: > Hello, > > I'd like to store a directory tree in a python script. > > Background: Maildirs with mails older than five years should be > archived. The folder structure should be kept in the target. > > I was very surprised, that there seems no readily usable module > available. (In Perl neither). > > What's the best way to save them? It's not clear what you're really looking for... File/directory usage is really an OS-specific thing, and most of the functionality you want seems like it would be in the os module. Have you looked at functions like os.renames() and os.makedirs()? Do you need something much more complicated than that? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
