Re: [Tutor] I want to learn how memory works!
On 29/09/17 19:47, Michael C wrote: > Could you point me to a source of information about all the things someone > should know before he starts writing a memory scanner? Not a single source but wikipedia is a good start for anything technical. In particular you need to understand the difference between virtual(VM) and physical memory(PM). Virtual memory is the memory that your program(process in OS speak) sees and physical memory is the RAM installed in your computer. The OS maps virtual memory to physical memory and depending on the OS that mapping can be done in many ways. In some OS (especially mainframes) you can specify in a config file how much VM each process is given at startup, in others you specify how much VM it needs so the OS won't allow it to start up unless there is that much available (this is often used on small machines and embedded systems). In others the VM is always a theoretical space determined by the address size (or built into the kernel). The PM is a combination of the theoretical address space, the installed RAM and the virtual memory page file(s). The OS swaps memory between RAM and page file as necessary. It is quite a complex topic and heavily OS dependent. It looks like you are using Windows and it's too long since I looked at that level of detail (around NT4!) to be confident of a reply, but ~ I'd start with wikipedia for the basic concepts then move to MSDN for the detail for your OS. The key point is that from inside a process you are seeing a virtualized version of memory, rarely, if ever, the actual physical RAM addresses. Only the OS sees that. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
Thank you for your detailed reply! I've tried to explain in my mails to
Alan and Mats what I'm trying to achieve.
On Sat, 30 Sep 2017 11:32:57 +1000
Steven D'Aprano wrote:
> On Fri, Sep 29, 2017 at 07:02:07PM +0200, Chris wrote:
>
> > Background: Maildirs with mails older than five years should be
> > archived. The folder structure should be kept in the target.
>
> Archived to what?
A separate Maildir on tape.
> > I was very surprised, that there seems no readily usable module
> > available. (In Perl neither).
>
> Reusable module to do what *precisely*? If you cannot explain what
> you need, how do you expect somebody to have anticipated your
> requirements and written a module to do it?
Represent the structure in memory.
> > What's the best way to save them?
>
> Depends on what you are doing. But coding the paths in your source
> code is almost certainly not what you want to do. Surely you want to
> read the paths from the maildir itself, as it *actually* exists,
> rather than try to hard-code what you expect it to be in your source
> code?
Well, if I had the structure in memory, I could save additional
information and could print different lists, e.g. sorted by attachment
size, sorted by project. A project can appear in different places
in the tree.
> Have you looked at the contents of a maildir? Its actually an almost
> flat structure. Nested mail folders are not nested on the disk: a
> user's mail folder structure that looks like:
>
> inbox
> sent
> trash
> personal
> ├── family
> └── friends
> work
> ├── critical
> ├── important
> └── low
>
>
> is stored on disk as:
>
> Maildir/
> ├── .sent/
> ├── .trash/
> ├── .personal/
> ├── .personal.family/
> ├── .person.friends/
> ├── .work
> ├── .work.critical
> ├── .work.important
> └── .work.low
Good objection. You can make dovecot use the first layout on disk.
Probably not a gain in disguise.
> So all you really need is to record the path to the top level maildir
> directories (the original, and the place where you are archiving
> them). The subdirectories, you read from the disk as you go.
Ok, I could even do this with the first structure.
> Actually, *you* don't read them at all. Have you looked at the
> mailbox module in the standard library? It supports Maildir. I expect
> that what you would do is something like:
>
> source = Maildir('path/to/source')
> archive = Maildir('path/to/archive')
> for each directory in source:
> for each mail in directory:
> if mail older than five years:
> copy mail to archive
> delete mail from source
I've used os.walk. I'll have a look at the Maildir library.
- Chris
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On Fri, Sep 29, 2017 at 07:02:07PM +0200, Chris wrote:
> Background: Maildirs with mails older than five years should be
> archived. The folder structure should be kept in the target.
Archived to what?
> I was very surprised, that there seems no readily usable module
> available. (In Perl neither).
Reusable module to do what *precisely*? If you cannot explain what you
need, how do you expect somebody to have anticipated your requirements
and written a module to do it?
> What's the best way to save them?
Depends on what you are doing. But coding the paths in your source code
is almost certainly not what you want to do. Surely you want to read the
paths from the maildir itself, as it *actually* exists, rather than try
to hard-code what you expect it to be in your source code?
Have you looked at the contents of a maildir? Its actually an almost
flat structure. Nested mail folders are not nested on the disk: a
user's mail folder structure that looks like:
inbox
sent
trash
personal
├── family
└── friends
work
├── critical
├── important
└── low
is stored on disk as:
Maildir/
├── .sent/
├── .trash/
├── .personal/
├── .personal.family/
├── .person.friends/
├── .work
├── .work.critical
├── .work.important
└── .work.low
(Not shown: the mails themselves, some dovecot specific data files
in the top level, and the special cur/ new/ tmp/ subdirectories found
in each mail directory.)
So all you really need is to record the path to the top level maildir
directories (the original, and the place where you are archiving them).
The subdirectories, you read from the disk as you go.
Actually, *you* don't read them at all. Have you looked at the mailbox
module in the standard library? It supports Maildir. I expect that what
you would do is something like:
source = Maildir('path/to/source')
archive = Maildir('path/to/archive')
for each directory in source:
for each mail in directory:
if mail older than five years:
copy mail to archive
delete mail from source
and let the mailbox module do the hard work. (Especially the part about
copying the mail, since the module will ensure that the flags and dates
are copied correctly.)
--
Steve
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] OT: how to best follow the posting style on this list
Hi, I often read messages to this list from my Android from the mobile Hotmail page. On other occasions I would read from my browser, from a Windows or a Linux desktop. Howver, Hotmail pretty much sucks when it comes to obeying the posting style [1]. It's even worse than Yahoo mail, which I've also tried (until they got hacked). Is there a recommended way to do this? Maybe gmail? I prefer to use more or less the same interface across platforms and devices, but this is not crucial. I've also been looking for pan-like apps (not necessarily free) for my Phone, but I couldn't find one that worked. Perhaps because I used a free news server (my ISP doesn't have one). Thanks in advance! Best wishes, Albert-Jan [1] https://en.wikipedia.org/wiki/Posting_style ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On 29/09/17 20:34, Chris wrote: > I want to store some kind of representation of the tree in memory when > the code runs. Then I could not only move the mails but also create > lists, e.g. a table with mail headers. Thanks for the extra detail but... Probably the reason you can't find any modules out there is this is not something anyone would normally want to do. It would consume quite a lot of memory and provide little advantage (a little bit of speed) compared to building the lists (or copying the files) dynamically as you traverse the real directories. > Directory tree, it's called Maildir. Server is dovecot. > > Create an archive elsewhere. Copything the mails isn't the problem. > Finding them is easy. I just don't know howto represent the tree in the > script. Are you sure you really need to? The oly reason I can see for trying to do that would be if you were trying to build some kind of dynamic query engine that processed a lot of different queries in a single session. If you know before you run the code what you need to extract you are usually better just scanning the folders and processing the data as you find it. That way you only traverse the tree once rather than once to build the tree then search the tree (in memory) and go back again to process the files. If you really need it in memory there are some generic modules for building tree structures in memory - you will need to define the mail objects of course the modules are data neutral, they just allow you to create and navigate the tree, populating it with whatever objects you want. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] problem with python3.5 headfirst python 2nd ed chpt 10 test drive example decorator issues
Hi
I don't see any session initializer , please try with that
session = web.session.Session(app,store,initializer={'login':
0,'privilege': 0,'username':'Guest','logged_in':False})
Regards
Rajesh
On Thu, Sep 28, 2017 at 3:35 PM, peter wrote:
> I am on chapter 10 of headfirst python second edition. got most of the
> prior codes to work but am stuck on this one. I will add the
> simple_webapp.py which is a decorator enabled and checker.py which is the
> decorator. when I go into 127.0.0.1:5000 and enter I get the correct
> response. 127.0.0.1:5000/page1 gets me 'you are not logged in' also
> correct.
>
> '127.0.0.1:5000/login' returns 'you are now logged in' which is correct
> but '127.0.0.1:5000/page1' after that should return 'this is page 1' but
> instead returns 'you are not logged in'.
>
> When I login to page 1 the 'session['logged_in'] = True' should still be
> true but apparently it has not been passed to 'check_logged_in'.
>
> I am stumped and do not know how to proceed to figure this out.
>
> Here are the codes
>
> Also they are attached to this email. I am hoping someone can show me the
> errors of my ways. hopefully it is something simple but I have gone over it
> a lot and think the code is correct.
>
> Thank you for your attention and help.
>
> Peter Risley
>
> checker.py
>
> from flask import session
> from functools import wraps
>
> def check_logged_in(func):
> @wraps(func)
> def wrapper(*args, **kwargs):
> if 'logged _in' in session:
> return func(*args, **kwargs)
> return 'You are not logged in.'
> return wrapper
>
> simple_webapp.py
>
> from flask import Flask, session
> from checker import check_logged_in
>
> """this 'simple_webapp.py' , which pulls all of chp 10 code together. When
> you need to restrict access to specific URLs, base your strategy on this
> webapp's mechanism.
> This uses checker.py check_logged_in and which is a decorator function to
> do the work."""
>
>
>
> app = Flask(__name__)
>
> @app.route('/')
> def hello() -> str:
> return 'Hello from the simple webapp.'
>
>
> @app.route('/page1')
> @check_logged_in
> def page1():
> return 'this is page 1.'
>
> @app.route('/page2')
> @check_logged_in
> def page2():
> return 'this is page 2.'
>
> @app.route('/page3')
> @check_logged_in
> def page3():
> return 'this is page 3.'
>
>
> @app.route('/login')
> def do_login() -> str:
> session['logged_in'] = True
> return 'you are now logged in.'
>
>
> @app.route('/logout')
> def do_logout() -> str:
> session.pop('logged_in')
> return 'you are now logged out.'
>
> app.secret_key = 'yes'
>
> if __name__ == '__main__':
> app.run(debug=True)
>
>
>
>
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On Fri, 29 Sep 2017 12:16:11 -0600 Mats Wichmann wrote: > It's not clear what you're really looking for... Sorry. Tried to ask more precisely in my reply to Alan. > File/directory usage is really an OS-specific thing, and most of the > functionality you want seems like it would be in the os module. > Have you looked at functions like os.renames() and os.makedirs()? Sure, but I'd like to know how to represent the directory structure in my script. Finding (os.walk) and copying the mails is not the problem. > Do you need something much more complicated than that? Well, I just like to know how to do that. Of course, I can find the mails with os.walk and move them. Create the necessary directory structure with something like mkdir -p. That's no problem. But when I want to print a list for example, I have to keep the structure, because folders have different meanings depending where they're in the tree. - Chris ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] I want to learn how memory works!
Hi all, after 1 week of on and off hacking, I realized I simply don't know
enough about how memory works fundamentally!
Could you point me to a source of information about all the things someone
should know before he starts writing a memory scanner? Attached is my
current code, which doesn't work and can't figure out why. That's where I
am at.
> code starts.
import ctypes
User32 = ctypes.WinDLL('User32', use_last_error=True)
Kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
PID = 5924
PROCESS_QUERY_INFORMATION = 0x0400
PROCESS_VM_READ = 0x0010
Process = Kernel32.OpenProcess(PROCESS_QUERY_INFORMATION|PROCESS_VM_READ,
False, PID)
ReadProcessMemory = Kernel32.ReadProcessMemory
buffer = ctypes.create_string_buffer(4)
bufferSize = (ctypes.sizeof(buffer))
# I think instead of using 10, I should use the size of the total
# memory used. but I don't know how to find this value.
for n in range(10):
if ReadProcessMemory(Process, n, buffer, bufferSize, None):
print('buffer: ',buffer)
else:
print('something is wrong!')
print('Done.')
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python programming for the absolute beginner
On Fri, Sep 29, 2017 at 7:20 PM, Alan Gauld via Tutor wrote: > If you want to follow the book use the version the book > uses - probably 2.6 or something close? > I think the book uses either Python 3.0 or 3.1. It's been a while since I read it but it doesn't use Python 2. The 2nd edition written in 2010 used Python 3 but the 1st edition used Python 2. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] problem with python3.5 headfirst python 2nd ed chpt 10 test drive example decorator issues
On 28/09/17 23:35, peter wrote:
> I am on chapter 10 of headfirst python second edition. got most of the
> prior codes to work but am stuck on this one.
I don;t know the book and only vaguely know Flask, but I'd start by
adding some debug print statements to the functions.
Something like
def
s = str(session[logged_in])
return s + ...
To see what the actual value is in each call.
> def check_logged_in(func):
> @wraps(func)
> def wrapper(*args, **kwargs):
> if 'logged _in' in session:
> return func(*args, **kwargs)
Maybe try returning session.keys() in this func?
> return 'You are not logged in.'
> return wrapper
>
> @app.route('/')
> def hello() -> str:
> return 'Hello from the simple webapp.'
>
>
> @app.route('/page1')
> @check_logged_in
> def page1():
> return 'this is page 1.'
>
> @app.route('/page2')
> @check_logged_in
> def page2():
> return 'this is page 2.'
>
> @app.route('/page3')
> @check_logged_in
> def page3():
> return 'this is page 3.'
>
>
> @app.route('/login')
> def do_login() -> str:
> session['logged_in'] = True
> return 'you are now logged in.'
>
>
> @app.route('/logout')
> def do_logout() -> str:
> session.pop('logged_in')
> return 'you are now logged out.'
>
> app.secret_key = 'yes'
>
> if __name__ == '__main__':
> app.run(debug=True)
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On 29/09/17 18:02, Chris wrote: > I'd like to store a directory tree in a python script. That doesn't make much sense. A directory tree is stored on the hard disk. A python script is the source code to a program you execute What do you mean by "store the directory tree in the script"? Do you mean you want the script to copy an existing directory tree? Or to create a new one? Or to store some kind of representation of the tree in memory when your code runs? Or create a static model of a directory tree in your source code? Or something else? - if so, what? > Background: Maildirs with mails older than five years should be > archived. The folder structure should be kept in the target. Again that's not very clear. Which mail system? Which OS? Some use a single file? Others use a directory tree (so I'm guess you mean one of those) and others use a database. (the representation as folders within a mail client bears little resemblance to how the physical data may be stored on disk) And what are you trying to do? remove old folders from the existing mail system - that could screw up its internal integrity checking (checksums for example)~ - or create an archive elsewhere? > I was very surprised, that there seems no readily usable module > available. (In Perl neither). Once we understand what you actually want to do we might find something suitable. But for now I have no clue what exactly you need. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python programming for the absolute beginner
Hi Peter My company primarily uses 2.7, but as I understand it 2.x flavors are compatible with each other. 2.x scripts are generally not compatible with 3.x versions and vice versa So, keep that in mind. Regards, -John -Original Message- From: Tutor [mailto:tutor-bounces+john.b.larocca=intel@python.org] On Behalf Of Peter Collidge Sent: Friday, September 29, 2017 12:51 AM To: tutor@python.org Subject: [Tutor] Python programming for the absolute beginner I have borrowed the above book from my local library but I believe it was written in 2010 and as a result I am having difficulty in deciding which version of Python to download. Can anyone help? Thanks Peter Collidge ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Directory Structure
On 09/29/2017 11:02 AM, Chris wrote: > Hello, > > I'd like to store a directory tree in a python script. > > Background: Maildirs with mails older than five years should be > archived. The folder structure should be kept in the target. > > I was very surprised, that there seems no readily usable module > available. (In Perl neither). > > What's the best way to save them? It's not clear what you're really looking for... File/directory usage is really an OS-specific thing, and most of the functionality you want seems like it would be in the os module. Have you looked at functions like os.renames() and os.makedirs()? Do you need something much more complicated than that? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python programming for the absolute beginner
On 29/09/17 08:51, Peter Collidge wrote: > I have borrowed the above book from my local library but I believe it was > written in 2010 and as a result I am having difficulty in deciding which > version of Python to download. > Can anyone help? If you want to follow the book use the version the book uses - probably 2.6 or something close? When you finish the book move to Python 3.6 and go through the official tutorial that accompanbies it, that should bring you up to date with the latest version. There won't be a huge difference but there are significant changes in moving from 2 to 3, so use the version you are studying first then upgrade. All the older versions are available on the download page of python.org. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] problem with python3.5 headfirst python 2nd ed chpt 10 test drive example decorator issues
peter wrote: > I am on chapter 10 of headfirst python second edition. got most of the > prior codes to work but am stuck on this one. I will add the > simple_webapp.py which is a decorator enabled and checker.py which is > the decorator. when I go into 127.0.0.1:5000 and enter I get the correct > response. 127.0.0.1:5000/page1 gets me 'you are not logged in' also > correct. > > '127.0.0.1:5000/login' returns 'you are now logged in' which is correct > but '127.0.0.1:5000/page1' after that should return 'this is page 1' but > instead returns 'you are not logged in'. > > When I login to page 1 the 'session['logged_in'] = True' should still be > true but apparently it has not been passed to 'check_logged_in'. > > I am stumped and do not know how to proceed to figure this out. > > Here are the codes > > Also they are attached to this email. I am hoping someone can show me > the errors of my ways. hopefully it is something simple but I have gone > over it a lot and think the code is correct. The best approach is usually to do something completely different, and than look at the code with fresh eyes. Or to ask someone else, of course... > Thank you for your attention and help. > session['logged_in'] = True > if 'logged _in' in session: It looks like an extra space slipped in here. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] problem with python3.5 headfirst python 2nd ed chpt 10 test drive example decorator issues
I am on chapter 10 of headfirst python second edition. got most of the
prior codes to work but am stuck on this one. I will add the
simple_webapp.py which is a decorator enabled and checker.py which is
the decorator. when I go into 127.0.0.1:5000 and enter I get the correct
response. 127.0.0.1:5000/page1 gets me 'you are not logged in' also correct.
'127.0.0.1:5000/login' returns 'you are now logged in' which is correct
but '127.0.0.1:5000/page1' after that should return 'this is page 1' but
instead returns 'you are not logged in'.
When I login to page 1 the 'session['logged_in'] = True' should still be
true but apparently it has not been passed to 'check_logged_in'.
I am stumped and do not know how to proceed to figure this out.
Here are the codes
Also they are attached to this email. I am hoping someone can show me
the errors of my ways. hopefully it is something simple but I have gone
over it a lot and think the code is correct.
Thank you for your attention and help.
Peter Risley
checker.py
from flask import session
from functools import wraps
def check_logged_in(func):
@wraps(func)
def wrapper(*args, **kwargs):
if 'logged _in' in session:
return func(*args, **kwargs)
return 'You are not logged in.'
return wrapper
simple_webapp.py
from flask import Flask, session
from checker import check_logged_in
"""this 'simple_webapp.py' , which pulls all of chp 10 code together. When
you need to restrict access to specific URLs, base your strategy on this
webapp's mechanism.
This uses checker.py check_logged_in and which is a decorator function
to do the work."""
app = Flask(__name__)
@app.route('/')
def hello() -> str:
return 'Hello from the simple webapp.'
@app.route('/page1')
@check_logged_in
def page1():
return 'this is page 1.'
@app.route('/page2')
@check_logged_in
def page2():
return 'this is page 2.'
@app.route('/page3')
@check_logged_in
def page3():
return 'this is page 3.'
@app.route('/login')
def do_login() -> str:
session['logged_in'] = True
return 'you are now logged in.'
@app.route('/logout')
def do_logout() -> str:
session.pop('logged_in')
return 'you are now logged out.'
app.secret_key = 'yes'
if __name__ == '__main__':
app.run(debug=True)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] Python programming for the absolute beginner
I have borrowed the above book from my local library but I believe it was written in 2010 and as a result I am having difficulty in deciding which version of Python to download. Can anyone help? Thanks Peter Collidge ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] logging to cmd.exe
Dear Mats, Peter and Eryk, THANK YOU for your replies. What a wealth of information! Have a great weekend! Albert-Jan ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
