Re: [Tutor] using while loop for read process memory

[Michael C]

Sorry Alan, Steve, everyone

Can you take a look of this please?



Here is my question about the memory:

So I have a base address of a chunk of memory from it's size, from
VirtualQueryEx
(if you dont use windows, it's ok, it's not about how u get these values,
because I think
the base concept is the same)

start = mbi.BaseAddress
finish = mbi.RegionSize

So at this time, I use while and this is how it looks like

while index < finish:
   # access the memory here:
   while memory function( index)
   # then index += 1, for the inner loop

## this line complete the outer while loop
index += mbi.RegionSize


so Why did I put down index += 1  ?

That's because what I think about the memory looks like this
(short)(int)(double)(int)(int)(int)(double)  and so on,

since I can't predict which address is the beginning of a double, the only
way
to deal with that is to use increment by 1.

Now, from what I have been reading, it seems there is a better way to do it,
for instance, a for loop.

for(start,finish, 8)

why 8? because double begins at exact 0 or multiple of 8 bytes, right?

On Thu, Oct 12, 2017 at 6:54 PM, Michael C 
wrote:

> Here is my question about the memory:
>
> So I have a base address of a chunk of memory from it's size, from
> VirtualQueryEx
> (if you dont use windows, it's ok, it's not about how u get these values,
> because I think
> the base concept is the same)
>
> start = mbi.BaseAddress
> finish = mbi.RegionSize
>
> So at this time, I use while and this is how it looks like
>
> while index < finish:
># access the memory here:
>while memory function( index)
># then index += 1, for the inner loop
>
> ## this line complete the outer while loop
> index += mbi.RegionSize
>
>
> so Why did I put down index += 1  ?
>
> That's because what I think about the memory looks like this
> (short)(int)(double)(int)(int)(int)(double)  and so on,
>
> since I can't predict which address is the beginning of a double, the only
> way
> to deal with that is to use increment by 1.
>
> Now, from what I have been reading, it seems there is a better way to do
> it,
> for instance, a for loop.
>
> for(start,finish, 8)
>
> why 8? because double begins at exact 0 or multiple of 8 bytes, right?
>
>
>
> On Sun, Oct 8, 2017 at 4:46 PM, Alan Gauld via Tutor 
> wrote:
>
>> On 08/10/17 20:18, Michael C wrote:
>> > This is the red part
>> >   index = current_address
>> > end = current_address + mbi.RegionSize
>> >
>> > while index < end:
>> > if ReadProcessMemory(Process, index, ctypes.byref(buffer), \
>> >  ctypes.sizeof(buffer),
>> > ctypes.byref(nread)):
>> > ## value comparison to be implemented.
>> > pass
>> > else:
>> > raise ctypes.WinError(ctypes.get_last_error())
>> >
>> > index += 1
>>
>> I haven't been following this closely so may be way off here,
>> but does this mean you are incrementing the memory address
>> by 1? If so you are only increasing the pointer by 1 byte
>> but you are, presumably, reading multiple bytes at a time
>> (the size of the buffer presumably).
>>
>> Do you perhaps need to treat the buffer as a byte array
>> and use something like the struct module to decode it?
>> (assuming you know what you are reading...?)
>>
>> But I may be way off, I'm just going on a cursory look.
>>
>> --
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>> http://www.amazon.com/author/alan_gauld
>> Follow my photo-blog on Flickr at:
>> http://www.flickr.com/photos/alangauldphotos
>>
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Alan Gauld via Tutor]

On 13/10/17 02:58, Michael C wrote:

>         end = current_address + mbi.RegionSize - 7
> 
> then it doesn't complain anymore. I think it's because I ran this in a
> while loop with start += 1
> so in the last 7 bytes, I'd be reading past the end of this memory chunk.
> 
> Is this right?

Yes, almost certainly. That's what both Steve and I were
alluding to in our earlier responses, you were incrementing
by 1 byte but reading more than one byte so there was a
high probability of you reading past the end.

But subtracting 7 is only the correct answer if you
are always reading 8 byte blocks, if you are reading
different length blocks (for int/short/char etc) then
you might need to do some kind of dynamic check based
on sizeof(chunk)...

if index+sizeof(chunk) > end
   data = read(chunk)
else break

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Michael C]

in fact, when I am using this:

end = start + mbi.RegionSize

I was getting error from the ReadProcessMemory function, and I couldn't
figure it out why.
Until I did this:
end = current_address + mbi.RegionSize - 7

then it doesn't complain anymore. I think it's because I ran this in a
while loop with start += 1
so in the last 7 bytes, I'd be reading past the end of this memory chunk.

Is this right?

On Thu, Oct 12, 2017 at 6:54 PM, Michael C 
wrote:

> Here is my question about the memory:
>
> So I have a base address of a chunk of memory from it's size, from
> VirtualQueryEx
> (if you dont use windows, it's ok, it's not about how u get these values,
> because I think
> the base concept is the same)
>
> start = mbi.BaseAddress
> finish = mbi.RegionSize
>
> So at this time, I use while and this is how it looks like
>
> while index < finish:
># access the memory here:
>while memory function( index)
># then index += 1, for the inner loop
>
> ## this line complete the outer while loop
> index += mbi.RegionSize
>
>
> so Why did I put down index += 1  ?
>
> That's because what I think about the memory looks like this
> (short)(int)(double)(int)(int)(int)(double)  and so on,
>
> since I can't predict which address is the beginning of a double, the only
> way
> to deal with that is to use increment by 1.
>
> Now, from what I have been reading, it seems there is a better way to do
> it,
> for instance, a for loop.
>
> for(start,finish, 8)
>
> why 8? because double begins at exact 0 or multiple of 8 bytes, right?
>
>
>
> On Sun, Oct 8, 2017 at 4:46 PM, Alan Gauld via Tutor 
> wrote:
>
>> On 08/10/17 20:18, Michael C wrote:
>> > This is the red part
>> >   index = current_address
>> > end = current_address + mbi.RegionSize
>> >
>> > while index < end:
>> > if ReadProcessMemory(Process, index, ctypes.byref(buffer), \
>> >  ctypes.sizeof(buffer),
>> > ctypes.byref(nread)):
>> > ## value comparison to be implemented.
>> > pass
>> > else:
>> > raise ctypes.WinError(ctypes.get_last_error())
>> >
>> > index += 1
>>
>> I haven't been following this closely so may be way off here,
>> but does this mean you are incrementing the memory address
>> by 1? If so you are only increasing the pointer by 1 byte
>> but you are, presumably, reading multiple bytes at a time
>> (the size of the buffer presumably).
>>
>> Do you perhaps need to treat the buffer as a byte array
>> and use something like the struct module to decode it?
>> (assuming you know what you are reading...?)
>>
>> But I may be way off, I'm just going on a cursory look.
>>
>> --
>> Alan G
>> Author of the Learn to Program web site
>> http://www.alan-g.me.uk/
>> http://www.amazon.com/author/alan_gauld
>> Follow my photo-blog on Flickr at:
>> http://www.flickr.com/photos/alangauldphotos
>>
>>
>> ___
>> Tutor maillist  -  Tutor@python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>>
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Michael C]

Here is my question about the memory:

So I have a base address of a chunk of memory from it's size, from
VirtualQueryEx
(if you dont use windows, it's ok, it's not about how u get these values,
because I think
the base concept is the same)

start = mbi.BaseAddress
finish = mbi.RegionSize

So at this time, I use while and this is how it looks like

while index < finish:
   # access the memory here:
   while memory function( index)
   # then index += 1, for the inner loop

## this line complete the outer while loop
index += mbi.RegionSize


so Why did I put down index += 1  ?

That's because what I think about the memory looks like this
(short)(int)(double)(int)(int)(int)(double)  and so on,

since I can't predict which address is the beginning of a double, the only
way
to deal with that is to use increment by 1.

Now, from what I have been reading, it seems there is a better way to do it,
for instance, a for loop.

for(start,finish, 8)

why 8? because double begins at exact 0 or multiple of 8 bytes, right?



On Sun, Oct 8, 2017 at 4:46 PM, Alan Gauld via Tutor 
wrote:

> On 08/10/17 20:18, Michael C wrote:
> > This is the red part
> >   index = current_address
> > end = current_address + mbi.RegionSize
> >
> > while index < end:
> > if ReadProcessMemory(Process, index, ctypes.byref(buffer), \
> >  ctypes.sizeof(buffer),
> > ctypes.byref(nread)):
> > ## value comparison to be implemented.
> > pass
> > else:
> > raise ctypes.WinError(ctypes.get_last_error())
> >
> > index += 1
>
> I haven't been following this closely so may be way off here,
> but does this mean you are incrementing the memory address
> by 1? If so you are only increasing the pointer by 1 byte
> but you are, presumably, reading multiple bytes at a time
> (the size of the buffer presumably).
>
> Do you perhaps need to treat the buffer as a byte array
> and use something like the struct module to decode it?
> (assuming you know what you are reading...?)
>
> But I may be way off, I'm just going on a cursory look.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Michael C]

thank for replying, but I am toast, so I'll reply tomorrow,
thanks!

On Sun, Oct 8, 2017 at 4:46 PM, Alan Gauld via Tutor 
wrote:

> On 08/10/17 20:18, Michael C wrote:
> > This is the red part
> >   index = current_address
> > end = current_address + mbi.RegionSize
> >
> > while index < end:
> > if ReadProcessMemory(Process, index, ctypes.byref(buffer), \
> >  ctypes.sizeof(buffer),
> > ctypes.byref(nread)):
> > ## value comparison to be implemented.
> > pass
> > else:
> > raise ctypes.WinError(ctypes.get_last_error())
> >
> > index += 1
>
> I haven't been following this closely so may be way off here,
> but does this mean you are incrementing the memory address
> by 1? If so you are only increasing the pointer by 1 byte
> but you are, presumably, reading multiple bytes at a time
> (the size of the buffer presumably).
>
> Do you perhaps need to treat the buffer as a byte array
> and use something like the struct module to decode it?
> (assuming you know what you are reading...?)
>
> But I may be way off, I'm just going on a cursory look.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Steven D'Aprano]

I have no idea about ctypes or Windows, but it seems to me that you are 
creating a rod for your own back by using a while loop here. Why use a 
primitive, low-level looping construct when Python gives you much better 
tools?

My *guess* is that somewhere you are miscalcuating when to stop, and 
trying to read beyond the valid region.

Your code uses nested while loops. But since you already know the 
beginning and end of the loop, that is much better written as 
for-loops (and will be faster too).

It's not clear to me how much memory you expect to be reading at a time. 
I *guess* that you read blocks of memory the size of mbi at a time. If 
your memory is:

abcdefghijklmnopqrstuvwxyz...

and mbi is (lets say) *six* chars long, then you want to read:

abcdef
ghijkl
mnopqr
stuvwx
yz...

Then, within each mbi-sized block, if each buffer is (say) *two* chars 
long, you want to read:

ab
cd
ef

Is that right? If not, you will have to adjust the following to better 
suit your intention.


# Untested, as I don't run Windows.
blocksize = ctypes.sizeof(mbi)
buffer_blocksize = ctypes.sizeof(buffer)
for current_address in range(
sysinfo.lpMinimumApplicationAddress,
sysinfo.lpMaximumApplicationAddress,
blocksize
):
# process the current address here
Kernel32.VirtualQueryEx(
Process,
current_address, 
ctypes.byref(mbi),
blocksize
)
# Note that there's no need for a backslash \ to continue
# lines inside open brackets and parentheses; by 
# convention such lines are indented extra to allow them
# to stand out. Feel free to make it a bit more compact if
# you prefer it that way.
if mbi.Protect == PAGE_READWRITE and mbi.State == MEM_COMMIT:
print('This region can be scanned!')  # which region?
for index in range(
current_address, 
current_address + mbi.RegionSize,
buffer_blocksize
):
if ReadProcessMemory(
Process, 
index, 
ctypes.byref(buffer),
buffer_blocksize,
ctypes.byref(nread)
):
## FIXME implement value comparison
pass
else:
raise ctypes.WinError(ctypes.get_last_error())



Hope this helps.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Alan Gauld via Tutor]

On 08/10/17 20:18, Michael C wrote:
> This is the red part 
>   index = current_address
>         end = current_address + mbi.RegionSize
> 
>         while index < end:
>             if ReadProcessMemory(Process, index, ctypes.byref(buffer), \
>                                  ctypes.sizeof(buffer),
> ctypes.byref(nread)):
>                 ## value comparison to be implemented.
>                 pass   
>             else:
>                     raise ctypes.WinError(ctypes.get_last_error())
> 
>             index += 1

I haven't been following this closely so may be way off here,
but does this mean you are incrementing the memory address
by 1? If so you are only increasing the pointer by 1 byte
but you are, presumably, reading multiple bytes at a time
(the size of the buffer presumably).

Do you perhaps need to treat the buffer as a byte array
and use something like the struct module to decode it?
(assuming you know what you are reading...?)

But I may be way off, I'm just going on a cursory look.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Michael C]

I'll explain better when I get on a pc.

On Oct 8, 2017 12:18 PM, "Michael C"  wrote:

> This is the red part
>   index = current_address
> end = current_address + mbi.RegionSize
>
> while index < end:
> if ReadProcessMemory(Process, index, ctypes.byref(buffer), \
>  ctypes.sizeof(buffer),
> ctypes.byref(nread)):
> ## value comparison to be implemented.
> pass
> else:
> raise ctypes.WinError(ctypes.get_last_error())
>
> index += 1
>
> On Oct 8, 2017 12:16 PM, "Mats Wichmann"  wrote:
>
>> On 10/08/2017 11:20 AM, Michael C wrote:
>> > Hi all:
>>
>> > Now, I know the problem is not with VirtualQueryEx, because if I
>> comment out
>> > the red part and just run VirtualQueryEx, it would actually skim through
>> > all regions
>> > without a single error.
>> >
>> > The red part is the problem.
>>
>> what red part?  colors don't come through mailers that use text-based
>> settings.  This is an example of what your mail looks like to many of us:
>>
>> https://mail-archive.com/tutor@python.org/msg77570.html
>>
>> please explain in words.
>>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Michael C]

This is the red part
  index = current_address
end = current_address + mbi.RegionSize

while index < end:
if ReadProcessMemory(Process, index, ctypes.byref(buffer), \
 ctypes.sizeof(buffer),
ctypes.byref(nread)):
## value comparison to be implemented.
pass
else:
raise ctypes.WinError(ctypes.get_last_error())

index += 1

On Oct 8, 2017 12:16 PM, "Mats Wichmann"  wrote:

> On 10/08/2017 11:20 AM, Michael C wrote:
> > Hi all:
>
> > Now, I know the problem is not with VirtualQueryEx, because if I comment
> out
> > the red part and just run VirtualQueryEx, it would actually skim through
> > all regions
> > without a single error.
> >
> > The red part is the problem.
>
> what red part?  colors don't come through mailers that use text-based
> settings.  This is an example of what your mail looks like to many of us:
>
> https://mail-archive.com/tutor@python.org/msg77570.html
>
> please explain in words.
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] using while loop for read process memory

[Mats Wichmann]

On 10/08/2017 11:20 AM, Michael C wrote:
> Hi all:

> Now, I know the problem is not with VirtualQueryEx, because if I comment out
> the red part and just run VirtualQueryEx, it would actually skim through
> all regions
> without a single error.
> 
> The red part is the problem. 

what red part?  colors don't come through mailers that use text-based
settings.  This is an example of what your mail looks like to many of us:

https://mail-archive.com/tutor@python.org/msg77570.html

please explain in words.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] using while loop for read process memory

[Michael C]

Hi all:

I have the following code, and somehow I must have fed the read process
Memory incorrectly. what the code does is to check a region of memory to
see
whether or not it can be scanned.

mbi.Protect == PAGE_READWRITE and mbi.State == MEM_COMMIT

If this is true,then it proceeds to scan the memory fro current_address to
current_address + mbi.RegionSize.

However, a strange thing happens: The loop runs twice successfully, and
then it
pops:

raise ctypes.WinError(ctypes.get_last_error())
OSError: [WinError 299] Only part of a ReadProcessMemory or
WriteProcessMemory request was completed.

Now, I know the problem is not with VirtualQueryEx, because if I comment out
the red part and just run VirtualQueryEx, it would actually skim through
all regions
without a single error.

The red part is the problem. I have tried to modify the loop.

Somehow, if I use this:

index = current_address
end = current_address + mbi.RegionSize - 7

Where the end is less by 7, the loop would not pop any error and it would
finish
the loop

What did I do wrong?

thanks!




>code starts



current_address = sysinfo.lpMinimumApplicationAddress
end_address = sysinfo.lpMaximumApplicationAddress

while current_address < end_address:
Kernel32.VirtualQueryEx(Process, \
current_address, ctypes.byref(mbi),ctypes.sizeof(mbi))

if mbi.Protect == PAGE_READWRITE and mbi.State == MEM_COMMIT :
print('This region can be scanned!')
index = current_address
end = current_address + mbi.RegionSize

while index < end:
if ReadProcessMemory(Process, index, ctypes.byref(buffer), \
 ctypes.sizeof(buffer),
ctypes.byref(nread)):
## value comparison to be implemented.
pass
else:
raise ctypes.WinError(ctypes.get_last_error())

index += 1

current_address += mbi.RegionSize
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor