Re: [Tutor] Example for read and readlines() (Asad)
Asad,
Like many projects, there may be many ways to do things BUT some rules do
apply.
You can only read an open file ONCE unless you seek back to the beginning or
reopen it.
string = f3.read()
string1 = f3.readlines()
The first line reads the entire file into a single buffer.
The second program line won't work as intended. The first consumed the
entire file.
Much of the rest is not organized well enough for me to understand what you
want to do. I find it important for people to try some simple things like
examining the values step by step. Had you typed
print (string)
print (string1)
on a small sample file, you might have fixed that before continuing. Then
each step along the way you could examine and verify it made sense up to
that point.
Try writing the outline of the logic of your program first in English or
your native language as an algorithm. Then see what tools are needed. Look
at a sample of the log you are evaluating and see what it takes to locate
the lines you want and then to break out the parts you want to keep for
further use.
What I see looks like this:
If you find one instance of the string "ERR1"
Then
You want to find ALL (nonoverlapping) regions consisting of an upper-case
letter followed by two lower-case letters and a space and either a space or
digits 1 to 3 and digits 0-9 and a space and ...
Fairly complex pattern.
But you are searching the contents of the ENTIRE file for this and since you
seem to have wanted to replace all newlines by spaces and your pattern
includes spaces, this would match something that wrapped around from line to
line. Is this what you wanted?
You then switch gears to using the readlines version and I decided to get
back to my regularly scheduled life. As noted, that probably is an empty
string or worse. Good luck.
-Original Message-
From: Tutor On Behalf Of
Asad
Sent: Sunday, November 11, 2018 8:54 PM
To: tutor@python.org
Subject: Re: [Tutor] Example for read and readlines() (Asad)
Hi All ,
Thanks for the reply . I am building a framework for the two error
conditions, therefore I need to read and readlines because in one only regex
is required and in other regex+ n-1 line is required to process :
#Here we are opening the file and substituting space " " for each \n
encountered
f3 = open (r"D:\QI\log.log", 'r')
string = f3.read()
string1 = f3.readlines()
regex = re.compile ( "\n" )
st = regex.sub ( " ", string )
if re.search('ERR1',st):
y=re.findall("[A-Z][a-z][a-z] [ 123][0-9]
[012][0-9]:[0-5][0-9]:[0-5][0-9] [0-9][0-9][0-9][0-9]",st)
print y
patchnumber = re.compile(r'(\d+)\/(\d+)')==> doesnot
work it only works if I use #string = f3.read() for j in
range(len(string1)):
if re.search ( r'ERR2', string1[j] ):
print "Error line \n", string1[j - 1]
mo = patchnumber.search (string1[j-1])
a = mo.group()
print a
print os.getcwd()
break
Please advice how to proceed.
Thanks,
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Example for read and readlines() (Asad)
On 12Nov2018 07:24, Asad wrote:
Thanks for the reply . I am building a framework for the two
error
conditions, therefore I need to read and readlines because in one only
regex is required and in other regex+ n-1 line is required to process :
#Here we are opening the file and substituting space " " for each \n
encountered
f3 = open (r"D:\QI\log.log", 'r')
string = f3.read()
string1 = f3.readlines()
My first remark is that both these lines read _and_ _consume_ the file
content. So "string" gets the entire file content, and "string1" gets an
empty array of lines, because the file is already at the end, where
there is no more data.
It is also better to use this idiom to read and then close a file:
with open(r"D:\QI\log.log", 'r') as f3:
string = f3.read()
This reliably closes f3 once the "with" suite completes, even if there's
some kind of exception.
You need 2 copies of the file data. You can do this 2 ways. The first
way is to read the file twice:
with open(r"D:\QI\log.log", 'r') as f3:
string = f3.read()
with open(r"D:\QI\log.log", 'r') as f3:
string1 = f3.readlines()
The efficient way is to read the file once, then make string from
string1, or string1 from string. For example:
with open(r"D:\QI\log.log", 'r') as f3:
string1 = f3.readlines()
string = ''.join(string1)
regex = re.compile ( "\n" )
st = regex.sub ( " ", string )
Using a regular expression to replace a fixed string such as "\n" is
overkill. Consider:
st = string.replace("\n", " ")
Python strings have a bunch of handy methods for common simple things.
Have a read of the docs for further detail.
if re.search('ERR1',st):
y=re.findall("[A-Z][a-z][a-z] [ 123][0-9]
[012][0-9]:[0-5][0-9]:[0-5][0-9] [0-9][0-9][0-9][0-9]",st)
print y
On the other hand, a regexp is a good tool for something like the above.
patchnumber = re.compile(r'(\d+)\/(\d+)')==> doesnot
work it only works if I use #string = f3.read()
This may be because "string" is a single string (the whole file text as
one string). "string1" is a _list_ of individual strings, one for each
line. Personally, i would call this "strings" or "lines" or some other
plural word; your code will be easier to read, and easier to debug.
Conversely, a misleading name makes debugging harder because you expect
the variable to contain what its name suggests, and if it doesn't this
will impede you in finding problems, because you will be thinking the
whrong thing about what your program is doing.
for j in range(len(string1)):
if re.search ( r'ERR2', string1[j] ):
print "Error line \n", string1[j - 1]
mo = patchnumber.search (string1[j-1])
a = mo.group()
print a
print os.getcwd()
break
Please advice how to proceed.
mo.group() returns the whole match. The above seems to look for the
string 'ERR2' in a line, and look for a patch number in the previous
line. Is that what is it supposed to do?
If the above isn't working, it would help to see the failing output and
a description of what good output is meant to look like.
Finally, please consider turning off "digest mode" in your list
subscription. It will make things easier for everyone.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Example for read and readlines() (Asad)
Hi All ,
Thanks for the reply . I am building a framework for the two error
conditions, therefore I need to read and readlines because in one only
regex is required and in other regex+ n-1 line is required to process :
#Here we are opening the file and substituting space " " for each \n
encountered
f3 = open (r"D:\QI\log.log", 'r')
string = f3.read()
string1 = f3.readlines()
regex = re.compile ( "\n" )
st = regex.sub ( " ", string )
if re.search('ERR1',st):
y=re.findall("[A-Z][a-z][a-z] [ 123][0-9]
[012][0-9]:[0-5][0-9]:[0-5][0-9] [0-9][0-9][0-9][0-9]",st)
print y
patchnumber = re.compile(r'(\d+)\/(\d+)')==> doesnot
work it only works if I use #string = f3.read()
for j in range(len(string1)):
if re.search ( r'ERR2', string1[j] ):
print "Error line \n", string1[j - 1]
mo = patchnumber.search (string1[j-1])
a = mo.group()
print a
print os.getcwd()
break
Please advice how to proceed.
Thanks,
On Sun, Nov 11, 2018 at 10:30 PM wrote:
> Send Tutor mailing list submissions to
> tutor@python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
> tutor-requ...@python.org
>
> You can reach the person managing the list at
> tutor-ow...@python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
> Today's Topics:
>
>1. Re: Require Python assistance (Alan Gauld)
>2. Re: Example for read and readlines() (Alan Gauld)
>3. Re: Example for read and readlines() (Alan Gauld)
>4. Re: Example for read and readlines() (Asad)
>5. Re: Example for read and readlines() (Alan Gauld)
>
>
>
> -- Forwarded message --
> From: Alan Gauld
> To: tutor@python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 09:53:23 +
> Subject: Re: [Tutor] Require Python assistance
> On 10/11/2018 18:10, Avi Gross wrote:
> > WARNING to any that care:
> >
> > As the following letter is a repeat request without any hint they read
> the earlier comments here, I did a little searching and see very much the
> same request on another forum asking how to do this in MATLAB:
>
> The OP has also repeated posted the same message to this list
> (which I rejected as moderator).
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
>
>
> -- Forwarded message --
> From: Alan Gauld
> To: tutor@python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 10:00:33 +
> Subject: Re: [Tutor] Example for read and readlines()
> On 11/11/2018 06:49, Asad wrote:
> > Hi All ,
> >
> > If I am loading a logfile what should I use from the option
> 1,2,3
> >
> > f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log",
> 'r' )
> >
> > 1) should only iterate over f3
>
> This is best for processing line by line which is the most
> common way to handle files. It saves memory and allows you
> to exit early, without reading the entire file if you are
> only looking for say a single entry.
>
> for line in file:
>if terminal_Condition: break
># process line here
>
> > 2) st = f3.read()
>
> The best solution if you want to process individual characters
> or small character groups. Also best if you want to process
> the entire file at once, for example using a regular expression
> which might span lines.
>
> > 3) st1 = f3.readlines()
>
> Mainly historical and superseded by iterating over the file.
> But sometimes useful if you need to do multiple passes over
> the lines since it only reads the file once. Very heavy
> memory footprint for big files.
>
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
>
>
> -- Forwarded message --
> From: Alan Gauld
> To: tutor@python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 10:02:40 +
> Subject: Re: [Tutor] Example for read and readlines()
> On 11/11/2018 09:40, Steven D'Aprano wrote:
>
> >> f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log",
> 'r' )
> >
> > Don't use raw strings r"..." for pathnames.
>
> Umm, Why not?
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
>
>
> -- Forwarded message --
> From: Asad
> To: tutor@python.org
> Cc:
> Bcc:
> Date: Sun, 11 Nov 2018 15:34:35 +0530
> Subject: Re: [Tutor] Example for read and readlines()
> Hi All,
>
> thanks for the reply so to put into context say I have a file
> l
