Re: Log File

[Peter J. Holzer]

On 2023-05-31 00:22:11 -0700, ahsan iqbal wrote:
> Why we need a log file ?

A log file contains information about what your program was doing. You
use it to check that your program was performing as intended after the
fact. This is especially useful for tracking down problems with programs
which are either long-running or are running unattended.

For example, if a user tells me that they were having a problem
yesterday evening I can read the log file to see what my program was
doing at the time which will help me to pin down the problem.

The important part to remember is that a log file will only contain
information the program writes. If you didn't think to log some
information then it won't be in the log file and you can't look it up
afterwards. On the other hand you don't want to log too much information
because that might cause performance problems, it might fill up your
disks and it will be a chore to find the relevant information in a sea
of irrelevant details. So deciding what to log is a bit of an art.

> If i read a large text file than how log file help me in this regard?

It won't help you with reading the file.

However, you might want to log some information about operation, e.g.
when you started (log files usually contain time stamps), when you
ended, how large the file was, etc. That way you can compare different
runs (has the file increased in size over the last month? Was reading
especially slow yesterday?). You could also log a status message every
once in a while (e.g. every 100 MB or every 10 lines). That will
give you reassurance that the program is working and a rough estimate
when it will be finished. Or you can log any other information you think
might be useful.

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Log File

[Tobiah]

On 5/31/23 00:22, ahsan iqbal wrote:

Why we need a log file ? If i read a large text file than how log file help me 
in this regard?


If you were parsing each line of this text file looking for information,
perhaps some of the lines would not be formatted correctly, and you would be 
unable
to get the information from those lines.  Maybe you would like to
write those bad lines to a log file so that you could analyze them later.



--
https://mail.python.org/mailman/listinfo/python-list

Log File

[ahsan iqbal]

Why we need a log file ? If i read a large text file than how log file help me 
in this regard?
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue44533] Where are the log file(s)

[Éric Araujo]

Change by Éric Araujo :


--
resolution: third party -> not a bug
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44533] Where are the log file(s)

[Éric Araujo]

Éric Araujo  added the comment:

The python interpreter does not create log files by itself, it would be the 
specific command or script that you ran that does that and prints the message 
you show.

--
nosy: +eric.araujo
resolution:  -> third party
status: open -> pending
type: behavior -> 
versions:  -Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44533] Where are the log file(s)

[tygrus]

New submission from tygrus :

The Python app shows errors and says "Check the logs for full command output."
But where are the logs?
And how to understand the data in the logs?

--
messages: 396704
nosy: tygrus
priority: normal
severity: normal
status: open
title: Where are the log file(s)
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue44533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Is there a log file that tracks every statement that is being executed when a program is running?

[Mladen Gogala via Python-list]

On Sun, 25 Oct 2020 12:14:52 +0100, Maxime S wrote:

> Hi,
> 
> You can use the trace module for that:
> https://docs.python.org/3.8/library/trace.html
> 
> Personally I tend to put print statement at strategic places instead, I
> find that easier to analyse than a full trace but YMMV.
> 
> Maxime

Thank you. I am a YAPN (yet another Python newbie) and this helps me a 
lot.

-- 
Mladen Gogala
Database Consultant
http://mgogala.byethost5.com
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Is there a log file that tracks every statement that is being executed when a program is running?

[Steve]

Yes, I have been doing this already.  I include the line number to help me 
find, or search for, the location of the print statement.  Unfortunately, it 
requires a keen understanding in which the program has progressed in order to 
trace.

A printout of every line number executed could help.

 

 

FootNote:
If money does not grow on trees, then why do banks have branches?

 

From: Maxime S  
Sent: Sunday, October 25, 2020 7:15 AM
To: Steve 
Cc: Python 
Subject: Re: Is there a log file that tracks every statement that is being 
executed when a program is running?

 

Hi,

 

You can use the trace module for that: 
https://docs.python.org/3.8/library/trace.html

 

Personally I tend to put print statement at strategic places instead, I find 
that easier to analyse than a full trace but YMMV.

 

Maxime

 

Le dim. 25 oct. 2020 à 01:25, Steve mailto:Gronicus@sga.ninja> > a écrit :

This would seriously help troubleshooting for me.  I updated a data file and
now my main program is choking on it.  When the program encounters an error,
it dumps a bit of information to the screen for a few steps before the error
but that is not enough.




Footnote:
English sprakers on a roller coaster: "W"
Spanish speakers on a rollercoaster:  " Nosostros"

-Original Message-
From: Python-list mailto:sga.ni...@python.org> > On
Behalf Of shrimp_banana
Sent: Saturday, October 17, 2020 9:47 PM
To: python-list@python.org <mailto:python-list@python.org> 
Subject: Re: File Name issue

On 10/17/20 4:12 PM, Steve wrote:
 > The line:
 > with open("HOURLYLOG.txt", 'r') as infile:
 > works but, when I rename the file, the line:
 > with open("HOURLY-LOG.txt", 'r') as infile:
 > does not.  The complaint is: Cannot Assign to operator  >  > However, I
have:
 > BPM_O2s=open("BPM-O2-Readings.txt","a")
 > And it works.
 >
 > At first, I thought the issue was due to having the - in the filename.
 >
 > Is there a fix or explanation for this?
 > Steve

I am unsure if this will help but you could try putting an r in front of the
quotes to make it take raw data only.
ie.

with open(r"HOURLY-LOG.txt", 'r') as infile
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a log file that tracks every statement that is being executed when a program is running?

[Maxime S]

Hi,

You can use the trace module for that:
https://docs.python.org/3.8/library/trace.html

Personally I tend to put print statement at strategic places instead, I
find that easier to analyse than a full trace but YMMV.

Maxime


Le dim. 25 oct. 2020 à 01:25, Steve  a écrit :

> This would seriously help troubleshooting for me.  I updated a data file
> and
> now my main program is choking on it.  When the program encounters an
> error,
> it dumps a bit of information to the screen for a few steps before the
> error
> but that is not enough.
>
>
>
>
> Footnote:
> English sprakers on a roller coaster: "W"
> Spanish speakers on a rollercoaster:  " Nosostros"
>
> -Original Message-
> From: Python-list  On
> Behalf Of shrimp_banana
> Sent: Saturday, October 17, 2020 9:47 PM
> To: python-list@python.org
> Subject: Re: File Name issue
>
> On 10/17/20 4:12 PM, Steve wrote:
>  > The line:
>  > with open("HOURLYLOG.txt", 'r') as infile:
>  > works but, when I rename the file, the line:
>  > with open("HOURLY-LOG.txt", 'r') as infile:
>  > does not.  The complaint is: Cannot Assign to operator  >  > However, I
> have:
>  > BPM_O2s=open("BPM-O2-Readings.txt","a")
>  > And it works.
>  >
>  > At first, I thought the issue was due to having the - in the filename.
>  >
>  > Is there a fix or explanation for this?
>  > Steve
>
> I am unsure if this will help but you could try putting an r in front of
> the
> quotes to make it take raw data only.
> ie.
>
> with open(r"HOURLY-LOG.txt", 'r') as infile
> --
> https://mail.python.org/mailman/listinfo/python-list
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a log file that tracks every statement that is being executed when a program is running?

[MRAB]

On 2020-10-25 00:21, Steve wrote:

This would seriously help troubleshooting for me.  I updated a data file and
now my main program is choking on it.  When the program encounters an error,
it dumps a bit of information to the screen for a few steps before the error
but that is not enough.

You could install Visual Studio Code and the extension for Python. That 
will let you single-step through the code.

--
https://mail.python.org/mailman/listinfo/python-list

Is there a log file that tracks every statement that is being executed when a program is running?

[Steve]

This would seriously help troubleshooting for me.  I updated a data file and
now my main program is choking on it.  When the program encounters an error,
it dumps a bit of information to the screen for a few steps before the error
but that is not enough.




Footnote:
English sprakers on a roller coaster: "W"
Spanish speakers on a rollercoaster:  " Nosostros"

-Original Message-
From: Python-list  On
Behalf Of shrimp_banana
Sent: Saturday, October 17, 2020 9:47 PM
To: python-list@python.org
Subject: Re: File Name issue

On 10/17/20 4:12 PM, Steve wrote:
 > The line:
 > with open("HOURLYLOG.txt", 'r') as infile:
 > works but, when I rename the file, the line:
 > with open("HOURLY-LOG.txt", 'r') as infile:
 > does not.  The complaint is: Cannot Assign to operator  >  > However, I
have:
 > BPM_O2s=open("BPM-O2-Readings.txt","a")
 > And it works.
 >
 > At first, I thought the issue was due to having the - in the filename.
 >
 > Is there a fix or explanation for this?
 > Steve

I am unsure if this will help but you could try putting an r in front of the
quotes to make it take raw data only.
ie.

with open(r"HOURLY-LOG.txt", 'r') as infile
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[Sharan Basappa]

On Sunday, 24 March 2019 14:20:36 UTC+5:30, Sharan Basappa  wrote:
> On Sunday, 24 March 2019 10:57:13 UTC+5:30, Cameron Simpson  wrote:
> > On 23Mar2019 21:47, Sharan Basappa  wrote:
> > >On Friday, 22 March 2019 09:13:18 UTC+5:30, MRAB  wrote:
> > >> On 2019-03-22 03:25, Sharan Basappa wrote:
> > >> > I am running a program and even though the program runs all fine, the 
> > >> > log file is missing. I have pasted first few lines of the code.
> > >> > Any suggestions where I maybe going wrong?
> > [...]
> > >> > #Create and configure logger
> > >> > logging.basicConfig(filename="test_1.log", filemode='w', 
> > >> > format='%(asctime)s %(message)s')
> > >> >
> > >> Are you sure that you know where it's putting the log file? You have a
> > >> relative path there, but relative to where? Try it with an absolute path.
> > >>
> > >> Are you sure that it's logging anything? Log a simple message just after
> > >> configuring to double-check.
> > >
> > >Would the file not get logged using the current directory?
> > 
> > It should be.
> > 
> > >BTW, I changed the file location as follows and still I don't see it:
> > >
> > >logging.basicConfig(filename="D:\Users\sharanb\OneDrive - HCL 
> > >Technologies Ltd\Projects\MyBackup\Projects\Initiatives\machine 
> > >learning\programs\assertion\CNN\test_1.log", filemode='w', 
> > >format='%(asctime)s %(message)s')
> > 
> > Did you read my other recent post? You should define strings with 
> > backslashes in them such are your file path as 'raw strings', which 
> > start with r' or r" instead of a plain quote character. Raw strings do 
> > not interpret \x escapes. In particular your path above has a '\a' in 
> > it, which is not a backslash followed by an "a", it is a BEL character.
> > 
> > That said, I can't see what's wrong with your original example which has 
> > no backslashes.
> > 
> > Cheers,
> > Cameron Simpson 
> 
> Ah. I finally solved the issue though I don't know what the problem itself it.
> This is how the new code looks like:
> 
> for handler in logging.root.handlers[:]:
> logging.root.removeHandler(handler)
> 
> #Create and configure logger
> filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), 
> 'test_1.log')
> logging.basicConfig(filename=filename, filemode='w', format='%(asctime)s 
> %(message)s')
> 
> I found the above tip from the following discussion:
> https://stackoverflow.com/questions/30861524/logging-basicconfig-not-creating-log-file-when-i-run-in-pycharm

I think I got some more idea about the issue though I still don't know the root 
cause.
So, my test program is importing design file.
Both test and design file are have the following lines:
#Create and configure logger
filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), 
'test_5.log')
logging.basicConfig(filename=filename, filemode='w', format='%(asctime)s 
%(message)s')
#Creating an object
logger = logging.getLogger()

I think this is causing the issue. Anyway, I am able to log from the test 
program. However, I am not able to log anything from design program.

I will open a separate thread on this topic.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[Luuk]

On 24-3-2019 09:50, Sharan Basappa wrote:


Ah. I finally solved the issue though I don't know what the problem itself it.



The problem, shown with a simple example

Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 22:20:52) [MSC v.1916 
32 bit (Intel)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> print ("hallo")
hallo
>>> print ("h\allo")
hllo
>>> print (r"h\allo")
h\allo
>>>

The first line is a simple print statement to print the text "hallo"   
(which is Dutch for "hello")


In the second line i added a '\' (backslash), the letter 'a' seems to be 
missing, unless audio on your computer works. If audio works you will 
hear a bell.

(see: https://docs.python.org/2.0/ref/strings.html )

In the third line a 'r' is put in front of the string, and now the 'a' 
is shown again (and also the '\').



In your path there is something like '...programs\assertion\CNN..'.

Python does see a '\a', after 'programs', and before 'ssertion', and 
tries to sound a 'bell'





--
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[Sharan Basappa]

On Sunday, 24 March 2019 10:57:13 UTC+5:30, Cameron Simpson  wrote:
> On 23Mar2019 21:47, Sharan Basappa  wrote:
> >On Friday, 22 March 2019 09:13:18 UTC+5:30, MRAB  wrote:
> >> On 2019-03-22 03:25, Sharan Basappa wrote:
> >> > I am running a program and even though the program runs all fine, the 
> >> > log file is missing. I have pasted first few lines of the code.
> >> > Any suggestions where I maybe going wrong?
> [...]
> >> > #Create and configure logger
> >> > logging.basicConfig(filename="test_1.log", filemode='w', 
> >> > format='%(asctime)s %(message)s')
> >> >
> >> Are you sure that you know where it's putting the log file? You have a
> >> relative path there, but relative to where? Try it with an absolute path.
> >>
> >> Are you sure that it's logging anything? Log a simple message just after
> >> configuring to double-check.
> >
> >Would the file not get logged using the current directory?
> 
> It should be.
> 
> >BTW, I changed the file location as follows and still I don't see it:
> >
> >logging.basicConfig(filename="D:\Users\sharanb\OneDrive - HCL 
> >Technologies Ltd\Projects\MyBackup\Projects\Initiatives\machine 
> >learning\programs\assertion\CNN\test_1.log", filemode='w', 
> >format='%(asctime)s %(message)s')
> 
> Did you read my other recent post? You should define strings with 
> backslashes in them such are your file path as 'raw strings', which 
> start with r' or r" instead of a plain quote character. Raw strings do 
> not interpret \x escapes. In particular your path above has a '\a' in 
> it, which is not a backslash followed by an "a", it is a BEL character.
> 
> That said, I can't see what's wrong with your original example which has 
> no backslashes.
> 
> Cheers,
> Cameron Simpson 

Ah. I finally solved the issue though I don't know what the problem itself it.
This is how the new code looks like:

for handler in logging.root.handlers[:]:
logging.root.removeHandler(handler)

#Create and configure logger
filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), 
'test_1.log')
logging.basicConfig(filename=filename, filemode='w', format='%(asctime)s 
%(message)s')

I found the above tip from the following discussion:
https://stackoverflow.com/questions/30861524/logging-basicconfig-not-creating-log-file-when-i-run-in-pycharm
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[Cameron Simpson]

On 23Mar2019 21:47, Sharan Basappa  wrote:

On Friday, 22 March 2019 09:13:18 UTC+5:30, MRAB  wrote:

On 2019-03-22 03:25, Sharan Basappa wrote:
> I am running a program and even though the program runs all fine, the log 
file is missing. I have pasted first few lines of the code.
> Any suggestions where I maybe going wrong?

[...]

> #Create and configure logger
> logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
%(message)s')
>
Are you sure that you know where it's putting the log file? You have a
relative path there, but relative to where? Try it with an absolute path.

Are you sure that it's logging anything? Log a simple message just after
configuring to double-check.


Would the file not get logged using the current directory?


It should be.


BTW, I changed the file location as follows and still I don't see it:

logging.basicConfig(filename="D:\Users\sharanb\OneDrive - HCL 
Technologies Ltd\Projects\MyBackup\Projects\Initiatives\machine 
learning\programs\assertion\CNN\test_1.log", filemode='w', 
format='%(asctime)s %(message)s')


Did you read my other recent post? You should define strings with 
backslashes in them such are your file path as 'raw strings', which 
start with r' or r" instead of a plain quote character. Raw strings do 
not interpret \x escapes. In particular your path above has a '\a' in 
it, which is not a backslash followed by an "a", it is a BEL character.


That said, I can't see what's wrong with your original example which has 
no backslashes.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[Sharan Basappa]

On Friday, 22 March 2019 09:13:18 UTC+5:30, MRAB  wrote:
> On 2019-03-22 03:25, Sharan Basappa wrote:
> > I am running a program and even though the program runs all fine, the log 
> > file is missing. I have pasted first few lines of the code.
> > 
> > Any suggestions where I maybe going wrong?
> > 
> > import os
> > import csv
> > import logging
> > import assertion_design as asd
> > import random
> > 
> > #Create and configure logger
> > logging.basicConfig(filename="test_1.log", filemode='w', 
> > format='%(asctime)s %(message)s')
> > import os
> > import csv
> > import logging
> > import assertion_design as asd
> > import random
> > 
> > #Create and configure logger
> > logging.basicConfig(filename="test_1.log", filemode='w', 
> > format='%(asctime)s %(message)s')
> > 
> Are you sure that you know where it's putting the log file? You have a 
> relative path there, but relative to where? Try it with an absolute path.
> 
> Are you sure that it's logging anything? Log a simple message just after 
> configuring to double-check.

Would the file not get logged using the current directory?

BTW, I changed the file location as follows and still I don't see it:

logging.basicConfig(filename="D:\Users\sharanb\OneDrive - HCL Technologies 
Ltd\Projects\MyBackup\Projects\Initiatives\machine 
learning\programs\assertion\CNN\test_1.log", filemode='w', format='%(asctime)s 
%(message)s')
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[Sharan Basappa]

On Friday, 22 March 2019 09:09:16 UTC+5:30, adam@gmail.com  wrote:
> On Thursday, March 21, 2019 at 10:26:14 PM UTC-5, Sharan Basappa wrote:
> > I am running a program and even though the program runs all fine, the log 
> > file is missing. I have pasted first few lines of the code.
> > 
> I am thinking--hoping, rather--that you just kind of double pasted there. 
> Anyways, you needed to specify the logging level in basicConfig:
> 
> import os
> import csv
> import logging
> import random
> 
> #Create and configure logger
> # Check out level=logging.INFO
> logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
> %(message)s', level=logging.INFO)
> log = logging.getLogger()
> log.info("Yay!")

Adam,

I don't understand. The logger is set to DEBUG level using the following line.
logger.setLevel(logging.DEBUG)

Do you see anything incorrect?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[Sharan Basappa]

On Friday, 22 March 2019 09:09:14 UTC+5:30, DL Neil  wrote:
> On 22/03/19 4:25 PM, Sharan Basappa wrote:
> > I am running a program and even though the program runs all fine, the log 
> > file is missing. I have pasted first few lines of the code.
> > 
> > Any suggestions where I maybe going wrong?
> > 
> > import os
> > import csv
> > import logging
> > import assertion_design as asd
> > import random
> > 
> > #Create and configure logger
> > logging.basicConfig(filename="test_1.log", filemode='w', 
> > format='%(asctime)s %(message)s')
> > import os
> > import csv
> > import logging
> > import assertion_design as asd
> > import random
> > 
> > #Create and configure logger
> > logging.basicConfig(filename="test_1.log", filemode='w', 
> > format='%(asctime)s %(message)s')
> > 
> 
> 
> Do all of these lines actually appear, or is it an accidental 
> copy-paste+paste?
> 
> What do you use to actually record data in the log? eg
> 
>   logging.info("Informational message")
> 
> Any error message from Python?
> 
> When you say "the log file is missing" do you mean that there is no such 
> file, that there is an empty file, or what? Which directory do you run 
> the code from, and in which directory do you expect to find the log file?
> 
> -- 
> Regards =dn

There is no log file at all in the directory where I am running.
I expected the log file to be in the directory where I am running the test from.

Let me describe it in more detail.
This is the directory - "D:\Users\sharanb\OneDrive - HCL Technologies 
Ltd\Projects\MyBackup\Projects\Initiatives\machine 
learning\programs\assertion\CNN"

The test program imports the design and runs some test on it.
Both the design file and test file are in the above directory.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[MRAB]

On 2019-03-22 03:25, Sharan Basappa wrote:

I am running a program and even though the program runs all fine, the log file 
is missing. I have pasted first few lines of the code.

Any suggestions where I maybe going wrong?

import os
import csv
import logging
import assertion_design as asd
import random

#Create and configure logger
logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
%(message)s')
import os
import csv
import logging
import assertion_design as asd
import random

#Create and configure logger
logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
%(message)s')

Are you sure that you know where it's putting the log file? You have a 
relative path there, but relative to where? Try it with an absolute path.


Are you sure that it's logging anything? Log a simple message just after 
configuring to double-check.

--
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[adam . preble]

On Thursday, March 21, 2019 at 10:26:14 PM UTC-5, Sharan Basappa wrote:
> I am running a program and even though the program runs all fine, the log 
> file is missing. I have pasted first few lines of the code.
> 
I am thinking--hoping, rather--that you just kind of double pasted there. 
Anyways, you needed to specify the logging level in basicConfig:

import os
import csv
import logging
import random

#Create and configure logger
# Check out level=logging.INFO
logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
%(message)s', level=logging.INFO)
log = logging.getLogger()
log.info("Yay!")
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file

[DL Neil]

On 22/03/19 4:25 PM, Sharan Basappa wrote:

I am running a program and even though the program runs all fine, the log file 
is missing. I have pasted first few lines of the code.

Any suggestions where I maybe going wrong?

import os
import csv
import logging
import assertion_design as asd
import random

#Create and configure logger
logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
%(message)s')
import os
import csv
import logging
import assertion_design as asd
import random

#Create and configure logger
logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
%(message)s')




Do all of these lines actually appear, or is it an accidental 
copy-paste+paste?


What do you use to actually record data in the log? eg

logging.info("Informational message")

Any error message from Python?

When you say "the log file is missing" do you mean that there is no such 
file, that there is an empty file, or what? Which directory do you run 
the code from, and in which directory do you expect to find the log file?


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

log file

[Sharan Basappa]

I am running a program and even though the program runs all fine, the log file 
is missing. I have pasted first few lines of the code.

Any suggestions where I maybe going wrong?

import os
import csv
import logging
import assertion_design as asd
import random

#Create and configure logger
logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
%(message)s')
import os
import csv
import logging
import assertion_design as asd
import random

#Create and configure logger
logging.basicConfig(filename="test_1.log", filemode='w', format='%(asctime)s 
%(message)s')
-- 
https://mail.python.org/mailman/listinfo/python-list

In ports of a log file, how to detect if the ports are dangerous using python?

[Rockzers]

I know basic python and I have a log file, also I have print the output of 
ports from the log file which there are so many ports in the output.
I want to know how to take only the dangerous ports from the printed ports - 
Also I need to take the IP addresses from the dangerous ports - Finally how to 
know if the IP addresses are local IP or global IP

import os
from collections import Counter
asc_order = []
def openfile(filename):
if os.path.exists(filename):
return open(filename, "r").read()
else:
return None
def parselog(logline):
c = logline.split(" ")
r = {}
i = -1
for var in c:
i += 1
if i == 1:
a = var.split("\t")
for el in a:
if el.startswith("date="): r["date"] = el.split("=")[1]
elif i > 1:
v = var.split("=", 1)
try:
r[v[0]] = v[1].strip("\"")
except:
pass
return r
def splitline(logall):
c = logall.split("\n")
r = []
for el in c:
r.append(el.strip("\r"))
return r
def main():
f = openfile("/Users/angelin/Desktop/new 
sec/2017-04-18_010.082.012.003.txt")
if f is None:
print("File not found")
return
s = splitline(f)
counts = {}
for el in s:
if len(el) > 50:
p = parselog(el)
if "dstport" in p:
# increment counter
if p["dstport"] in counts:
counts[str(p["dstport"])] += 1
else:
counts[str(p["dstport"])] = 1
asc_order.append(p["dstport"])
ascending = map(int, asc_order)
ascending.sort()
for port in ascending:
print ("Dest Port : %d" % port)
print ""
k = map(int, counts.keys())
k.sort()
sorted(k, key=counts.get)
y = sorted(counts.items(), key=lambda x: x[1], reverse=True)
for x, z in y:
print  ('Dest Port %s Count: %s' % (x, z))


if __name__ == "__main__": main()

example log file

2017-04-17 00:00:00 Local7.Info 10.82.12.3 date=2017-04-16 time=23:59:59 
devname=IDS-DC14-001 devid=FGT90D3Z15018997 logid=1059028704 type=utm 
subtype=app-ctrl eventtype=app-ctrl-all level=information vd=root appid=27946 
user="" srcip=10.80.10.249 srcport=9170 srcintf="wan1" dstip=208.91.112.198 
dstport=53 dstintf="wan1" profiletype="applist" proto=17 service="DNS" 
policyid=3 sessionid=39717767 applist="sniffer-profile" appcat="Cloud.IT" 
app="Fortiguard.Search" action=pass msg="Cloud.IT: Fortiguard.Search," 
apprisk=medium

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file.

[Paul Rudin]

Joaquin Alzola  writes:

> This email is confidential and may be subject to privilege. If you are
> not the intended recipient, please do not copy or disclose its content
> but contact the sender immediately upon receipt.

Probably not a good idea to send it to a publicly accessible resource
then :)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file.

[Paul Owen]

Thank you Steven.
that is just what I need.

regards Paul.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file.

[Steven D'Aprano]

On Wed, 15 Jun 2016 05:54 am, Paul Owen wrote:

> logging please.
> 
> my pump programme works but I want to log the time etc. when the pump runs
> and stops.
> 
> I am trying to improve the programme
> 
> I am a novice!

As a novice, start with the simplest thing. Whenever the pump runs or stops,
just call print:


start(pump)
print("Starting")
...
stop(pump)
print("Stopping")


To log to a file instead, change the program to:


import logging
logging.basicConfig(filename='example.log',level=logging.DEBUG)
start(pump)
logging.info("Starting")
...
stop(pump)
logging.info("Stopping")


More information in the logging tutorial:

https://docs.python.org/2/howto/logging.html
https://docs.python.org/3/howto/logging.html



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

RE: log file.

[Paul Owen]

Gmail

Google+

Calendar

Web

more

Inbox

pump programme

Paul Owen

to me
1 hour ago

Details

from gpiozero import LED,Button

from signal import pause

print ("Pump Programme Running")

led = LED(17)

low = Button (2)

high = Button (3)

high.when_pressed = led.on 

low.when_released = led.off

#as you can see I need a simple way to log when led(pump) runs

hope seeing this helps.

regards Paul.
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: log file.

[Joaquin Alzola]

>thanks I will look at them.

As an example for your guide:

##log_handler###
import sys
import logging
import logging.handlers
from configfile_read import *

def LogHandler(logger):
CONFIG_FILE = ('./conf/' + 'config.ini')
config = configfile(CONFIG_FILE)

FORMAT = '%(asctime)-15s %(name)s %(levelname)s-%(levelno)s %(message)s'
LOG_LEVEL = config.get("logging","Logging_Level")
LOG_FILE = config.get("logging","Logging_File")
LOG_DIRECTORY = config.get("logging","Logging_Directory")
LOG_BYTES = config.get("logging","Logging_bytes")
LOG_COUNT = config.get("logging","Logging_count")

LEVEL = logging.getLevelName(LOG_LEVEL)
logger.setLevel(LEVEL)
handler = logging.handlers.RotatingFileHandler(LOG_DIRECTORY + 
LOG_FILE,maxBytes=int(LOG_BYTES),backupCount=int(LOG_COUNT))
handler.setFormatter(logging.Formatter(FORMAT))
return handler


On each part of your code start a logger:
logger = logging.getLogger('__main__.' + __name__)

And in the main part:
from log_handler import *
#Logging Config Starts
logger = logging.getLogger(__name__)
logger.addHandler(LogHandler(logger))
#Logging Config Ends

Config_parser where log parameters are taken
import configparser
import os, sys
import logging
import logging.handlers
from log_handler import *

logger = logging.getLogger('__main__.' + __name__)

class configfile:

def __init__(self,CONFIG_FILE):
self.CONFIG_FILE = CONFIG_FILE
if not os.path.isfile(self.CONFIG_FILE):
logger.critical('File NOT found: %s' % CONFIG_FILE + '. 
This file is needed for the system to work')
sys.exit()

def get(self,general,parameter):
config = configparser.ConfigParser()
config.read(self.CONFIG_FILE)
try:
result = config.get(general,parameter)
except configparser.NoSectionError as e:
logger.critical('File %s is there but need information 
inside: %s'%(self.CONFIG_FILE,e))
sys.exit()
logger.debug('Parameters pass to configparser-->' + general + 
':' + parameter + ' with result-->' + result)
return result


testbedocg:/usr/local/statistics_server/conf # more config.ini
[config]
Host=172.1.1.1
Port=12346

[logging]
#Possbile Values: CRITICAL ERROR WARNING INFO DEBUG NOTSET
Logging_Level=DEBUG
Logging_File=server_log.log
Logging_Directory=./logs/
Logging_bytes=2097152
Logging_count=5

This email is confidential and may be subject to privilege. If you are not the 
intended recipient, please do not copy or disclose its content but contact the 
sender immediately upon receipt.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file.

[Joel Goldstick]

On Tue, Jun 14, 2016 at 4:14 PM, Paul Owen <owenpaul...@gmail.com> wrote:
> thanks I will look at them.
> --
> https://mail.python.org/mailman/listinfo/python-list


If your program is called 'my_program.py'  you can write print
statements (or print functions in python 3) and when you run your
program, those prints will show on your screen.  If you want to save
them to a log file do this:

python my_program.py >> my_log_file.log

When your program ends, open up my_log_file.log to see what happened.
The two > characters will write to the file if it is new, or append to
the file if it exists


-- 
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: log file.

[Paul Owen]

thanks I will look at them.
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: log file.

[Joaquin Alzola]

>logging please.

Check this modules:
import logging
import logging.handlers
This email is confidential and may be subject to privilege. If you are not the 
intended recipient, please do not copy or disclose its content but contact the 
sender immediately upon receipt.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file.

[Paul Owen]

logging please.

my pump programme works but I want to log the time etc. when the pump runs and 
stops.

I am trying to improve the programme 

I am a novice!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file.

[John Gordon]

In <05d2df77-8cd0-467b-8ab3-54bf730d8...@googlegroups.com> 
owenpaul...@gmail.com writes:

> I have a programme to pump out water from a sump and would like to
> log to a readable file when the pump operates. what is the easiest
> way to acheive this with python 3.

Are you asking for help with logging, or communicating with the pump?

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file.

[Paul Owen]

I am very inexperienced at programming.!

is there a lot of code needed to use those modules.

regards paul.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: log file.

[Jussi Piitulainen]

owenpaul...@gmail.com writes:

> I have a programme to pump out water from a sump and would like to log
> to a readable file when the pump operates. what is the easiest way to
> acheive this with python 3.

Depending on any number of details, the easiest may be to just print.
Direct stdout and stderr to files when you run the program.
-- 
https://mail.python.org/mailman/listinfo/python-list

log file.

[owenpaul . po]

 I have a programme to pump out water from a sump and would like to log to a 
readable file when the pump operates. what is the easiest way to acheive this 
with python 3.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to log the output from the multiple telnet sessions to separate log file

[dieter]

manjunatha.mahalinga...@gmail.com writes:
> ...
>  I created the my own class MyLogger and passing log file name to it. I'm 
> seeing no log is being written to passed log file instead everything is 
> written to the logfilename [actually logfilename is variable with file name] 
> I'm trying to create MyLogger object for each telnet session. and use that 
> object.
>
> MyLogger is beeing called inside the Telnetsession class. those are present 
> in PmTelnet3.py file.
>
>
> class MyLogger():
>   import logging
>   def __init__(self,logfilename):
>   #create logger with 'spam_application'
>   self.logger = self.logging.getLogger('TelnetSession')

This will always give you the same logger object.

I recommend to carefully read the "logging" related documentation
to understand its architecture. In your context, the logger
hierarchy and the concept of "logging handler" are of great importance.

> ...
>   self.logging.basicConfig(filename = logfilename,
>   level = 
> self.logging.INFO ,
>   format= 
> '%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
>   datefmt='%m-%d 
> %H:%M',
>   filemode='w')
This will configure the so called "root" logger - the object at
the root of the logger hierarchy. Loggers lower down in the hiearrchy
will "inherit" from loggers higher up, unless they override things.

Therefore, you will usually have a single "basicConfig" call, which
globally (independent of the individual telnet sessions) configures
common aspects. In the individual telnet sessions, you would
use session specific loggers (below the "root" logger) which would
inherit the common configuration and override session specific things
(e.g. the log file via a specific handler).

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to log the output from the multiple telnet sessions to separate log file

[manjunatha . mahalingappa]

On Friday, October 16, 2015 at 11:11:58 PM UTC-7, dieter wrote:
> manjunatha.mahalinga...@gmail.com writes:
> > I'm very much new to python. 
> > I'm doing the automation for networking device testing , I will be opening 
> > the 4 telnet session, and doing some testing operations on each of those  
> > telnet sessions and capture or log the respective output in 4 different log 
> > files.
> 
> Personally, I find it a bit strange that each (telnet) session
> should get its own logfile, but, if that is what you need, I would
> approach it as follows:
> 
>   * define a class "TelnetSession"; create a logger in its "__init__" method;
> use this logger for all operations inside a "TelnetSession"
> 
>   * instantiate the class "TelnetSession" for each telnet session you
> want to open.
> Use those objects method to operate on the sessions.

Hello Dieter,

 I created the my own class MyLogger and passing log file name to it. I'm 
seeing no log is being written to passed log file instead everything is written 
to the logfilename [actually logfilename is variable with file name] I'm trying 
to create MyLogger object for each telnet session. and use that object.

MyLogger is beeing called inside the Telnetsession class. those are present in 
PmTelnet3.py file.


class MyLogger():
import logging
def __init__(self,logfilename):
#create logger with 'spam_application'
self.logger = self.logging.getLogger('TelnetSession')
self.logger.setLevel(self.logging.INFO)
# create file handler which logs even debug messages
#self.fh = self.logging.FileHandler(logfile)
print "The log file name is %s\n" % logfilename
self.logging.basicConfig(filename = logfilename,
level = 
self.logging.INFO ,
format= 
'%(asctime)s %(name)-12s %(levelname)-8s %(message)s',
datefmt='%m-%d 
%H:%M',
filemode='w')
# create console handler with a higher log level
self.ch = self.logging.StreamHandler()


def Log(self):
self.logger.setLevel(self.logging.INFO)
self.fh.setLevel(self.logging.INFO)
self.ch.setLevel(self.logging.INFO)
self.fh.setFormatter(self.formatter)
self.ch.setFormatter(self.formatter)
self.logger.addHandler(self.ch)


class TelnetSession:

import telnetlib
import logging
 
def __init__(self, inHost, inPort, Logname):
self.log = MyLogger(Logname)
self.telnet = self.telnetlib.Telnet(inHost, inPort)


def IP_port(file) :
global Testbed_info
Testbed_info = []
F = open(file, 'r')
F.seek(0)
line = F.read()
tuples = re.findall(r'(.+ \d)\s+(.+?)\s+(\d+)', line)
for (dname, ip, port) in tuples :
LogFileName = dname.replace(" ","") + ".log"
#Log = open(logfile, "a")
#Log = MyLogger(LogFileName)
Telnet_handle=PmTelnet3.TelnetSession(ip,port,LogFileName)
tuple = (dname, ip, port, Telnet_handle )
print tuple
Testbed_info.append(tuple)
#T.append(T1)
return(Testbed_info)


#This function is to switch the console

file = '/users/manmahal/MANJU/IP_port.txt'
Testbed_info = IP_port(file)
print "Iam done"


for (dname, ip, port, Telnet) in Testbed_info :

My_handle = Telnet
#My_log = log

My_handle.Write("\n")
My_handle.Write("show config \n")
time.sleep(2)
output = My_handle.ReadBuffer()
#My_fp.write(My_handle.ReadBuffer())
My_handle.log.logger.info(output)
My_handle.log.logger.info("HI THERE")

Kindly let me know how can I fix this..
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to log the output from the multiple telnet sessions to separate log file

[Chris Angelico]

On Sat, Oct 17, 2015 at 4:47 PM,   wrote:
> class Logger():
> def __init__(self,log):
> self.terminal = sys.stdout
> self.log = log
>
> def write(self, message):
> self.terminal.write(message)
> self.log.write(message)
>
>
> for (dname, ip, port) in tuples :
> sys.stdout =Logger(log)

Every time you construct a Logger, it snapshots the previous value of
sys.stdout. By the time you've set them all up, sys.stdout is the last
Logger created, which will chain to the one created previous to it,
then the one before that, and finally to the actual console.

Do you need to use sys.stdout at all? It would be a lot easier to
ignore stdout and just write your logs directly to files.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to log the output from the multiple telnet sessions to separate log file

[dieter]

manjunatha.mahalinga...@gmail.com writes:
> I'm very much new to python. 
> I'm doing the automation for networking device testing , I will be opening 
> the 4 telnet session, and doing some testing operations on each of those  
> telnet sessions and capture or log the respective output in 4 different log 
> files.

Personally, I find it a bit strange that each (telnet) session
should get its own logfile, but, if that is what you need, I would
approach it as follows:

  * define a class "TelnetSession"; create a logger in its "__init__" method;
use this logger for all operations inside a "TelnetSession"

  * instantiate the class "TelnetSession" for each telnet session you
want to open.
Use those objects method to operate on the sessions.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to log the output from the multiple telnet sessions to separate log file

[manjunatha . mahalingappa]

On Friday, October 16, 2015 at 11:06:24 PM UTC-7, Chris Angelico wrote:
> On Sat, Oct 17, 2015 at 4:47 PM,   wrote:
> > class Logger():
> > def __init__(self,log):
> > self.terminal = sys.stdout
> > self.log = log
> >
> > def write(self, message):
> > self.terminal.write(message)
> > self.log.write(message)
> >
> >
> > for (dname, ip, port) in tuples :
> > sys.stdout =Logger(log)
> 
> Every time you construct a Logger, it snapshots the previous value of
> sys.stdout. By the time you've set them all up, sys.stdout is the last
> Logger created, which will chain to the one created previous to it,
> then the one before that, and finally to the actual console.
> 
> Do you need to use sys.stdout at all? It would be a lot easier to
> ignore stdout and just write your logs directly to files.
> 
> ChrisA


Hello ChrisA,

Thank you so much for the replay,  objective is to collect all terminal output, 
if we have any other option to do the same with out using  stdout your most 
welcome to suggest me. 

Regards
Manju
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to log the output from the multiple telnet sessions to separate log file

[manjunatha . mahalingappa]

On Friday, October 16, 2015 at 11:11:58 PM UTC-7, dieter wrote:
> manjunatha.mahalinga...@gmail.com writes:
> > I'm very much new to python. 
> > I'm doing the automation for networking device testing , I will be opening 
> > the 4 telnet session, and doing some testing operations on each of those  
> > telnet sessions and capture or log the respective output in 4 different log 
> > files.
> 
> Personally, I find it a bit strange that each (telnet) session
> should get its own logfile, but, if that is what you need, I would
> approach it as follows:
> 
>   * define a class "TelnetSession"; create a logger in its "__init__" method;
> use this logger for all operations inside a "TelnetSession"
> 
>   * instantiate the class "TelnetSession" for each telnet session you
> want to open.
> Use those objects method to operate on the sessions.


Hello Dieter,

Thank you so much for the reply and suggestion.. I believe this approch should 
work for me..!! thanks for your time..


Regards
Manju 
-- 
https://mail.python.org/mailman/listinfo/python-list

How to log the output from the multiple telnet sessions to separate log file

[manjunatha . mahalingappa]

Hello All,

I'm very much new to python. 
I'm doing the automation for networking device testing , I will be opening the 
4 telnet session, and doing some testing operations on each of those  telnet 
sessions and capture or log the respective output in 4 different log files. As 
of now all the 4 log files have the same  content  kindly help me to  log the 
output respective log file.

Code snippet is given below:

import sys
import PmTelnet2
import logging
import re
import time

class Logger():
def __init__(self,log):
self.terminal = sys.stdout
self.log = log

def write(self, message):
self.terminal.write(message)
self.log.write(message)


#This function is to open the telnet session  and open the file for each device.
def IP_port(file) :
global Testbed_info
Testbed_info = []
F = open(file, 'r')
F.seek(0)
line = F.read()
tuples = re.findall(r'(.+ \d)\s+(.+?)\s+(\d+)', line)
for (dname, ip, port) in tuples :
logfile = dname.replace(" ","") + ".log"
log = open(logfile, "a")
Telnet_handle=PmTelnet2.TelnetSession(ip,port)
sys.stdout =Logger(log)
tuple = (dname, ip, port, Telnet_handle)
Testbed_info.append(tuple)
#T.append(T1)
return(Testbed_info)


# Here I'm  passing the device name, IP, port details using the IP_port.txt file

file = '/users/manmahal/MANJU/IP_port.txt'
Testbed_info = IP_port(file)


#Here I'm using the  telnet object to execute some command, What ever I execute 
here on each telnet session to be logged into separate log file.

for (dname, ip, port, Telnet, fp ) in Testbed_info :
My_handle = Telnet
My_handle.Write("\n")
My_handle.Write("show config \n")
time.sleep(2)
print My_handle.ReadBuffer()

Content of the IP_port file is as below:
-
RSP 0   172.27.40.602001
RSP 1   172.27.40.602002
LC 0172.27.40.602010
LC 1172.27.40.592011


Please note that total number of the  telnet session may change as requirement 
of Testing. Hence It would be great if some give me generic solution to log the 
 log messages for each telnet session and output should to logged into 
respective log file. 
 
Thanks in advance...!!

Regards
Manju
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue25121] python logger RotatingFileHandler can't wrap log file and blows with traceback

[R. David Murray]

R. David Murray added the comment:

Vinay has closed other issues reporting this two-open-files problem in the past 
(which was why i recommended you read through some of them :).  Windows causes 
problems when there are two open file handles and a rename is attempted, so no, 
that is not a supported configuration for the logging module.

So I'm going to close it as not a bug.  If you can propose a way to make things 
work better, by all means open an enhancement issue.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger can't wrap log file and blows with traceback

[Alexander Belchenko]

New submission from Alexander Belchenko:

We're using standard logging library for logs. On machine of my colleague there 
is constantly traceback like this:

[11:21:29]  PermissionError: [WinError 32] The process cannot access 
the file because it is
being used by another process: 'C:\\Users\\Andrew\\Desktop\\server\\logs\\2015-0
9-09_10-44-03\\2015-09-09_10-44-04-middleman-684.log.1'
Logged from file middleman.py, line 379
Traceback (most recent call last):
  File "c:\python33\lib\logging\handlers.py", line 73, in emit
self.doRollover()
  File "c:\python33\lib\logging\handlers.py", line 176, in doRollover
self.rotate(self.baseFilename, dfn)
  File "c:\python33\lib\logging\handlers.py", line 116, in rotate
os.rename(source, dest)

middleman.py, line 379 is simple call to logger.debug:

self.logger.debug('node %s is already processing, packet %s postponed', 
node_id, packet_no) 

It's strange that another log file with different basename in the same logs 
directory is wrapping without problems.

Anyway, my complain is about traceback. I don't think it's good behavior that 
my application crashes because logging library can't wrap file. The problem for 
me - I have no idea how to catch and ignore such problem, because it's in the 
logging internals.

--
components: Library (Lib), Windows
messages: 250747
nosy: bialix, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: python logger can't wrap log file and blows with traceback
type: crash
versions: Python 3.3

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25121>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger can't wrap log file and blows with traceback

[Alexander Belchenko]

Alexander Belchenko added the comment:

PermissionError mentions file name 
"C:\\Users\\Andrew\\Desktop\\server\\logs\\2015-0
9-09_10-44-03\\2015-09-09_10-44-04-middleman-684.log.1" - but this file does 
not exist in log directory. There is only 
"C:\\Users\\Andrew\\Desktop\\server\\logs\\2015-0
9-09_10-44-03\\2015-09-09_10-44-04-middleman-684.log" so I think it's about 
unable to rename.

Python 3.3.5 64 bits
Windows 7 64 bits.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger RotatingFileHandler can't wrap log file and blows with traceback

[Alexander Belchenko]

Alexander Belchenko added the comment:

Based on my last assumption I'm able to reproduce this issue with simple test 
attached. If I comment out the line

  setup_logger(loggerB)

The everything works OK.

Once this line in - it's traceback.

I guess it's fair to say the bug in my code and one never should use this 
approach. On Linux it never traceback but I think it may have other unvisible 
issues.

I expect you mark this as "won't fix" but maybe you want to look at it.

--
Added file: http://bugs.python.org/file40474/test-issue25121.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger RotatingFileHandler can't wrap log file and blows with traceback

[Alexander Belchenko]

Alexander Belchenko added the comment:

I have suspicion about this issue.

In my application tornado framework is used. I setup logger for my own code, 
but use the same logger for tornado, so all messages from tornado itself go 
into the same log file.

As I said earlier it's strange but one log file works as expected. This log 
file is not used by tornado.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25121>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger RotatingFileHandler can't wrap log file and blows with traceback

[Alexander Belchenko]

Alexander Belchenko added the comment:

Update to previous comment. I use the same settings for tornado logger (e.g. 
filename).

logger = logging.getLogger('tornado')
setup_logger(logger, log_config)

So I have 2 loggers in one application which are trying to write to the same 
file. I guess it's out of supported use cases?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger RotatingFileHandler can't wrap log file and blows with traceback

[Alexander Belchenko]

Alexander Belchenko added the comment:

According to documentation on RotateFileHandler

"When this file is filled, it is closed and renamed to app.log.1, and if files 
app.log.1, app.log.2, etc. exist, then they are renamed to app.log.2, app.log.3 
etc. respectively."

But we have 2 loggers which holds open file inside the same process. And while 
one handler is trying to close and rename file, another handler (tornado, which 
writes something to log only when there are errors actually) is holding file 
open.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger can't wrap log file and blows with traceback

[R. David Murray]

R. David Murray added the comment:

Are you using a rotating file handler for the logger?  You should take a look 
at some of the other issues in this tracker about RotatingFileHandler and 
Windows.

I will be interested in Vinay's answer to your question about the exception 
being allowed to bubble up.  I suspect it has to do with the fact that you 
would be completely losing your logging data if it were ignored.

--
nosy: +r.david.murray, vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger can't wrap log file and blows with traceback

[Alexander Belchenko]

Alexander Belchenko added the comment:

Yes. it's rotating file handler.

I'll try to recreate this issue with simpler test application, so I can provide 
something as example.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25121] python logger RotatingFileHandler can't wrap log file and blows with traceback

[Alexander Belchenko]

Changes by Alexander Belchenko <bia...@ukr.net>:


--
title: python logger can't wrap log file and blows with traceback -> python 
logger RotatingFileHandler can't wrap log file and blows with traceback

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25121>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Python - parsing nested information and provide it in proper format from log file

[Peter Otten]

Jay T wrote:

  have some log file which has nested data which i want to filter and
  provide specific for student with total counts
 
 Here is my log file sample:
 Student name is ABC
 Student age is 12
 student was late
 student was late
 student was late
 Student name is DEF
 student age is 13
 student was late
 student was late
 
 i want to parse and show data as Student name, student age , number of
 counts how many times student was late e:g Name Age TotalCount
 ABC 12   3
 DEF 132
 
 Please help me with solution that will be really grateful.

What have you tried? Please show us some code.

The basic idea would be to iterate over the lines and split the current line 
into words. 

If the second word is name and it's not the first iteration print the 
student's name, age, and was_late count. Then set the name variable to the 
new name and reset age and was_late to 0. To detect the first iteration you 
can set 

name = None

before you enter the loop and then check for that value before printing:

if name is not None:
... # print student data

If the second word is age convert the 4th word to integer and set the age 
variable.

If the second word is was increment the was_late counter.

Remember that when the loop ends and the file was not empty you have one 
more student's data to print.



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python - parsing nested information and provide it in proper format from log file

[jt11380]

On Friday, February 20, 2015 at 8:11:59 AM UTC-5, Peter Otten wrote:
 Jay T wrote:
 
   have some log file which has nested data which i want to filter and
   provide specific for student with total counts
  
  Here is my log file sample:
  Student name is ABC
  Student age is 12
  student was late
  student was late
  student was late
  Student name is DEF
  student age is 13
  student was late
  student was late
  
  i want to parse and show data as Student name, student age , number of
  counts how many times student was late e:g Name Age TotalCount
  ABC 12   3
  DEF 132
  
  Please help me with solution that will be really grateful.
 
 What have you tried? Please show us some code.
 
 The basic idea would be to iterate over the lines and split the current line 
 into words. 
 
 If the second word is name and it's not the first iteration print the 
 student's name, age, and was_late count. Then set the name variable to the 
 new name and reset age and was_late to 0. To detect the first iteration you 
 can set 
 
 name = None
 
 before you enter the loop and then check for that value before printing:
 
 if name is not None:
 ... # print student data
 
 If the second word is age convert the 4th word to integer and set the age 
 variable.
 
 If the second word is was increment the was_late counter.
 
 Remember that when the loop ends and the file was not empty you have one 
 more student's data to print.

I tried to implent below code and got stucked how to do nested loop to count 
instead doing another logic and parsing:

import re
def GetName(input_string):
  myName=input_string.split()
  myName1= myName[1]
  return myName1
def GetAge(input_string):
  myAge=input_string.split()
  myAge1= myAge[2]
  return myAge1
 
  

 
file = open('mylogfile')
log_data = file.readlines()
print 'entered'
for eachline in log_data:
   input_string = eachline
   if 'name' in input_string:
sometextval = GetName(input_string)
print name, sometextval
   if 'Age' in input_string:
 sometextval2 = GetAge(input_string)
 print Age, sometextval2

Now get stuck to get count for total_late time as it is part of name, age so 
how to write logic which counts as a part of group.

any help will be grateful.

-J
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python - parsing nested information and provide it in proper format from log file

[Peter Otten]

jt11...@gmail.com wrote:

 On Friday, February 20, 2015 at 8:11:59 AM UTC-5, Peter Otten wrote:
 Jay T wrote:
 
   have some log file which has nested data which i want to filter and
   provide specific for student with total counts
  
  Here is my log file sample:
  Student name is ABC
  Student age is 12
  student was late
  student was late
  student was late
  Student name is DEF
  student age is 13
  student was late
  student was late
  
  i want to parse and show data as Student name, student age , number of
  counts how many times student was late e:g Name Age TotalCount
  ABC 12   3
  DEF 132
  
  Please help me with solution that will be really grateful.
 
 What have you tried? Please show us some code.
 
 The basic idea would be to iterate over the lines and split the current
 line into words.
 
 If the second word is name and it's not the first iteration print the
 student's name, age, and was_late count. Then set the name variable to
 the new name and reset age and was_late to 0. To detect the first
 iteration you can set
 
 name = None
 
 before you enter the loop and then check for that value before printing:
 
 if name is not None:
 ... # print student data
 
 If the second word is age convert the 4th word to integer and set the
 age variable.
 
 If the second word is was increment the was_late counter.
 
 Remember that when the loop ends and the file was not empty you have one
 more student's data to print.


 Now get stuck to get count for total_late time as it is part of name, age
 so how to write logic which counts as a part of group.
 
 any help will be grateful.

Try to write code that does what I describe in my outline. Initialise name 
before the loop and dump the collected data when you encounter a new name.

name = None
total_late = 0
age = unknown

with open(student.txt) as instream:
for line in instream:
words = line.split()
if words[1] == name:
if name is not None:
print name, age, total_late
name =  .join(words[3:])
age = unknown
total_late = 0
elif words[1] == was:
total_late += 1
elif words[1] == age:
age = int(words[3])
else:
print don't know what to do with line %r %line
if name is not None:
print name, age, total_late

Checking whole words has the advantage that there will be no match if the 
string name or age is part of the student's name.

 I tried to implent below code and got stucked how to do nested loop to
 count instead doing another logic and parsing:
 
 import re
 def GetName(input_string):
   myName=input_string.split()
   myName1= myName[1]

That's the wrong index.

   return myName1
 def GetAge(input_string):
   myAge=input_string.split()
   myAge1= myAge[2]

That's the wrong index.

   return myAge1

In general you should test your functions independently from the whole 
program. That way you can build on known-good components and thus reduce the 
area where to look for remaining bugs.
  
 file = open('mylogfile')
 log_data = file.readlines()

The file is probably short enough that it doesn't matter here, but iterating 
over the file directly is good habit to get into. Example:

with open(mylogfile) as log_data:
   for eachline in log_data:
   ...

 print 'entered'
 for eachline in log_data:
input_string = eachline
if 'name' in input_string:
 sometextval = GetName(input_string)
 print name, sometextval
if 'Age' in input_string:

This test is problematic because Python takes case into account when 
comparing strings:

 age == AGE
False
 s = RAGE
 age in s
False

If case isn't consistent you should convert the string to lowercase:

 age == AGE.lower()
True
 age in s.lower()
True

  sometextval2 = GetAge(input_string)
  print Age, sometextval2
 


-- 
https://mail.python.org/mailman/listinfo/python-list

Python - parsing nested information and provide it in proper format from log file

[Jay T]

 have some log file which has nested data which i want to filter and provide 
specific for student with total counts

Here is my log file sample: 
Student name is ABC 
Student age is 12 
student was late 
student was late 
student was late 
Student name is DEF 
student age is 13 
student was late 
student was late

i want to parse and show data as Student name, student age , number of counts 
how many times student was late e:g 
Name Age TotalCount 
ABC 12   3 
DEF 132

Please help me with solution that will be really grateful.

thanks, Jt
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Need help in pulling SQL query out of log file...

[Sagar Deshmukh]

On Monday, 13 October 2014 22:40:07 UTC-7, alex23  wrote:
 On 14/10/2014 11:47 AM, Sagar Deshmukh wrote:
 
  I have a log file which has lot of information like..SQL query.. number of 
  records read...records loaded etc..
 
  My requirement is i would like to read the SQL query completly and write it 
  to another txt file..
 
 
 
 Generally we encourage people to post what they've tried to the list. It 
 
 helps us identify what you know and what you need help with.
 
 
 
 However, given:
 
 
 
   the log file may not be always same so can not make static choices...
 
 
 
 You'll probably want to use regular expressions:
 
 
 
  https://docs.python.org/howto/regex.html
 
 
 
 Regexps let you search through the text for known patterns and extract 
 
 any that match. To extract all SQL query sections, you'll need to come 
 
 up with a way of uniquely identifying them from all other sections. 
 
 Looking at your example log file, it looks like they're all of the format:
 
 
 
  SQL Query [the actual sql query]
 
 
 
  From that we can determine that all SQL queries are prefixed by 'SQL 
 
 Query [' and suffixed by ']', so the content you want is everything 
 
 between those markers. So a possible regular expression might be:
 
 
 
  SQL Query \[(.*?)\]
 
 
 
 To quickly explain this:
 
 
 
  1. SQL Query  matches on that string
 
  2. Because [] have meaning for regexes, to match on literal 
 
 brackets you need to escape them via \[ and \]
 
  3. ( ) is a group, whats contained in here will be returned
 
  4. .* means to grab all matching text
 
  5. ? means to do an ungreedy grab ie it'll stop at the first \] 
 
 it encounters.
 
 
 
 Pulling the queries out of your log file should be as simple as:
 
 
 
  import re
 
 
 
  log = open('logfile').read()
 
  queries = re.findall(SQL Query \[(.*?)\], log, re.DOTALL)
 
 
 
 Because the queries can fall across multiple lines, the re.DOTALL flag 
 
 is required to treat EOL markers as characters.
 
 
 
 Hope this helps.

Hi,

This helps and its working... Thank you so much
-- 
https://mail.python.org/mailman/listinfo/python-list

Need help in pulling SQL query out of log file...

[Sagar Deshmukh]

Hi,

I have a log file which has lot of information like..SQL query.. number of 
records read...records loaded etc..

My requirement is i would like to read the SQL query completly and write it to 
another txt file.. also the log file may not be always same so can not make 
static choices...

my logfile is like below :

*LOg file starts**


Fri Aug 08 16:00:04 2014 : WRITER_1_*_1 WRT_8005 Writer run started.
Fri Aug 08 16:00:04 2014 : READER_1_2_1 BLKR_16007 Reader run started.
Fri Aug 08 16:00:04 2014 : WRITER_1_*_1 WRT_8158 

*START LOAD SESSION*

Load Start Time: Fri Aug 08 16:00:04 2014

Target tables:

 EIS_REQUEST_LOG_26MAYBKP
 T_delta_parm_file


Fri Aug 08 16:00:04 2014 : READER_1_2_1 RR_4010 SQ instance 
[SQ_Shortcut_to_EIS_REQUEST_LOG] SQL Query [SELECT 
EIS_REQUEST_LOG_26MAYBKP.RqstId, EIS_REQUEST_LOG_26MAYBKP.RQSTLoadStatCd FROM 
EIS_REQUEST_LOG_26MAYBKP]
Fri Aug 08 16:00:04 2014 : READER_1_2_1 RR_4049 RR_4049 SQL Query issued to 
database : (Fri Aug 08 16:00:04 2014)
Fri Aug 08 16:00:04 2014 : READER_1_2_1 RR_4035 SQL Error [
FnName: Prepare -- [Teradata][ODBC Teradata Driver][Teradata Database] Object 
'EIS_REQUEST_LOG_26MAYBKP' does not exist. ].
Fri Aug 08 16:00:04 2014 : READER_1_2_1 BLKR_16004 ERROR: Prepare failed.
Fri Aug 08 16:00:04 2014 : READER_1_1_1 RR_4029 SQ Instance [SQ_RSTS_Tables] 
User specified SQL Query [--- Approved By ICC Team---
---SELECT A.LOGSYS , 
---D./BIC/NIGNRCSYS ,
---B.ODSNAME ,
---C.ODSNAME_TECH , 
---C.PARTNO , 
---C.REQUEST ,
---E.SID 
---FROM
---sapbzd.RSISOSMAP  A, 
---sapbzd.RSTSODS  B, 
---sapbzd.RSTSODSPART   C,
---sapbzd./BIC/PNIGNRCSYS   D,
---sapbzd./BI0/SREQUID   E,
---sapbzd.RSMONMESSF
---WHERE A.OLTPSOURCE = 
('ZNK_SHP_DDLN_CREATE_BE','ZNK_KNVP2_BD','ZNK_ZVBW_RTN_ORD_ITM_BN','2LIS_02_SCL_BE','ZNK_FX_CRCY_HIS_BE','ZNK_PO_FX_CALC_LOG_BD','2LIS_12_VCHDR_BE','2LIS_02_HDR_BN','ZNK_SHP_DDLN_CHANGE_BD','1_CO_PAGL11000N1_BE','0CUSTOMER_ATTR_BE','2LIS_08TRTLP_BD','2LIS_02_SCN_BD','2LIS_02_HDR_BD','2LIS_13_VDITM_BE','0CO_OM_CCA_9_BE','ZNK_SO_BDSI_OPNDMD_BE','2LIS_11_VAHDR_BE','ZNK_ZVBW_MBEW_BE','2LIS_13_VDHDR_BN','ZNK_SHP_DDLN_CHANGE_BE','NK_ADDR_NUMBR_BN','0CUSTOMER_TEXT_BE','6DB_J_3ABD_DELTA_AFFL_AD','0MAT_PLANT_ATTR_BE','ZNK_BDCPV_BD','1_CO_PAGL11000N1_BD','2LIS_11_VASTI_BD','ZNK_ZVBW_MSKU_BE','ZNK_SHP_DDLN_CREATE_BD','0SCEM_1_BC','2LIS_11_VAHDR_BD','2LIS_11_VASCL_BD','0MATERIAL_TEXT_BE','0MATERIAL_ATTR_BE','ZNK_BDCPV_BE','2LIS_02_ITM_BN','2LIS_11_VASCL_BE','2LIS_11_VAITM_BN','NK_ADDR_NUMBR_BE','2LIS_08TRTK_BE','ZNK_SD_LIKPPS_BN','2LIS_03_BF_BE','ZNK_SO_BDBS_ALLOC_BD','ZNK_TD_3AVASSO_BN','0EC_PCA_3_BD','ZNK_TD_3AVAP_BE','2LIS_11_VAITM_BE','0CUST_SALES_ATTR_BN','0EC_PCA_3_
 
BE','2LIS_13_VDITM_BN','2LIS_11_VASTH_BD','2LIS_13_VDITM_BD','0CUST_SALES_ATTR_BD','ZNK_TD_3AVASSO_BD','2LIS_02_SCN_BE','2LIS_08TRTS_BD','0CUSTOMER_ATTR_BN','ZNK_TD_3AVASSO_BE','ZNK_ZVBW_MSLB_BE','ZNK_TD_3AVAP_BD','0CUSTOMER_TEXT_BN','6DB_J_3ABD_DELTA_US_AD','0CUSTOMER_TEXT_BD','2LIS_11_VAHDR_BN','ZNK_SO_BDBS_ALLOC_BN','0GL_ACCOUNT_TEXT_BE','0GL_ACCOUNT_TEXT_BD','2LIS_11_VAITM_BD','ZNK_TD_3AVATL_BE','ZNK_SO_BDBS_ALLOC_BE','ZNK_EBAN_BE','ZNK_SO_BDSI_OPNDMD_BN','ZNK_SD_LIKPPS_BD','ZNK_ZVBW_RTN_ORD_ITM_BE','2LIS_08TRTS_BN','2LIS_02_HDR_BE','ZNK_TD_3AVATL_BD','ZNK_VBPA_BE','ZNK_FX_CRCY_HIS_BD','2LIS_13_VDHDR_BE','NK_ADDR_NUMBR_BD','2LIS_12_VCITM_BD','2LIS_08TRTK_BD','2LIS_11_VASCL_BN','ZNK_ZVBW_MCHB_BE','6DB_J_3ABD_SCL_DELTA_AP_AE','ZNK_SO_BDSI_OPNDMD_BD','ZNK_KNVP2_BE','0MAT_SALES_ATTR_BE','ZNK_TD_3AVAP_BN','2LIS_13_VDHDR_BD','0GL_ACCOUNT_ATTR_BD','2LIS_02_SCL_BD','ZNK_VBPA_BD','2LIS_02_ITM_BD','ZNK_TD_3AVATL_BN','ZNK_ZVBW_RTN_ORD_ITM_BD','ZNK_PO_FX_CALC_LOG_BE','6DB_J_3ABD_DELTA_EMEA_
 AD','0GL_ACCOUNT_ATTR_BE','2LIS_03_BF_BD','2!
 LIS_11_V
ASTI_BE','0CO_OM_CCA_9_BD','0CUST_SALES_ATTR_BE','2LIS_12_VCITM_BE','0CUSTOMER_ATTR_BD','2LIS_02_ITM_BE','2LIS_08TRTLP_BE','2LIS_12_VCHDR_BD','ZNK_EBAN_BD','2LIS_08TRTS_BE','2LIS_02_SCL_BN','2LIS_11_VASTH_BE','ZNK_SD_LIKPPS_BE')
---AND A.LOGSYS = D./BIC/NKLOGSYST
---AND A.OBJVERS = 'A'
---AND A.TRANSTRU =  B.ODSNAME 
---AND B.DATETO = 0101
---AND B.OBJSTAT = 'ACT'
---AND B.ODSNAME_TECH = C.ODSNAME_TECH
---AND C.DELFLAG  'X'
---AND D./BIC/NIGNRCSYS = ('R3_PRA','R3_PRD','R3_PRF','EM_EMP')
---AND D.OBJVERS = 'A'
---AND E.REQUID = C.REQUEST
---AND F.RNR = C.REQUEST
---AND F.MSGNO = '344'
---AND F.AUFRUFER = '09'---

SELECT A.LOGSYS , 
D./BIC/NIGNRCSYS ,
B.ODSNAME ,
C.ODSNAME_TECH , 
C.PARTNO , 
C.REQUEST ,
E.SID 
FROM
sapbzd.RSISOSMAP  A, 
sapbzd.RSTSODS  B, 
sapbzd.RSTSODSPART   C,
sapbzd./BIC/PNIGNRCSYS   D,
sapbzd./BI0/SREQUID   E,
sapbzd.RSMONMESSF
WHERE 
A.OLTPSOURCE in 
('ZNK_SHP_DDLN_CREATE_BE','ZNK_KNVP2_BD','ZNK_ZVBW_RTN_ORD_ITM_BN','2LIS_02_SCL_BE','ZNK_FX_CRCY_HIS_BE','ZNK_PO_FX_CALC_LOG_BD','2LIS_12_VCHDR_BE','2LIS_02_HDR_BN','ZNK_SHP_DDLN_CHANGE_BD','1_CO_PAGL11000N1_BE','0CUSTOMER_ATTR_BE','2LIS_08TRTLP_BD','2LIS_02_SCN_BD','2LIS_02_HDR_BD','2LIS_13_VDITM_BE','0CO_OM_CCA_9_BE

Re: Need help in pulling SQL query out of log file...

[alex23]

On 14/10/2014 11:47 AM, Sagar Deshmukh wrote:

I have a log file which has lot of information like..SQL query.. number of 
records read...records loaded etc..
My requirement is i would like to read the SQL query completly and write it to 
another txt file..


Generally we encourage people to post what they've tried to the list. It 
helps us identify what you know and what you need help with.


However, given:

 the log file may not be always same so can not make static choices...

You'll probably want to use regular expressions:

https://docs.python.org/howto/regex.html

Regexps let you search through the text for known patterns and extract 
any that match. To extract all SQL query sections, you'll need to come 
up with a way of uniquely identifying them from all other sections. 
Looking at your example log file, it looks like they're all of the format:


SQL Query [the actual sql query]

From that we can determine that all SQL queries are prefixed by 'SQL 
Query [' and suffixed by ']', so the content you want is everything 
between those markers. So a possible regular expression might be:


SQL Query \[(.*?)\]

To quickly explain this:

1. SQL Query  matches on that string
2. Because [] have meaning for regexes, to match on literal 
brackets you need to escape them via \[ and \]

3. ( ) is a group, whats contained in here will be returned
4. .* means to grab all matching text
5. ? means to do an ungreedy grab ie it'll stop at the first \] 
it encounters.


Pulling the queries out of your log file should be as simple as:

import re

log = open('logfile').read()
queries = re.findall(SQL Query \[(.*?)\], log, re.DOTALL)

Because the queries can fall across multiple lines, the re.DOTALL flag 
is required to treat EOL markers as characters.


Hope this helps.
--
https://mail.python.org/mailman/listinfo/python-list

RE: creating log file with Python logging module

[Arulnambi Nandagoban]

-Message d'origine-
De : Python-list
[mailto:python-list-bounces+a.nandagoban=traxens@python.org] De la part
de Peter Otten
Envoyé : Monday, August 4, 2014 4:03 PM
À : python-list@python.org
Objet : Re: creating log file with Python logging module

Peter Otten wrote:

 Peter Otten wrote:
 
 You won't see a rollover if you restart it.
 
 Sorry, I tried it and the above statement is wrong.

[Arulnambi Nandagoban]

 logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - 
 %(message)s', datefmt='%a, %d %b %Y %H:%M:%S', level = logging.DEBUG, 
 filename=Ffilename, filemode='w')
 
 logger = logging.getLogger(__name__)
 
 hdlr = TimedRotatingFileHandler(Ffilename, when='midnight')

My alternative theory about what might be going wrong: you are using the
same file in logging.basicConfig() and the TimedRotatingFileHandler.

But I cannot replicate the problem on my (linux) system.

--
https://mail.python.org/mailman/listinfo/python-list


Hello,

Thank you  for response !!!

-- Config.py:

import logging
from logging.handlers import TimedRotatingFileHandler
import os
import time

logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s -
%(message)s', datefmt='%a, %d %b %Y %H:%M:%S')
logHandler = TimedRotatingFileHandler(Ffilename,when=D)
logFormatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s -
%(message)s', datefmt='%a, %d %b %Y %H:%M:%S')
logHandler.setFormatter( logFormatter )
logger = logging.getLogger(__name__ )
logger.addHandler( logHandler )
logger.setLevel( logging.DEBUG )


The above code works according to my need that is, 
- print log in console as well as save log in a file every day

By adding logging.basicConfig, log is printed in both console and saved in
log file.

--
nambi





-- 
https://mail.python.org/mailman/listinfo/python-list

creating log file with Python logging module

[Arulnambi Nandagoban]

Hello all,

 

I am using logging module for my application to log all debug information. I
configured it create a new log file every day with

TimedRotatingFileHandler.  I display debug message in console as well.
But I didn't see creation of new file. Can someone help me to sort out this
problem. Following is config code:

 

import logging

from logging.handlers import TimedRotatingFileHandler

 

Ffilename = os.path.join(dir_path, 'Pyserverlog')

logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s -
%(message)s', datefmt='%a, %d %b %Y %H:%M:%S', level = logging.DEBUG,
filename=Ffilename, filemode='w')

logger = logging.getLogger(__name__)

hdlr = TimedRotatingFileHandler(Ffilename, when='midnight')

logger.addHandler(hdlr)

# define a Handler which writes INFO messages or higher to the sys.stderr

console = logging.StreamHandler()

console.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s -
%(message)s')

# tell the handler to use this format

console.setFormatter(formatter)

# add the handler to the root logger

logging.getLogger('').addHandler(console)

 

--

nambi

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: creating log file with Python logging module

[Peter Otten]

Arulnambi Nandagoban wrote:

 I am using logging module for my application to log all debug information.
 I configured it create a new log file every day with
 
 TimedRotatingFileHandler.  I display debug message in console as well.
 But I didn't see creation of new file. 

Is the script running continuously? You won't see a rollover if you restart 
it. Working example (with shorter time interval):

$ cat rollover.py 
import logging
import logging.handlers
import time

logger = logging.getLogger()

handler = logging.handlers.TimedRotatingFileHandler(logfile, when='S')

logger.addHandler(handler)
logger.setLevel(logging.INFO)

for i in range(100):
logger.info(message #%s % i)
time.sleep(.1)

$ ls
rollover.py
$ python rollover.py 
$ ls
logfile  logfile.2014-08-04_15-21-26
logfile.2014-08-04_15-21-21  logfile.2014-08-04_15-21-27
logfile.2014-08-04_15-21-22  logfile.2014-08-04_15-21-28
logfile.2014-08-04_15-21-23  logfile.2014-08-04_15-21-29
logfile.2014-08-04_15-21-24  logfile.2014-08-04_15-21-30
logfile.2014-08-04_15-21-25  rollover.py
$ cat logfile
message #93
message #94
message #95
message #96
message #97
message #98
message #99
$ 


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: creating log file with Python logging module

[Peter Otten]

Peter Otten wrote:

 You won't see a rollover if you restart it.

Sorry, I tried it and the above statement is wrong.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: creating log file with Python logging module

[Peter Otten]

Peter Otten wrote:

 Peter Otten wrote:
 
 You won't see a rollover if you restart it.
 
 Sorry, I tried it and the above statement is wrong.

[Arulnambi Nandagoban]

 logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s -
 %(message)s', datefmt='%a, %d %b %Y %H:%M:%S', level = logging.DEBUG,
 filename=Ffilename, filemode='w')
 
 logger = logging.getLogger(__name__)
 
 hdlr = TimedRotatingFileHandler(Ffilename, when='midnight')

My alternative theory about what might be going wrong: you are using the 
same file in logging.basicConfig() and the TimedRotatingFileHandler.

But I cannot replicate the problem on my (linux) system.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)

[Mark H Harris]

On 3/25/14 6:38 PM, Dennis Lee Bieber wrote:



A couple of us managed to steal the school login/password (don't
think we ever used it, but...)... The teaching assistant didn't notice the
paper tape punch was active when persuaded to login to let us run a short
program (high school BASIC class, with a dial-up teletype). Playing back
the tape and manually spinning the platen during the password
obscuration/input phase gave us the plain text.



I still have one of my old BASIC tapes from way back in the day; I 
wanted to get the code back, or just remember why I had saved the tape?


One of my linux user group buddies locally rigged up an optical reader 
(IF) to a single board micro controller... we pulled the tape by hand 
using the center drive holes (sprocket holes) as a strobe and after a 
couple of false attempts read the entire tape into a text file.


That tape still have the caster oil smell of the tty that produced it; 
smell has a very strong memory association /  I can still hear that 
thing running in my head.  ... haven't seen one physically in years.


marcus

--
https://mail.python.org/mailman/listinfo/python-list

Re: Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)

[Mark H Harris]

On 3/24/14 6:30 PM, Dennis Lee Bieber wrote:

{And I recall standard practice was to hit \r, to return the carriage, \n
for next line, and one RUBOUT to provide a delay while the carriage
returned to the leftG}


Yes, yes... I remember well, there had to be a delay (of some type) to 
wait for the horse and carriage to get from the right side of the field 
to the left.  Aaah, the good 'ol days.


marcus
--
https://mail.python.org/mailman/listinfo/python-list

Re: Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)

[Steven D'Aprano]

On Sun, 23 Mar 2014 12:37:43 +1100, Chris Angelico wrote:

 On Sun, Mar 23, 2014 at 12:07 PM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
 On Sun, 23 Mar 2014 02:09:20 +1100, Chris Angelico wrote:

 On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
 Line endings are terminators: they end the line. Whether you consider
 the terminator part of the line or not is a matter of opinion (is the
 cover of a book part of the book?) but consider this:

 If you say that the end of lines are *not* part of the line, then
 that implies that some parts of the file are not inside any line
 at all. And that would be just weird.

 Not so weird IMO. A file is not a concatenation of lines; it is a
 stream of bytes.

 But a *text file* is a concatenation of lines. The text file model is
 important enough that nearly all programming languages offer a
 line-based interface to files, and some (Python at least, possibly
 others) make it the default interface so that iterating over the file
 gives you lines rather than bytes -- even in binary mode.
 
 And lines are delimited entities. A text file is a sequence of lines,
 separated by certain characters.

Are they really separated, or are they terminated?

a\nb\n

Three lines or two? If you say three, then you consider \n to be a 
separator; if you say two, you consider it a terminator.

The thing is, both points of view are valid. If \n is a terminator, then 
the above is valid text, but this may not be:

a\nb\nc

since the last line is unterminated. (You might be generous and allow 
that every line must be terminated except possibly the last. Or you might 
be strict and consider the last line to be broken.)

In practice, most people swap between one point of view and the other 
without warning: I might say that a\nb\n has two lines terminated with 
\n, and then an instant later say that the file ends with a blank line, 
which means it has three lines, not two. Or you might say that a\nb\n 
has three lines separated by \n, and an instant later claim that the last 
line contains the letter b. So common language about text files tends 
to be inconsistent and flip-flop between the two points of view, a bit 
like the Necker Cube optical illusion.

Given that the two points of view are legitimate and useful, how should a 
programming language treat lines? If the language treats the newline as 
separator, and strips it, then those who want to treat it as terminator 
are screwed -- you cannot tell if the last line is terminated or not. But 
if the language treats the newline as a terminator, and so part of the 
line, it is easy for the caller to remove it. The decision ought to be a 
no-brainer: keep the newline in place, let the user strip it if they 
don't want it.

Here's another thought for you: words are separated by spaces. Nobody 
ever considers the space to be part of the word[1]. I think that nearly 
everyone agrees that both spam eggs and spam  eggs contain two 
words, spam and eggs. I don't think anyone would say that the second 
example includes seven words, five of which are blank. Would we like to 
say that spam\n\n\n\n\n\neggs contains two lines rather than seven?


 
 (Both interpretations make sense. I just wish the most obvious form of
 iteration gave the cleaner/tidier version, or at very least that there
 be some really obvious way to ask for lines-without-endings.)

 There is: call strip('\n') on the line after reading it. Perl and Ruby
 spell it chomp(). Other languages may spell it differently. I don't
 know of any language that automatically strips newlines, probably
 because you can easily strip the newline from the line, but if the
 language did it for you, you cannot reliably reverse it.
 
 That's not a tidy way to iterate, that's a way to iterate and then do
 stuff. Compare:
 
 for line in f:
 # process line with newline
 
 for line in f:
 line = line.strip(\n)
 # process line without newline, as long as it doesn't have \r\n or
 something

With universal newline support, you can completely ignore the difference 
in platform-specific end-of-line markers. By default, Python will convert 
them to and from \n when you read or write a text file, and you'll never 
see any difference. Just program using \n in your source code, and let 
Python do the right thing. (If you need to handle end of line markers 
yourself, you can easily disable universal newline support.)

f = (line.rstrip('\n') for line in f)
for line in f:
# process line

Everything[1] in computer science can be solved by an additional layer of 
indirection :-)


[...]
 So why is the delimiter excluded when you treat the file as CSV, but
 included when you treat the file as lines of text?

Because reading lines of text is more general than reading CSV records. 
Therefore it has to make fewer modifications to the raw content.

I once had a Pascal compiler that would insert spaces, indentation, even 

Re: Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)

[Chris Angelico]

On Mon, Mar 24, 2014 at 1:37 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 On Sun, 23 Mar 2014 12:37:43 +1100, Chris Angelico wrote:
 And lines are delimited entities. A text file is a sequence of lines,
 separated by certain characters.

 Are they really separated, or are they terminated?

 a\nb\n

 Three lines or two? If you say three, then you consider \n to be a
 separator; if you say two, you consider it a terminator.

 The thing is, both points of view are valid. If \n is a terminator, then
 the above is valid text, but this may not be:

 a\nb\nc

 since the last line is unterminated. (You might be generous and allow
 that every line must be terminated except possibly the last. Or you might
 be strict and consider the last line to be broken.)

It is a problem, and the correct usage depends on context.

I'd normally say that the first consists of two lines, the first being
a and the second being b, and there is no third blank line. The
first line still doesn't consist of a\n, though. It's more like how
environment variables are provided to a C program: separated by \0 and
the last one has to be terminated too.

In some situations, you would completely ignore the c in the last
example. When you're watching a growing log file, buffering might mean
that you see half of a line. When you're reading MUD text from a
socket, a partial line probably means it's broken across two packets,
and the rest of the line is coming. Either way, you don't process the
c in case it's the beginning of a line; you wait till you see the
\n separator that says that you now have a complete line. Got some
out-of-band indication that there won't be any more (like an EOF
signal)? Assume that c is the whole line, or assume the file is
damaged, and proceed accordingly.

 Given that the two points of view are legitimate and useful, how should a
 programming language treat lines? If the language treats the newline as
 separator, and strips it, then those who want to treat it as terminator
 are screwed -- you cannot tell if the last line is terminated or not.

That's my point, though. If you want to treat a file as lines, you
usually won't care whether the last one is terminated or not. You'll
have some means of defining lines, which might mean discarding the
last, or whatever it is, but the stream a\nb\nc will either become
[a, b, c] or [a, b] or ValueError or something, and that
list of lines is really all you care about. Universal newlines, as you
mention, means that a\r\nb\r\n will become the exact same thing as
a\nb\n, and there's no way to recreate that difference - because it
*does not matter*.

 Here's another thought for you: words are separated by spaces. Nobody
 ever considers the space to be part of the word[1]. I think that nearly
 everyone agrees that both spam eggs and spam  eggs contain two
 words, spam and eggs. I don't think anyone would say that the second
 example includes seven words, five of which are blank. Would we like to
 say that spam\n\n\n\n\n\neggs contains two lines rather than seven?

Ahh, that's a tricky one. For the simple concept of iterating over the
lines in a file, I would have to say that it's seven lines, five of
which are blank, same as spam  eggs.split( ) returns a
seven-element list. The tricky bit is that the term word means
*non-empty* sequence of characters, which means that after splitting
on spaces, you discard all empty tokens in the list; but normally
line does NOT have that non-empty qualifier. However, a double
newline often means paragraph break as opposed to line break, so
there's additional meaning applied there; that might be four
paragraphs, the last one unterminated (and a paragraph might well be
terminated by a single newline rather than two), and in some cases
might be squished to just two paragraphs because the paragraph itself
is required to be non-empty.

 With universal newline support, you can completely ignore the difference
 in platform-specific end-of-line markers. By default, Python will convert
 them to and from \n when you read or write a text file, and you'll never
 see any difference. Just program using \n in your source code, and let
 Python do the right thing. (If you need to handle end of line markers
 yourself, you can easily disable universal newline support.)

So why should we have to explicitly disable universal newlines to undo
the folding of \r\n and \n down to a single end of line indication,
but automatically get handling of \n or absence at the end of the
file? Surely that's parallel. In each case, you're taking the set of
lines as your important content, and folding together distinctions
that don't matter.

 I once had a Pascal compiler that would insert spaces, indentation, even
 change the case of words. Regardless of what you actually typed, it would
 pretty-print your code, then write the pretty-printed output when you
 saved. Likewise, if you read in a Pascal source file from an external
 editor, then saved

Re: Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)

[Mark H Harris]

On 3/23/14 10:17 PM, Chris Angelico wrote:

Newline style IS relevant. You're saying that this will copy a file perfectly:

out = open(out, w)
for line in open(in):
 out.write(line)

but it wouldn't if the iteration and write stripped and recreated
newlines? Incorrect, because this version will collapse \r\n into \n.
It's still a *text file copy*. (And yes, I know about 'with'. Shut
up.) It's idempotent, not byte-for-byte perfect.


Which was my point in the first place about new-line standards. We all 
know why its important to collapse \r\n into \n,  but why(?) in a 
general way would this be the universal desired end? (rhetorical)  Your 
example of byte-for-byte perfect copy is one good case (they are not). 
Another might be controller code (maybe ancient) where the \r is 
'required' and collapsing it to \n won't work on the device (tty, or 
other).


There does need to be a text file standard where what is desired is a 
file of lines. Iterating over the file object should return the 
lines on any system platform, without the user being required to strip 
off the line-end (newline  \n) delimiter U+000a. The delimiter does not 
matter.


What python has done by collapsing the \r\n into \n is to hide the real 
problem (non standard delimiters between platforms) and in the process 
actually 'removes' possibly important information  (\r). {lossy}


We don't really use real tty devices any longer which require one code 
to bring the print head carriage back (\r) and one code to index the 
paper platten (\n).  Screen I/O doesn't work that way any longer either. 
Its time to standardize the newline and/or file text line end delimiters.


marcus
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[wxjmfauth]

Le samedi 22 mars 2014 05:59:34 UTC+1, Mark H. Harris a écrit :
 On 3/21/14 11:46 PM, Chris Angelico wrote:
 
  (Side point: You have your 0d and your 0a backwards; the Unix line
 
  ending is U+000A, and the Windows default is U+000D U+000A.)
 
 
 
 Yeah, I know... smart apple.
 
 
 
  How are you going to make people change? What are you going to make
 
  them change to? Who controls this standard, and how do you convince
 
  all OSes to comply with it?
 
 
 
 Well, we're already doing this to some extent; baby steps.  Well, we 
 
 have open document standards (evolving) and we have a really good sense 
 
 for unicode (and python is being a genuine leader there) and the 
 
 flat-file is just another open document (very simple no doubt), not 
 
 different from a standards viewpoint than rft, odt, {whatever}; txt?
 
 
 
 My idea is that as we are morphing open document standards we need 
 
 to keep the flat-file in mind too.  The ASCII ship has sailed too. 
 
 Unicode is in, ASCII is out (for all intents and purposes) except at 
 
 Microsoft---and its time to rethink what a flat unicode text file 
 
 really is. That's all.
 
 

No offense. A good start would be to understand unicode
instead of bashing MS.

jmf
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Steven D'Aprano]

On Sat, 22 Mar 2014 01:24:33 -0400, Terry Reedy wrote:

 On 3/22/2014 12:30 AM, Mark H Harris wrote:
 On 3/21/14 11:15 PM, Chris Angelico wrote:
 It compounds. One reply makes for double spacing... two makes
 quadruple, three means we have seven wasted lines between every pair
 of real lines. That gets pretty annoying. And considering that most
 people who reply without cleaning up the lines also keep the entire
 quoted text (and usually top-post as well), this gets big fast.
 
 Before Mark started asking people adjust to the foibles of gg, we used
 to get such posts. I refused to read them. I have not seen one lately,

Luck you. I see them quite frequently.


 say maybe his nudging has had some positive effect.
 
 Yes, I can see that readily/  I get it, it just seems that fixing
 it
 at the source (gg) is the answer;
 
 I completely agree. However, Google seems immune to suggestions,
 including requests that it try to stop being a major source of spam
 posts.

Remember, we are not Google's customers. We are Google's product. The 
customers are the advertisers.

[...]
 If I were in charge of the software used for this list, I would replace
 Mark with a custom addition to return mis-formated posts (more blank
 lines than not) with instructions on how to fix them. But I am not.

Wouldn't it be less obnoxious and more useful to pass the posts through a 
filter that deletes the annoying blank lines?



-- 
Steven D'Aprano
http://import-that.dreamwidth.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Terry Reedy]

On 3/22/2014 5:50 AM, Steven D'Aprano wrote:

On Sat, 22 Mar 2014 01:24:33 -0400, Terry Reedy wrote:



If I were in charge of the software used for this list, I would replace
Mark with a custom addition to return mis-formated posts (more blank
lines than not) with instructions on how to fix them. But I am not.


Wouldn't it be less obnoxious and more useful to pass the posts through a
filter that deletes the annoying blank lines?


I have thought of that too, and may have suggested it. It would be 
slightly harder as a decision would be required as to which to delete.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Mark Lawrence]

On 22/03/2014 03:58, Mark H Harris wrote:

On 3/21/14 5:44 PM, Mark Lawrence wrote:

I'm pleased to see that you have answers. In return would you either use
the mailing list https://mail.python.org/mailman/listinfo/python-list or
read and action this https://wiki.python.org/moin/GoogleGroupsPython to
prevent us seeing double line spacing and single line paragraphs, thanks.


I perceive that this is your singular pet peeve, or, you were elected by
the python community some time ago to police the line-end problem ?



It's a pet peeve as:-

a) trying to read something that's the fourth level of reply or higher 
to the original gets to be almost impossible as it's all white space and 
no substance.


b) better tools exist

c) the work around is shown on the Python wiki, not on the crappy, bug 
ridden gg site itself.


Wow, it's like a sauna in here :)

I doubt that the Python community would elect me to do anything.  Anyhow 
I start my new job in the diplomatic corp next week.


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Mark Lawrence]

On 22/03/2014 08:54, wxjmfa...@gmail.com wrote:

Le samedi 22 mars 2014 05:59:34 UTC+1, Mark H. Harris a écrit :

On 3/21/14 11:46 PM, Chris Angelico wrote:


(Side point: You have your 0d and your 0a backwards; the Unix line



ending is U+000A, and the Windows default is U+000D U+000A.)




 Yeah, I know... smart apple.




How are you going to make people change? What are you going to make



them change to? Who controls this standard, and how do you convince



all OSes to comply with it?




 Well, we're already doing this to some extent; baby steps.  Well, we

have open document standards (evolving) and we have a really good sense

for unicode (and python is being a genuine leader there) and the

flat-file is just another open document (very simple no doubt), not

different from a standards viewpoint than rft, odt, {whatever}; txt?



 My idea is that as we are morphing open document standards we need

to keep the flat-file in mind too.  The ASCII ship has sailed too.

Unicode is in, ASCII is out (for all intents and purposes) except at

Microsoft---and its time to rethink what a flat unicode text file

really is. That's all.




No offense. A good start would be to understand unicode
instead of bashing MS.

jmf



How apt given how this thread has moved :)

--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[cool-RR]

On Saturday, March 22, 2014 3:39:21 AM UTC+2, Terry Reedy wrote:
 Does your .b2 install work? Can you delete it thru the programs list?

I uninstalled it before this entire adventure.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Steven D'Aprano]

On Fri, 21 Mar 2014 22:58:37 -0500, Mark H Harris wrote:

 I notice (since moving my stuff to Thunderbird two weeks back) the
 double spacing you keep squawking about, but I don't find it the big
 nuisance you're talking about; ok, so we have to scroll a bit further.

It's not the scrolling that interferes

with readability, it's the interruption

to the flow of text by having excess 

blank lines within a paragraph of text.



 I am honestly convinced that this might even be a python problem.  More
 likely than not, gg is written in python, and this is the goofy line-end
 character problem we have to deal with when we read lines in python.

Well, that's certainly a novel idea. Why you think that Google Groups is 
written in Python? Does every post from GG end with Powered By Python?


 Why do we suck in the new-line character as though it were part of the
 line?  

Because it is the only sensible way to handle it. If you don't, then 
there is no way to distinguish between a file that ends with a newline 
and one which doesn't.


 This is asinine behavior.  The new-line is a file delimiter
 character and NOT intended to be part of the line.

Line endings are terminators: they end the line. Whether you consider the 
terminator part of the line or not is a matter of opinion (is the cover 
of a book part of the book?) but consider this:

If you say that the end of lines are *not* part of the line, then 
that implies that some parts of the file are not inside any line 
at all. And that would be just weird.


 Thinking this through a bit

Yes, that helps :-)


 I've noticed that a blank line comes back
 with a '\n'  which differentiates it from file end which comes back
 without the new-line.  So, it appears that python is using the
 new-line character (or lack there-of) to have meaning which the new=line
 in a unix file was never suppose to carry.

I don't understand what meaning you are referring to here. Blank lines 
comes back as a \n because a blank line *is* a \n with nothing before it. 
Python isn't doing anything funny here, at least not on Linux. If you 
open a text editor, and type:

spam ENTER ENTER ENTER ENTER eggs ENTER

(where ENTER means to hit the Enter key, not the letters E N T E R) and 
then save, your editor will show the words spam and eggs separated by 
three blank lines. If you open that file in a hex editor, you will see 
something like:

73 70 61 6d 0a 0a 0a 0a  65 67 67 73 0a

Notice the four 0a bytes in a row? That gives you three blank lines. 
Python is no adding any extra newlines or putting them where they aren't, 
so I don't really understand what point you're trying to make here.


 
 If python would just return EOF like every other language at file end,
 and a test line without '\n' tacked to the end of it, this little snag
 with gg would probably go away.  What say you?

There is no evidence that Google Group's difficulty is because of Python. 
More likely it is related to the translation between rich text 
formatted using HTML, and plain text.

By the way, Python *does* return EOF at the end of the file. It is just 
that EOF in Python is spelled the empty string instead of some other 
special value. Both Ruby and Lua behave like Python, returning the empty 
string on end of file:

steve@orac:~$ irb
irb(main):001:0 f = File.open('/tmp/io.txt')
= #File:/tmp/io.txt
irb(main):002:0 f.read()
= hello
irb(main):003:0 f.read()
= 


[steve@ando ~]$ lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
 fp = io.open('/tmp/io.txt', 'r')
 print(fp:read(*all))
hello
 print(fp:read(*all))




Similarly, Rust returns None:
http://static.rust-lang.org/doc/0.9/std/io/trait.Reader.html#tymethod.read

And Java's java.io.BufferedReader is also similar, returning null.




-- 
Steven D'Aprano
http://import-that.dreamwidth.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Chris Angelico]

On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 Line endings are terminators: they end the line. Whether you consider the
 terminator part of the line or not is a matter of opinion (is the cover
 of a book part of the book?) but consider this:

 If you say that the end of lines are *not* part of the line, then
 that implies that some parts of the file are not inside any line
 at all. And that would be just weird.

Not so weird IMO. A file is not a concatenation of lines; it is a
stream of bytes. Now, if you ask Python to read you 512 bytes from a
binary file, and then ask for another 512 bytes, and so on until you
reach the end, then it would indeed be VERY weird if there were parts
of the file that weren't in the returned (byte) strings. But if you
ask for a line, and then another line, and another line, then it's
quite reasonable to interpret U+000A as line separation rather than
line termination, and not return it. (Both interpretations make
sense. I just wish the most obvious form of iteration gave the
cleaner/tidier version, or at very least that there be some really
obvious way to ask for lines-without-endings.) Imagine the output of
GNU find as a series of records. You can ask for those to be separated
by newlines (the default, or -print), or by NULs (with the -print0
command). In either case, the records do not *contain* that value,
they're separated by it; the records consist of file names.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Steven D'Aprano]

On Sun, 23 Mar 2014 02:09:20 +1100, Chris Angelico wrote:

 On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
 Line endings are terminators: they end the line. Whether you consider
 the terminator part of the line or not is a matter of opinion (is the
 cover of a book part of the book?) but consider this:

 If you say that the end of lines are *not* part of the line, then
 that implies that some parts of the file are not inside any line at
 all. And that would be just weird.
 
 Not so weird IMO. A file is not a concatenation of lines; it is a stream
 of bytes. 

But a *text file* is a concatenation of lines. The text file model is 
important enough that nearly all programming languages offer a line-based 
interface to files, and some (Python at least, possibly others) make it 
the default interface so that iterating over the file gives you lines 
rather than bytes -- even in binary mode.


 Now, if you ask Python to read you 512 bytes from a binary
 file, and then ask for another 512 bytes, and so on until you reach the
 end, then it would indeed be VERY weird if there were parts of the file
 that weren't in the returned (byte) strings. But if you ask for a line,
 and then another line, and another line, then it's quite reasonable to
 interpret U+000A as line separation rather than line termination,
 and not return it. (Both interpretations make sense. I just wish the
 most obvious form of iteration gave the cleaner/tidier version, or at
 very least that there be some really obvious way to ask for
 lines-without-endings.)

There is: call strip('\n') on the line after reading it. Perl and Ruby 
spell it chomp(). Other languages may spell it differently. I don't know 
of any language that automatically strips newlines, probably because you 
can easily strip the newline from the line, but if the language did it 
for you, you cannot reliably reverse it.


 Imagine the output of GNU find as a series of
 records. You can ask for those to be separated by newlines (the default,
 or -print), or by NULs (with the -print0 command). In either case, the
 records do not *contain* that value, they're separated by it; the
 records consist of file names.

I have no problem with that: when interpreting text as a record with 
delimiters, e.g. from a CSV file, you normally exclude the delimiter. 
Sometimes the line terminator does double-duty as a record delimiter as 
well.

Reading from a file is considered a low-level operation. Reading 
individual bytes in binary mode is the lowest level; reading lines in 
text mode is the next level, built on top of the lower binary mode. You 
build higher protocols on top of one or the other of that mode, e.g. 
read a zip file would be built on top of binary mode, read a csv file 
would be built on top of text mode.

As a low-level protocol, you ought to be able to copy a file without 
changing it by reading it in then writing it out:

for blob in infile:
outfile.write(blob)


ought to work whether you are in text mode or binary mode, so long as the 
infile and outfile are opened in the same mode. If Python were to strip 
newlines, that would no longer be the case.

(Even high-level protocols should avoid unnecessary modifications to 
files. One of the more annoying, if not crippling, limitations to the 
configparser module is that reading an INI file in, then writing it out 
again destroys the high-level structure of the file: comments and blank 
lines are stripped, and records may be re-ordered.)



-- 
Steven D'Aprano
http://import-that.dreamwidth.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)

[Chris Angelico]

On Sun, Mar 23, 2014 at 12:07 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
 On Sun, 23 Mar 2014 02:09:20 +1100, Chris Angelico wrote:

 On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
 Line endings are terminators: they end the line. Whether you consider
 the terminator part of the line or not is a matter of opinion (is the
 cover of a book part of the book?) but consider this:

 If you say that the end of lines are *not* part of the line, then
 that implies that some parts of the file are not inside any line at
 all. And that would be just weird.

 Not so weird IMO. A file is not a concatenation of lines; it is a stream
 of bytes.

 But a *text file* is a concatenation of lines. The text file model is
 important enough that nearly all programming languages offer a line-based
 interface to files, and some (Python at least, possibly others) make it
 the default interface so that iterating over the file gives you lines
 rather than bytes -- even in binary mode.

And lines are delimited entities. A text file is a sequence of lines,
separated by certain characters.

 (Both interpretations make sense. I just wish the
 most obvious form of iteration gave the cleaner/tidier version, or at
 very least that there be some really obvious way to ask for
 lines-without-endings.)

 There is: call strip('\n') on the line after reading it. Perl and Ruby
 spell it chomp(). Other languages may spell it differently. I don't know
 of any language that automatically strips newlines, probably because you
 can easily strip the newline from the line, but if the language did it
 for you, you cannot reliably reverse it.

That's not a tidy way to iterate, that's a way to iterate and then do
stuff. Compare:

for line in f:
# process line with newline

for line in f:
line = line.strip(\n)
# process line without newline, as long as it doesn't have \r\n or something

for line in f:
line = line.split($)
# process line as a series of dollar-delimited fields

The second one is more like the third than the first. Python does not
offer a tidy way to do the common thing, which is reading the content
of the line without its terminator.

 Imagine the output of GNU find as a series of
 records. You can ask for those to be separated by newlines (the default,
 or -print), or by NULs (with the -print0 command). In either case, the
 records do not *contain* that value, they're separated by it; the
 records consist of file names.

 I have no problem with that: when interpreting text as a record with
 delimiters, e.g. from a CSV file, you normally exclude the delimiter.
 Sometimes the line terminator does double-duty as a record delimiter as
 well.

So why is the delimiter excluded when you treat the file as CSV, but
included when you treat the file as lines of text?

 Reading from a file is considered a low-level operation. Reading
 individual bytes in binary mode is the lowest level; reading lines in
 text mode is the next level, built on top of the lower binary mode. You
 build higher protocols on top of one or the other of that mode, e.g.
 read a zip file would be built on top of binary mode, read a csv file
 would be built on top of text mode.

I agree that reading a binary file is the lowest level. Reading a text
file is higher level, but to me reading a text file means reading a
binary file and decoding it into Unicode text, and not ... and
dividing it into lines. Bear in mind that reading a CSV file can be
built on top of a Unicode decode, but not on a line-based iteration
(in case there are newlines inside quotes).

 As a low-level protocol, you ought to be able to copy a file without
 changing it by reading it in then writing it out:

 for blob in infile:
 outfile.write(blob)


 ought to work whether you are in text mode or binary mode, so long as the
 infile and outfile are opened in the same mode. If Python were to strip
 newlines, that would no longer be the case.

All you need is a writeln method that re-adds the newline, and then
it's correctly round-tripping, based on what you've already stated
about the file: that it's a series of lines of text. It might not be a
byte-equivalent round-trip if you're changing newline style, any more
than it already won't be for other reasons (file encoding, for
instance). By reading the file as a series of Unicode lines, you're
declaring that it contains lines of Unicode text, not arbitrary bytes,
and so a valid representation of those lines of Unicode text is a
faithful reproduction of the file. If you want a byte-for-byte
identical file, open it in binary mode to do the copy; that's what we
learn from FTPing files between Linux and Windows.

 (Even high-level protocols should avoid unnecessary modifications to
 files. One of the more annoying, if not crippling, limitations to the
 configparser module is that reading an INI file in, then writing it out
 again destroys the high-level structure of the 

Re: Reading in cooked mode (was Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary)

[Cameron Simpson]

On 23Mar2014 12:37, Chris Angelico ros...@gmail.com wrote:
 On Sun, Mar 23, 2014 at 12:07 PM, Steven D'Aprano
 steve+comp.lang.pyt...@pearwood.info wrote:
  On Sun, 23 Mar 2014 02:09:20 +1100, Chris Angelico wrote:
  On Sun, Mar 23, 2014 at 1:50 AM, Steven D'Aprano
  steve+comp.lang.pyt...@pearwood.info wrote:
  Line endings are terminators: they end the line. Whether you consider
  the terminator part of the line or not is a matter of opinion (is the
  cover of a book part of the book?) but consider this:
 
  If you say that the end of lines are *not* part of the line, then
  that implies that some parts of the file are not inside any line at
  all. And that would be just weird.
 
  Not so weird IMO. A file is not a concatenation of lines; it is a stream
  of bytes.
 
  But a *text file* is a concatenation of lines. The text file model is
  important enough that nearly all programming languages offer a line-based
  interface to files, and some (Python at least, possibly others) make it
  the default interface so that iterating over the file gives you lines
  rather than bytes -- even in binary mode.
 
 And lines are delimited entities. A text file is a sequence of lines,
 separated by certain characters.
[...snip...]

As far as I'm concerned, a text file is a sequence lines, each of
which is _terminated_ by a newline (or the OS end-of-line flavour).

So I say terminated by, not separated by.

Plenty of people use editors that consider end-of-line to be a
separator and not a terminator, leading to supposed text files
lacking trailing newlines (or end-of-line of OS).

I consider this sloppy and error prone.

I like to be able to read a file and if it lacks a final newline
then I have a good clue that the file was incompletely written.
Editors (and other tools) that won't enforce a trailing newline as
omitting an easy way to give a fairly robust indication of completion
at no benefit to the user. (Not to mention the visual annoyance of
cat file when there's no trailing newline.)

So I'm happy to write code that errors if a line lacks a trailing
newline, and thus I consider the newline to be an intergral part
of the line.

Having passed that sanity check, for most machine readable text
formats I'm usually happy to use:

  line = line.rstrip()

to get the salient part of the line.

(Of course, lines extended with slosh-extension or the like need
pickier handling.)

Cheers,
-- 
Cameron Simpson c...@zip.com.au

If at first you don't succeed, your sky-diving days are over.
- Paul Blumstein, pa...@harley.tti.com, DoD #36
-- 
https://mail.python.org/mailman/listinfo/python-list

Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[cool-RR]

Hi everybody,

I need to install Python 3.4 final urgently, because my IDE stopped supporting 
Python 3.4 beta2, and I need it urgently to work.

I downloaded it, but the MSI won't install. It didn't work on both of my 
computers (Windows 7 64bit).

I managed to have the MSI dump data to log, file attached.

I couldn't help but notice this line in the log:

The installer has encountered an unexpected error installing this package. 
This may indicate a problem with this package. The error code is 2726. The 
arguments are: HO_CHI~1|Ho_Chi_Minh

A Google search shows this: http://en.wikipedia.org/wiki/Ho_Chi_Minh

What the hell. Was python.org hacked by communists?

Please, I need to get back to work and I can't. 


Thanks,
Ram.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[cool-RR]

Sorry, couldn't attach the file, here's the log file:

https://gist.github.com/anonymous/9697505

On Saturday, March 22, 2014 12:05:59 AM UTC+2, cool-RR wrote:
 Hi everybody,
 
 
 
 I need to install Python 3.4 final urgently, because my IDE stopped 
 supporting Python 3.4 beta2, and I need it urgently to work.
 
 
 
 I downloaded it, but the MSI won't install. It didn't work on both of my 
 computers (Windows 7 64bit).
 
 
 
 I managed to have the MSI dump data to log, file attached.
 
 
 
 I couldn't help but notice this line in the log:
 
 
 
 The installer has encountered an unexpected error installing this 
 package. This may indicate a problem with this package. The error code is 
 2726. The arguments are: HO_CHI~1|Ho_Chi_Minh
 
 
 
 A Google search shows this: http://en.wikipedia.org/wiki/Ho_Chi_Minh
 
 
 
 What the hell. Was python.org hacked by communists?
 
 
 
 Please, I need to get back to work and I can't. 
 
 
 
 
 
 Thanks,
 
 Ram.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Chris Angelico]

On Sat, Mar 22, 2014 at 9:05 AM, cool-RR ram.rac...@gmail.com wrote:
 I downloaded it, but the MSI won't install. It didn't work on both of my 
 computers (Windows 7 64bit).

 What the hell. Was python.org hacked by communists?

First question: Where did you download from? What file did you get?

(First and a halfth question: When you say won't install, exactly
what do you mean? Error message? Hard drive exploded in a fiery
inferno? Your boss tapped you on the shoulder and said Kill that
process?)

Secondly, do you have a tool for checking the MD5 hash of a file?
Compare the file you have against the official checksum:

https://www.python.org/downloads/release/python-340/

I'm slightly surprised the python.org installers have MD5s and not
something more cryptographically secure; there are GPG signatures, but
it takes a bit more fiddling to check those.

Finally, take the simple approach: Re-download the file, straight from
python.org, and see if it happens again.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[cool-RR]

I did download from python.org. I checked the md5, it was incorrect, then I 
downloaded again by using a proxy in Austria. (Which hopefully the communists 
haven't be able to infiltrate? ;)

Now it worked! Woohoo!

I'm still curious about the bad installation file... And what Ho Chi Minh is 
doing in the Python MSI. (I'm guessing it's timezone-related, but it's still 
far-fetched, because why would an obscure time zone file appear in the MSI log?)

On Saturday, March 22, 2014 12:25:03 AM UTC+2, Chris Angelico wrote:
 On Sat, Mar 22, 2014 at 9:05 AM, cool-RR ram.rac...@gmail.com wrote:
 
  I downloaded it, but the MSI won't install. It didn't work on both of my 
  computers (Windows 7 64bit).
 
 
 
  What the hell. Was python.org hacked by communists?
 
 
 
 First question: Where did you download from? What file did you get?
 
 
 
 (First and a halfth question: When you say won't install, exactly
 
 what do you mean? Error message? Hard drive exploded in a fiery
 
 inferno? Your boss tapped you on the shoulder and said Kill that
 
 process?)
 
 
 
 Secondly, do you have a tool for checking the MD5 hash of a file?
 
 Compare the file you have against the official checksum:
 
 
 
 https://www.python.org/downloads/release/python-340/
 
 
 
 I'm slightly surprised the python.org installers have MD5s and not
 
 something more cryptographically secure; there are GPG signatures, but
 
 it takes a bit more fiddling to check those.
 
 
 
 Finally, take the simple approach: Re-download the file, straight from
 
 python.org, and see if it happens again.
 
 
 
 ChrisA

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[cool-RR]

Here's the offending MSI, if anyone wants to investigate: 
https://dl.dropboxusercontent.com/u/1927707/python-3.4.0.amd64.msi


On Saturday, March 22, 2014 12:34:06 AM UTC+2, cool-RR wrote:
 I did download from python.org. I checked the md5, it was incorrect, then I 
 downloaded again by using a proxy in Austria. (Which hopefully the communists 
 haven't be able to infiltrate? ;)
 
 
 
 Now it worked! Woohoo!
 
 
 
 I'm still curious about the bad installation file... And what Ho Chi Minh is 
 doing in the Python MSI. (I'm guessing it's timezone-related, but it's still 
 far-fetched, because why would an obscure time zone file appear in the MSI 
 log?)
 
 
 
 On Saturday, March 22, 2014 12:25:03 AM UTC+2, Chris Angelico wrote:
 
  On Sat, Mar 22, 2014 at 9:05 AM, cool-RR ram.rac...@gmail.com wrote:
 
  
 
   I downloaded it, but the MSI won't install. It didn't work on both of my 
   computers (Windows 7 64bit).
 
  
 
  
 
  
 
   What the hell. Was python.org hacked by communists?
 
  
 
  
 
  
 
  First question: Where did you download from? What file did you get?
 
  
 
  
 
  
 
  (First and a halfth question: When you say won't install, exactly
 
  
 
  what do you mean? Error message? Hard drive exploded in a fiery
 
  
 
  inferno? Your boss tapped you on the shoulder and said Kill that
 
  
 
  process?)
 
  
 
  
 
  
 
  Secondly, do you have a tool for checking the MD5 hash of a file?
 
  
 
  Compare the file you have against the official checksum:
 
  
 
  
 
  
 
  https://www.python.org/downloads/release/python-340/
 
  
 
  
 
  
 
  I'm slightly surprised the python.org installers have MD5s and not
 
  
 
  something more cryptographically secure; there are GPG signatures, but
 
  
 
  it takes a bit more fiddling to check those.
 
  
 
  
 
  
 
  Finally, take the simple approach: Re-download the file, straight from
 
  
 
  python.org, and see if it happens again.
 
  
 
  
 
  
 
  ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Chris Angelico]

On Sat, Mar 22, 2014 at 9:34 AM, cool-RR ram.rac...@gmail.com wrote:
 I did download from python.org. I checked the md5, it was incorrect, then I 
 downloaded again by using a proxy in Austria. (Which hopefully the communists 
 haven't be able to infiltrate? ;)


I think you should follow the internet version of Hanlon's Razor here:
Damaged transmission before deliberate tampering. :) It's far more
likely something simply got misdownloaded, and your guess about
timezones is the most likely one - the Asia/Ho_Chi_Minh timezone [1],
and I believe Python Windows installers include a full zoneinfo
database.

ChrisA

[1] See https://en.wikipedia.org/wiki/Ho_Chi_Minh_City
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Mark Lawrence]

On 21/03/2014 22:34, cool-RR wrote:

I'm pleased to see that you have answers.  In return would you either 
use the mailing list 
https://mail.python.org/mailman/listinfo/python-list or read and action 
this https://wiki.python.org/moin/GoogleGroupsPython to prevent us 
seeing double line spacing and single line paragraphs, thanks.


--
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.


Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection 
is active.
http://www.avast.com


--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[cool-RR]

On Saturday, March 22, 2014 12:42:56 AM UTC+2, Chris Angelico wrote:
 I think you should follow the internet version of Hanlon's Razor here:
 Damaged transmission before deliberate tampering. :) It's far more
 likely something simply got misdownloaded, and your guess about
 timezones is the most likely one - the Asia/Ho_Chi_Minh timezone [1],
 and I believe Python Windows installers include a full zoneinfo
 database.


The thing is, I then tried other versions of Python 3.4, like b2, and the 32 
bit version, and they didn't work either... So damaged transmission of 3 
separate files? I'm not seeing any other damaged files when surfing using this 
connection.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[cool-RR]

On Saturday, March 22, 2014 12:25:03 AM UTC+2, Chris Angelico wrote:
 (First and a halfth question: When you say won't install, exactly
 what do you mean?

For completeness, I'll answer this question I forgot to answer, in case someone 
still wants to investigate: It just showed the first dialog (Something like 
Preparing to install...), after a few seconds it disappeared and then nothing.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Terry Reedy]

On 3/21/2014 6:55 PM, cool-RR wrote:

On Saturday, March 22, 2014 12:25:03 AM UTC+2, Chris Angelico wrote:

(First and a halfth question: When you say won't install, exactly
what do you mean?


For completeness, I'll answer this question I forgot to answer, in case someone still 
wants to investigate: It just showed the first dialog (Something like Preparing to 
install...), after a few seconds it disappeared and then nothing.


Does your .b2 install work? Can you delete it thru the programs list?

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Mark H Harris]

On 3/21/14 5:44 PM, Mark Lawrence wrote:

I'm pleased to see that you have answers. In return would you either use
the mailing list https://mail.python.org/mailman/listinfo/python-list or
read and action this https://wiki.python.org/moin/GoogleGroupsPython to
prevent us seeing double line spacing and single line paragraphs, thanks.


I perceive that this is your singular pet peeve, or, you were elected by 
the python community some time ago to police the line-end problem ?


I notice (since moving my stuff to Thunderbird two weeks back) the 
double spacing you keep squawking about, but I don't find it the big 
nuisance you're talking about; ok, so we have to scroll a bit further.


I am honestly convinced that this might even be a python problem.  More 
likely than not, gg is written in python, and this is the goofy line-end 
character problem we have to deal with when we read lines in python.


Why do we suck in the new-line character as though it were part of the 
line?  This is asinine behavior.  The new-line is a file delimiter 
character and NOT intended to be part of the line.


Thinking this through a bit I've noticed that a blank line comes back 
with a '\n'  which differentiates it from file end which comes back 
without the new-line.  So, it appears that python is using the 
new-line character (or lack there-of) to have meaning which the new=line 
in a unix file was never suppose to carry.


If python would just return EOF like every other language at file end, 
and a test line without '\n' tacked to the end of it, this little snag 
with gg would probably go away.  What say you?


Of course, that does not alleviate all of the rest of gg's short 
comings!   sigh


marcus

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Chris Angelico]

On Sat, Mar 22, 2014 at 2:58 PM, Mark H Harris harrismh...@gmail.com wrote:
 On 3/21/14 5:44 PM, Mark Lawrence wrote:

 I'm pleased to see that you have answers. In return would you either use
 the mailing list https://mail.python.org/mailman/listinfo/python-list or
 read and action this https://wiki.python.org/moin/GoogleGroupsPython to
 prevent us seeing double line spacing and single line paragraphs, thanks.


 I perceive that this is your singular pet peeve, or, you were elected by the
 python community some time ago to police the line-end problem ?

Communities maintain standards by policing them. If nobody cares
enough to post, the problem will remain. He happens to be one of the
most frequent commenters on this particular issue, but believe you me,
he is not the only one to wish it were solved; I used to speak up too,
but became tired of saying it (and of the blowback), and now I ignore
all double-spaced junk. If I can't figure out what's going on without
the context, I'll just move on to the next post.

 I notice (since moving my stuff to Thunderbird two weeks back) the double
 spacing you keep squawking about, but I don't find it the big nuisance
 you're talking about; ok, so we have to scroll a bit further.

It compounds. One reply makes for double spacing... two makes
quadruple, three means we have seven wasted lines between every pair
of real lines. That gets pretty annoying. And considering that most
people who reply without cleaning up the lines also keep the entire
quoted text (and usually top-post as well), this gets big fast.

 I am honestly convinced that this might even be a python problem.  More
 likely than not, gg is written in python, and this is the goofy line-end
 character problem we have to deal with when we read lines in python.

No idea why you should think this; it's more likely to be based on
HTML parsing (newline - paragraph - double newline), especially
since the original text is usually unwrapped.

 Why do we suck in the new-line character as though it were part of the line?
 This is asinine behavior.  The new-line is a file delimiter character and
 NOT intended to be part of the line.

 Thinking this through a bit I've noticed that a blank line comes back with a
 '\n'  which differentiates it from file end which comes back without the
 new-line.  So, it appears that python is using the new-line character (or
 lack there-of) to have meaning which the new=line in a unix file was never
 suppose to carry.

 If python would just return EOF like every other language at file end, and a
 test line without '\n' tacked to the end of it, this little snag with gg
 would probably go away.  What say you?

Personally, I think that iterating over the lines in a file (in the
most obvious way) should strip delimiters. There should be a
less-obvious way to iterate over the lines complete with their line
ends, which can then be used to distinguish between the different
line endings (including the absence of one at the end of the file),
but most of the time you want to just treat a file as a series of
lines, and don't care about the distinctions. However, that ship has
sailed; it's way WAY too late to make any such change.

And changing it would not fix the Google Groups problem unless it also
broke a whole pile of other code out there.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Mark H Harris]

On 3/21/14 11:15 PM, Chris Angelico wrote:

It compounds. One reply makes for double spacing... two makes
quadruple, three means we have seven wasted lines between every pair
of real lines. That gets pretty annoying. And considering that most
people who reply without cleaning up the lines also keep the entire
quoted text (and usually top-post as well), this gets big fast.


   Yes, I can see that readily/  I get it, it just seems that fixing it 
at the source (gg) is the answer; because there will always be someone 
new to the list who is using gg because its part of their suite and 
because its convenient (I mean, that's why I tried to use it). I think 
we need to be beating up on the gg people so they get their stuff working.



Personally, I think that iterating over the lines in a file (in the
most obvious way) should strip delimiters.


   agreed.


most of the time you want to just treat a file as a series of
lines, and don't care about the distinctions. However, that ship has
sailed; it's way WAY too late to make any such change.


   agreed again, on both counts.


And changing it would not fix the Google Groups problem unless it also
broke a whole pile of other code out there.


   No doubt.  Well, I don't mind dealing with it/  and frankly, I don't 
really think its a python problem anyway at the root. All files should 
have standard delimiters.  What I used to call flat-text files should 
have standard line-end delimiters, and standard file-end EOF markers. 
All OS's should comply with the standard... for instance, there should 
not be a windows x'0a' x'0d' line ending, and a unix  x'0d' line ending.


   Well, and now that I'm thinking about this again, since we have 
unicode, maybe we should have an entire set of standard file 
delimiters for flat-files.


   But the bottom line (pun intended) is that I just want to suck the 
lines in, and I only want the system to have to handle the delimiters; 
I'm not asking for any changes mind you (at this point) just thinking 
out-loud.


Cheers dude.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Mark H Harris]

On 3/21/14 11:30 PM, Mark H Harris wrote:

All OS's should comply with the standard... for instance, there should
not be a windows x'0a' x'0d' line ending, and a unix x'0d' line ending.



whoops...  I meant  unix  x'0a' line ending...;-)


'\n'

:-))
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Chris Angelico]

On Sat, Mar 22, 2014 at 3:30 PM, Mark H Harris harrismh...@gmail.com wrote:
 All files should have standard delimiters.  What I used to call flat-text
 files should have standard line-end delimiters, and standard file-end EOF
 markers. All OS's should comply with the standard... for instance, there
 should not be a windows x'0a' x'0d' line ending, and a unix  x'0d' line
 ending.

(Side point: You have your 0d and your 0a backwards; the Unix line
ending is U+000A, and the Windows default is U+000D U+000A.)

How are you going to make people change? What are you going to make
them change to? Who controls this standard, and how do you convince
all OSes to comply with it?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Mark H Harris]

On 3/21/14 11:46 PM, Chris Angelico wrote:

(Side point: You have your 0d and your 0a backwards; the Unix line
ending is U+000A, and the Windows default is U+000D U+000A.)


   Yeah, I know... smart apple.


How are you going to make people change? What are you going to make
them change to? Who controls this standard, and how do you convince
all OSes to comply with it?


   Well, we're already doing this to some extent; baby steps.  Well, we 
have open document standards (evolving) and we have a really good sense 
for unicode (and python is being a genuine leader there) and the 
flat-file is just another open document (very simple no doubt), not 
different from a standards viewpoint than rft, odt, {whatever}; txt?


   My idea is that as we are morphing open document standards we need 
to keep the flat-file in mind too.  The ASCII ship has sailed too. 
Unicode is in, ASCII is out (for all intents and purposes) except at 
Microsoft---and its time to rethink what a flat unicode text file 
really is. That's all.


marcus
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Terry Reedy]

On 3/22/2014 12:30 AM, Mark H Harris wrote:

On 3/21/14 11:15 PM, Chris Angelico wrote:

It compounds. One reply makes for double spacing... two makes
quadruple, three means we have seven wasted lines between every pair
of real lines. That gets pretty annoying. And considering that most
people who reply without cleaning up the lines also keep the entire
quoted text (and usually top-post as well), this gets big fast.


Before Mark started asking people adjust to the foibles of gg, we used 
to get such posts. I refused to read them. I have not seen one lately, 
say maybe his nudging has had some positive effect.



Yes, I can see that readily/  I get it, it just seems that fixing it
at the source (gg) is the answer;


I completely agree. However, Google seems immune to suggestions, 
including requests that it try to stop being a major source of spam posts.


 because there will always be someone

new to the list who is using gg because its part of their suite and
because its convenient (I mean, that's why I tried to use it).


If I were in charge of the software used for this list, I would replace 
Mark with a custom addition to return mis-formated posts (more blank 
lines than not) with instructions on how to fix them. But I am not.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python MSI not installing, log file showing name of a Viatnemese communist revolutionary

[Chris Angelico]

On Sat, Mar 22, 2014 at 4:24 PM, Terry Reedy tjre...@udel.edu wrote:
 If I were in charge of the software used for this list, I would replace Mark
 with a custom addition to return mis-formated posts (more blank lines than
 not) with instructions on how to fix them. But I am not.

I love how this makes it sound as if Mark is part of Mailman...

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Re for Apache log file format

[Piet van Oostrum]

Sam Giraffe s...@giraffetech.biz writes:

 Hi,

 I am trying to split up the re pattern for Apache log file format and seem to 
 be having some
 trouble in getting Python to understand multi-line pattern:

 #!/usr/bin/python

 import re

 #this is a single line
 string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] GET / HTTP/1.0 302 
 276 - check_http/
 v1.4.16 (nagios-plugins 1.4.16)'

 #trying to break up the pattern match for easy to read code
 pattern = re.compile(r'(?Pip\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+'
  r'(?Pident\-)\s+'
  r'(?Pusername\-)\s+'
  r'(?PTZ\[(.*?)\])\s+'
  r'(?Purl\(.*?)\)\s+'
  r'(?Phttpcode\d{3})\s+'
  r'(?Psize\d+)\s+'
  r'(?Preferrer\\)\s+'
  r'(?Pagent\((.*?)\))')

 match = re.search(pattern, string)

 if match:
     print match.group('ip')
 else:
     print 'not found'

 The python interpreter is skipping to the 'math = re.search' and then the 
 'if' statement right
 after it looks at the ip, instead of moving onto ident and so on.

Although you have written the regexp as a sequence of lines, in reality it is a 
single string, and therefore pdb will do only a single step, and not go into 
its parts, which really are not parts.

 mybox:~ user$ python -m pdb /Users/user/Documents/Python/apache.py
 /Users/user/Documents/Python/apache.py(3)module()
 - import re
 (Pdb) n
 /Users/user/Documents/Python/apache.py(5)module()
 - string = '192.168.122.3 - - [29/Sep/2013:03:52:33 -0700] GET / HTTP/1.0 
 302 276 -
 check_http/v1.4.16 (nagios-plugins 1.4.16)'
 (Pdb) n
 /Users/user/Documents/Python/apache.py(7)module()
 - pattern = re.compile(r'(?Pip\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+'
 (Pdb) n
 /Users/user/Documents/Python/apache.py(17)module()
 - match = re.search(pattern, string)
 (Pdb)

Also as Andreas has noted the r'(?Preferrer\\)\s+' part is wrong. It should 
probably be 
r'(?Preferrer\.*?\)\s+'

And the r'(?Pagent\((.*?)\))') will also not match as there is text outside 
the (). Should probably also be
r'(?Pagent\.*?\)') or something like it.
-- 
Piet van Oostrum p...@vanoostrum.org
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list