Re: Python 3.11.0 installation and Tkinter does not work
> > I want learn python for 4 weeks and have problems, installing Tkinter. If > I installed 3.11.0 for my windows 8.1 from python.org and type > > >>> import _tkinter > > Traceback (most recent call last): > >File "", line 1, in > > ImportError: DLL load failed while importing _tkinter: Das angegebene > > Modul wurde nicht gefunden. > > > So I it is a tkinter Problem and I tried this: > > > >>> import _tkinter > > Traceback (most recent call last): > >File "", line 1, in > > ImportError: DLL load failed while importing _tkinter: Das angegebene > > Modul wurde nicht gefunden. > > How can I fix this and make it work? > Have a look at https://stackoverflow.com/questions/25905540/importerror-no-module-named-tkinter . Most of the answers are for Linux/Mac, but there are Windows answers there, too. -- https://mail.python.org/mailman/listinfo/python-list
Re: Preprocessing not quite fixed-width file before parsing
On 11/23/2022 11:00 AM, Loris Bennett wrote:
Hi,
I am using pandas to parse a file with the following structure:
Name filesettype KB quota limit in_doubt
grace |files quotalimit in_doubtgrace
shortname sharedhome USR14097664 524288000 545259520 0
none | 107110 000 none
gracedays sharedhome USR 774858944 524288000 775946240 0 5
days | 1115717 000 none
nametoolong sharedhome USR27418496 524288000 545259520 0
none |11581 000 none
I was initially able to use
df = pandas.read_csv(file_name, delimiter=r"\s+")
because all the values for 'grace' were 'none'. Now, however,
non-"none" values have appeared and this fails.
I can't use
pandas.read_fwf
even with an explicit colspec, because the names in the first column
which are too long for the column will displace the rest of the data to
the right.
The report which produces the file could in fact also generate a
properly delimited CSV file, but I have a lot of historical data in the
readable but poorly parsable format above that I need to deal with.
If I were doing something similar in the shell, I would just pipe the
file through sed or something to replace '5 days' with, say '5_days'.
How could I achieve a similar sort of preprocessing in Python, ideally
without having to generate a lot of temporary files?
This is really annoying, isn't it? A space-separated line with spaces
in data entries. If the example you give is typical, I don't think
there is a general solution. If you know there are only certain values
like that, then you could do a search-and-replace for them in Python
just like the example you gave for "5 days".
If you know that the field that might contain entries with spaces is the
same one, e.g., the one just before the "|" marker, you could make use
of that. But it could be tricky.
I don't know how many files like this you will need to process, nor how
many rows they might contain. If I were to do tackle this job, I would
probably do some quality checking first. Using this example file,
figure out how many fields there are supposed to be. First, split the
file into lines:
with open("filename") as f:
lines = f.readlines()
# Check space-separated fields defined in first row:
fields = lines[0].split()
num_fields = len(fields)
print(num_fields) # e.g., 100)
# Find lines that have the wrong number of fields
bad_lines = []
for line in lines:
fields = line.split()
if len(fields) != num_fields:
bad_lines.append(line)
print(len(bad_lines))
# Inspect a sample
for line in bad_lines[:10]:
print(line)
This will give you an idea of how many problems lines there are, and if
they can all be fixed by a simple replacement. If they can and this is
the only file you need to handle, just fix it up and run it. I would
replace the spaces with tabs or commas. Splitting a line on spaces
(split()) takes care of the issue of having a variable number of spaces,
so that's easy enough.
If you will need to handle many files, and you can automate the fixes -
possibly with a regular expression - then you should preprocess each
file before giving it to pandas. Something like this:
def fix_line(line):
"""Test line for field errors and fix errors if any."""
#
return fixed
# For each file
with open("filename") as f:
lines = f.readlines()
fixed_lines = []
for line in lines:
fixed = fix_line(line)
fields = fixed.split()
tabified = '\t'.join(fields) # Could be done by fix_line()
fixed_lines.append(tabified)
# Now use an IOString to feed the file to pandas
# From memory, some details may not be right
f = IOString()
f.writelines(fixed_lines)
# Give f to pandas as if it were an external file
# ...
--
https://mail.python.org/mailman/listinfo/python-list
Re: Preprocessing not quite fixed-width file before parsing
This seems to work. I’m inferring the | is present in each line that needs to
be fixed.
import pandas
import logging
class Wrapper:
"""Wrap file to fix up data"""
def __init__(self, filename):
self.filename = filename
def __enter__(self):
self.fh = open(self.filename,'r')
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.fh.close()
def __iter__(self):
"""This is required by pandas for some reason, even though it doesn't
seem to be called"""
raise ValueError("Unsupported operation")
def read(self, n: int):
"""Read data. Replace 'grace' before | if it has underscores in it"""
try:
data = self.fh.readline()
ht = data.split('|', maxsplit=2)
if len(ht) == 2:
head,tail = ht
hparts = head.split(maxsplit=7)
assert len(hparts) == 8
if ' ' in hparts[7].strip():
hparts[7] = hparts[7].strip().replace(' ','_')
fixed_data = f"{' '.join(hparts)} | {tail}"
return fixed_data
return data
except:
logging.exception("read")
logging.basicConfig()
with Wrapper('data.txt') as f:
df = pandas.read_csv(f, delimiter=r"\s+")
print(df)
From: Python-list on
behalf of Loris Bennett
Date: Wednesday, November 23, 2022 at 2:00 PM
To: python-list@python.org
Subject: Preprocessing not quite fixed-width file before parsing
*** Attention: This is an external email. Use caution responding, opening
attachments or clicking on links. ***
Hi,
I am using pandas to parse a file with the following structure:
Name filesettype KB quota limit in_doubt
grace |files quotalimit in_doubtgrace
shortname sharedhome USR14097664 524288000 545259520 0
none | 107110 000 none
gracedays sharedhome USR 774858944 524288000 775946240 0 5
days | 1115717 000 none
nametoolong sharedhome USR27418496 524288000 545259520 0
none |11581 000 none
I was initially able to use
df = pandas.read_csv(file_name, delimiter=r"\s+")
because all the values for 'grace' were 'none'. Now, however,
non-"none" values have appeared and this fails.
I can't use
pandas.read_fwf
even with an explicit colspec, because the names in the first column
which are too long for the column will displace the rest of the data to
the right.
The report which produces the file could in fact also generate a
properly delimited CSV file, but I have a lot of historical data in the
readable but poorly parsable format above that I need to deal with.
If I were doing something similar in the shell, I would just pipe the
file through sed or something to replace '5 days' with, say '5_days'.
How could I achieve a similar sort of preprocessing in Python, ideally
without having to generate a lot of temporary files?
Cheers,
Loris
--
This signature is currently under constuction.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hBypaGqqmBaUa_w_PNTK9VelYEJCChO6c7d8k1yz6N56806CJ0wtAfLhvj5UaWrGaccJTzKxrjQJCil9DJ470VZWO4fOfhk$
--
https://mail.python.org/mailman/listinfo/python-list
Re: Preprocessing not quite fixed-width file before parsing
Oops. Forgot to Reformat file before sending. Here’s the proper PEP-8 (at least
according to PyCharm)
import pandas
import logging
class Wrapper:
"""Wrap file to fix up data"""
def __init__(self, filename):
self.filename = filename
def __enter__(self):
self.fh = open(self.filename, 'r')
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.fh.close()
def __iter__(self):
"""This is required by pandas for some reason, even though it doesn't
seem to be called"""
raise ValueError("Unsupported operation")
def read(self, n: int):
"""Read data. Replace 'grace' before | if it has underscores in it"""
try:
data = self.fh.readline()
ht = data.split('|', maxsplit=2)
if len(ht) == 2:
head, tail = ht
hparts = head.split(maxsplit=7)
assert len(hparts) == 8
if ' ' in hparts[7].strip():
hparts[7] = hparts[7].strip().replace(' ', '_')
fixed_data = f"{' '.join(hparts)} | {tail}"
return fixed_data
return data
except:
logging.exception("read")
logging.basicConfig()
with Wrapper('data.txt') as f:
df = pandas.read_csv(f, delimiter=r"\s+")
print(df)
From: Weatherby,Gerard
Date: Wednesday, November 23, 2022 at 3:38 PM
To: Loris Bennett , python-list@python.org
Subject: Re: Preprocessing not quite fixed-width file before parsing
This seems to work. I’m inferring the | is present in each line that needs to
be fixed.
import pandas
import logging
class Wrapper:
"""Wrap file to fix up data"""
def __init__(self, filename):
self.filename = filename
def __enter__(self):
self.fh = open(self.filename,'r')
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.fh.close()
def __iter__(self):
"""This is required by pandas for some reason, even though it doesn't
seem to be called"""
raise ValueError("Unsupported operation")
def read(self, n: int):
"""Read data. Replace 'grace' before | if it has underscores in it"""
try:
data = self.fh.readline()
ht = data.split('|', maxsplit=2)
if len(ht) == 2:
head,tail = ht
hparts = head.split(maxsplit=7)
assert len(hparts) == 8
if ' ' in hparts[7].strip():
hparts[7] = hparts[7].strip().replace(' ','_')
fixed_data = f"{' '.join(hparts)} | {tail}"
return fixed_data
return data
except:
logging.exception("read")
logging.basicConfig()
with Wrapper('data.txt') as f:
df = pandas.read_csv(f, delimiter=r"\s+")
print(df)
From: Python-list on
behalf of Loris Bennett
Date: Wednesday, November 23, 2022 at 2:00 PM
To: python-list@python.org
Subject: Preprocessing not quite fixed-width file before parsing
*** Attention: This is an external email. Use caution responding, opening
attachments or clicking on links. ***
Hi,
I am using pandas to parse a file with the following structure:
Name filesettype KB quota limit in_doubt
grace |files quotalimit in_doubtgrace
shortname sharedhome USR14097664 524288000 545259520 0
none | 107110 000 none
gracedays sharedhome USR 774858944 524288000 775946240 0 5
days | 1115717 000 none
nametoolong sharedhome USR27418496 524288000 545259520 0
none |11581 000 none
I was initially able to use
df = pandas.read_csv(file_name, delimiter=r"\s+")
because all the values for 'grace' were 'none'. Now, however,
non-"none" values have appeared and this fails.
I can't use
pandas.read_fwf
even with an explicit colspec, because the names in the first column
which are too long for the column will displace the rest of the data to
the right.
The report which produces the file could in fact also generate a
properly delimited CSV file, but I have a lot of historical data in the
readable but poorly parsable format above that I need to deal with.
If I were doing something similar in the shell, I would just pipe the
file through sed or something to replace '5 days' with, say '5_days'.
How could I achieve a similar sort of preprocessing in Python, ideally
without having to generate a lot of temporary files?
Cheers,
Loris
--
This signature is currently under constuction.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!hBypaGqqmBaUa_w_PNTK9VelYEJCChO6c7d8k1yz6N56806CJ0wtAfLhvj5UaWrGaccJTzKxrjQJCil9DJ470VZWO4fOfhk$
Re: Python 3.11.0 installation and Tkinter does not work
> On 23 Nov 2022, at 06:31, Stefan Ram wrote: > > darkst...@o2online.de writes: >> I want learn python for 4 weeks and have problems, installing Tkinter. If I= >> installed 3.11.0 for my windows 8.1 from python.org and type > > Ok, so you already installed from python.org. I wonder a > little about the wording "installing Tkinter". Here, > I installed /Python/ from python.org, and this /included/ tkinter. > If you have really attempted a separate installation of tkinter, > it may help to uninstall and instead install Python /including/ > tkinter. > >>> ImportError: DLL load failed while importing _tkinter: Das angegebene >>> Modul wurde nicht gefunden. > > Another possibility of analysis is to watch the Python > process using "Process Monitor" (formerly from Sysinternals) > under Microsoft® Windows (not to be confused with "Process > Explorer"). This program requires some familiarization, > but then it can show you in which directories a process is > searching for which DLLs. This might help you to find the > name of the DLL missing and in which directory it should be. I think the depends.exe tool from sysintenals will do this as well. There is a mode where you run a program and it collects the data for all the DLLs that are used either statically linked or dynamicall loaded, that is the case for _tkinter. I have not used in in a very long time but I recall it shows errors from DLLs that failed to load. Barry > > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: In code, list.clear doesn't throw error - it's just ignored
On Thu, 24 Nov 2022 at 06:26, Stefan Ram wrote: > > Jon Ribbens writes: > >If you want to catch this sort of mistake automatically then you need > >a linter such as pylint: > > output > > , line 1 > list.clear > Warning: Attribute used as statement. > > , line 5 > list.clear > Warning: Attribute used as statement. > > source code > > import ast, sys > > def check( point, source ): > if isinstance( point, ast.Expr ) and\ > type( point.value )== ast.Attribute: > print( ", line", point.lineno, file=sys.stderr ) > print( source.split( '\n' )[ point.lineno-1 ], file=sys.stderr ) > print\ > ( "Warning:", "Attribute used as statement.", file=sys.stderr ) > print() > > def mylinter( source ): > for point in ast.walk( ast.parse( example )): >check( point, source ) > > example = """\ > list.clear > list.clear() > x = list.clear > print( list.clear ) > list.clear > """ > > mylinter( example ) > Uhh, yes? You just created an extremely simplistic linter. Your point? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Preprocessing not quite fixed-width file before parsing
Hi, I am using pandas to parse a file with the following structure: Name filesettype KB quota limit in_doubt grace |files quotalimit in_doubtgrace shortname sharedhome USR14097664 524288000 545259520 0 none | 107110 000 none gracedays sharedhome USR 774858944 524288000 775946240 0 5 days | 1115717 000 none nametoolong sharedhome USR27418496 524288000 545259520 0 none |11581 000 none I was initially able to use df = pandas.read_csv(file_name, delimiter=r"\s+") because all the values for 'grace' were 'none'. Now, however, non-"none" values have appeared and this fails. I can't use pandas.read_fwf even with an explicit colspec, because the names in the first column which are too long for the column will displace the rest of the data to the right. The report which produces the file could in fact also generate a properly delimited CSV file, but I have a lot of historical data in the readable but poorly parsable format above that I need to deal with. If I were doing something similar in the shell, I would just pipe the file through sed or something to replace '5 days' with, say '5_days'. How could I achieve a similar sort of preprocessing in Python, ideally without having to generate a lot of temporary files? Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
