Re: How to test characters of a string

[Christian Gollwitzer]

Am 08.06.22 um 19:57 schrieb De ongekruisigde:

On 2022-06-08, 2qdxy4rzwzuui...@potatochowder.com 
<2qdxy4rzwzuui...@potatochowder.com> wrote:

On 2022-06-09 at 04:15:46 +1000,
Chris Angelico  wrote:

If you insist:

 >>> s = 'nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin'
 >>> print(s.split(':'))
 ['nm-iodine', 'x', '996', '57', '', '/var/empty', 
'/run/current-system/sw/bin/nologin']

Hesitantly, because this is the Python mailing list, I claim (a) ':' is
simpler than r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$', and
(b) string.split covers pretty much the same edge cases as re.search.


Ah, but you don't catch the be numeric of fields (0-based) 2 and 3! But
agreed, it's not the best of examples.



Yes, that is a simplistic example, since the : can't even be quoted to 
appear in that format (which would require higher-order parsing than a 
simple split)


Fortunately, the OP has just given another requirement to recognise 
different patterns, and now it went into a direction where it will 
become quite ugly if you avoid REs or any other pattern matching tools.


This is also the main reason I suggested REs initially - often if you 
see other patterns in the data, you can easily adapt a RE solution, 
whereas you'll have to write the thing from ground up anew if you do it 
manually.


Christian

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Christian Gollwitzer]

Am 09.06.22 um 07:50 schrieb Dave:

Hi,

I’ve found you also need to take care of multiple disk CD releases. These have 
a format of

“1-01 Track Name”
“2-02  Trackl Name"

Meaning Disk 1 Track1, Disk 2, Track 2.

Also A and B Sides (from Vinyl LPs)

“A1-Track Name”
“B2-Track Name”

Side A, Track 1, etc.



Now you're getting into the complexity that is better handled by REs 
than by individual character examination.

The first of your formats matches the RE

\d-\d{2}

(one digit, - two digits). Anchor that to check for a match at the 
beginning.


The second one matches

(A|B)\d-

As long as one digit is enough. What is your goal, to extract these 
numbers or to strip them? Regexes can do both relatively easily.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Avi Gross via Python-list]

Dave,

Sometimes a task is done faster by NOT programming anything in any language!

Not only have you spent a lot of your own time but many dozens of messages here 
have dragged in others, who gain nothing ;-)

The domain you are operating in seems to have lots of variants in how the 
titles are stored as names and you keep finding new variants. Yes, if your goal 
is to use this as a way to learn more in general about Python, clearly it may 
meet that goal!

Contrary to what I (and some others) said earlier, it may be time to consider 
regular expressions and other heavier artillery! LOL!

I do not plan on sticking with your twists and turns but will quickly address 
your variants.

Sides that come in two's like records are presumably limited to using A and B 
in your example. But multiple disks connected can mean the digit(s) following 
can have double digits or even more. Your python code may have to contain lots 
of functions you create that match some pattern and return some value and 
perhaps other functions that know how to convert from that format to a common 
canonical format of your own so they can be compared.

Your main code may need to try them in various sequences till it finds a match 
and so on.

But when you are done, in what format do you save them? The original or your 
minimal? 

Still confusing to me, as someone who does not give a darn, is the reality that 
many songs may have the same name but be different as in a song from Sinatra 
when he was young and a later recording  with a different orchestra or by a 
Sinatra imitator. They may all be titled something like "New York, New York" or 
"NEW YORK -- NEW YORK" which your algorithm folds into the same characters.

So I am guessing you also need to access other data about the person who sings 
it or what year it was released to make comparisons. At some point you may want 
to create or borrow some sort of class/object that encapsulates your data as 
well as methods that let you do things like make a canonical version of the 
Title and then a way to ask if Object A is reasonably equal to object B might 
happen if you define a function/method of __eq__ for that class.

It might take you years and need periodic refining as you encounter ever more 
varied ways people have chosen to label their music, but so what? LOL!

Humor or sarcasm aside, your incremental exploratory method reminds me why it 
is a good idea to first scope out the outlines of your problem space and make 
some key decisions and write out a fairly detailed set of requirements before 
seriously making more than prototypes. You might get very different help from 
people if they understood that your first request was far from complete but 
only one of many that may better be worked on some other way.
And I wonder if you did any search of the internet to see if anyone had done 
anything similar in Python (or another language) that may handle parts of what 
you need before asking here. I note lots of people who come with what they 
consider a good programming background have to adjust to aspects of languages 
like python as what they know is in some ways wrong or inadequate in a new 
environment. 

-Original Message-
From: Dave 
To: python-list@python.org
Sent: Thu, Jun 9, 2022 2:50 am
Subject: Re: How to test characters of a string

Hi,

I’ve found you also need to take care of multiple disk CD releases. These have 
a format of

“1-01 Track Name”
“2-02  Trackl Name"

Meaning Disk 1 Track1, Disk 2, Track 2.

Also A and B Sides (from Vinyl LPs)

“A1-Track Name”
“B2-Track Name”

Side A, Track 1, etc.

Cheers
Dave


> On 8 Jun 2022, at 19:36, Dennis Lee Bieber  wrote:
> 
> On Wed, 8 Jun 2022 01:53:26 + (UTC), Avi Gross 
> declaimed the following:
> 
> 
>> 
>> So is it necessary to insist on an exact pattern of two digits followed by a 
>> space? 
>> 
>> 
>> That would fail on "44 Minutes", "40 Oz. Dream", "50 Mission Cap", "50 Ways 
>> to Say Goodbye", "99 Ways to Die" 
>> 
>> It looks to me like you need to compare TWICE just in case. If it matches in 
>> the original (perhaps with some normalization of case and whitespace, fine. 
>> If not will they match if one or both have something to remove as a prefix 
>> such as "02 ". And if you are comparing items where the same song is in two 
>> different numeric sequences on different disks, ...
> 
>     I suspect the OP really needs to extract the /track number/ from the
> ID3 information, and (converting to a 2digit formatted string) see if the
> file name begins with that track number... The format of the those
> filenames appear to be those generated by some software when ripping CDs to
> MP3s -- for example:
> 
> -=-=-
> c:\Music\Roger Miller\All Time Greatest Hits>dir
> Volume in drive C is OS
> Vo

Re: How to test characters of a string

[Dave]

Hi,

I’ve found you also need to take care of multiple disk CD releases. These have 
a format of

“1-01 Track Name”
“2-02  Trackl Name"

Meaning Disk 1 Track1, Disk 2, Track 2.

Also A and B Sides (from Vinyl LPs)

“A1-Track Name”
“B2-Track Name”

Side A, Track 1, etc.

Cheers
Dave


> On 8 Jun 2022, at 19:36, Dennis Lee Bieber  wrote:
> 
> On Wed, 8 Jun 2022 01:53:26 + (UTC), Avi Gross 
> declaimed the following:
> 
> 
>> 
>> So is it necessary to insist on an exact pattern of two digits followed by a 
>> space? 
>> 
>> 
>> That would fail on "44 Minutes", "40 Oz. Dream", "50 Mission Cap", "50 Ways 
>> to Say Goodbye", "99 Ways to Die" 
>> 
>> It looks to me like you need to compare TWICE just in case. If it matches in 
>> the original (perhaps with some normalization of case and whitespace, fine. 
>> If not will they match if one or both have something to remove as a prefix 
>> such as "02 ". And if you are comparing items where the same song is in two 
>> different numeric sequences on different disks, ...
> 
>   I suspect the OP really needs to extract the /track number/ from the
> ID3 information, and (converting to a 2digit formatted string) see if the
> file name begins with that track number... The format of the those
> filenames appear to be those generated by some software when ripping CDs to
> MP3s -- for example:
> 
> -=-=-
> c:\Music\Roger Miller\All Time Greatest Hits>dir
> Volume in drive C is OS
> Volume Serial Number is 4ACC-3CB4
> 
> Directory of c:\Music\Roger Miller\All Time Greatest Hits
> 
> 04/11/2022  05:06 PM  .
> 04/11/2022  05:06 PM  ..
> 07/26/2018  11:20 AM 4,493,279 01 Dang Me.mp3
> 07/26/2018  11:20 AM 5,072,414 02 Chug-A-Lug.mp3
> 07/26/2018  11:20 AM 4,275,844 03 Do-Wacka-Do.mp3
> 07/26/2018  11:20 AM 4,284,208 04 In the Summertime.mp3
> 07/26/2018  11:20 AM 6,028,730 05 King of the Road.mp3
> 07/26/2018  11:20 AM 4,662,182 06 You Can't Roller Skate in a
> Buffalo Herd.mp3
> 07/26/2018  11:20 AM 5,624,704 07 Engine, Engine #9.mp3
> 07/26/2018  11:20 AM 5,002,492 08 One Dyin' and a Buryin'.mp3
> 07/26/2018  11:21 AM 6,799,224 09 Last Word in Lonesome Is Me.mp3
> 07/26/2018  11:21 AM 5,637,230 10 Kansas City Star.mp3
> 07/26/2018  11:21 AM 4,656,910 11 England Swings.mp3
> 07/26/2018  11:21 AM 5,836,638 12 Husbands and Wives.mp3
> 07/26/2018  11:21 AM 5,470,216 13 I've Been a Long Time Leavin'.mp3
> 07/26/2018  11:21 AM 6,230,236 14 Walkin' in the Sunshine.mp3
> 07/26/2018  11:21 AM 6,416,060 15 Little Green Apples.mp3
> 07/26/2018  11:21 AM 9,794,442 16 Me and Bobby McGee.mp3
> 07/26/2018  11:22 AM 7,330,642 17 Where Have All the Average People
> Gone.mp3
> 07/26/2018  11:22 AM 7,334,752 18 South.mp3
> 07/26/2018  11:22 AM 6,981,924 19 Tomorrow Night in Baltimore.mp3
> 07/26/2018  11:22 AM 9,353,872 20 River in the Rain.mp3
>  20 File(s)121,285,999 bytes
>   2 Dir(s)  295,427,198,976 bytes free
> 
> c:\Music\Roger Miller\All Time Greatest Hits>
> -=-=-
> 
>   Untested (especially the ID3 "variable" -- substitute variables as
> needed to match the original code):
> 
 id3Track = 2
 track_number = "%2.2d " % id3Track
 track_number
> '02 '
 filename = "02 This is the life.mp3"
 if filename.startswith(track_number):
> ...   nametitle = filename[3:]
> ... else:
> ...   nametitle = filename
> ...   
 if nametitle.endswith(".mp3"):
> ...   nametitle = nametitle[:-4]
> ...   
 nametitle
> 'This is the life'
> 
>   Handling ASCII ' and " vs Unicode "smart" quotes is a different matter.
> 
>   One may still run the risk of having a filename without a track number
> BUT having a number that just manages to match the track number. To account
> for that I'd suggest using the sequence:
> 
> * Strip extension (if filename.lower().endswith(".mp3"): ...)
> * Handle any Unicode/ASCII quotes in both filename AND ID3 track title
> * Compare filename and title.
> * IF MATCHED -- done
> * IF NOT MATCHED
> * Format ID3 track number as shown above
> * Compare filename to (formatted track number + track 
> title)
> * IF MATCHED -- done
> * IF NOT MATCHED
> * Log full filename and ID3 track 
> title/track number to a
> log for later examination.
> 
> 
> 
> -- 
>   Wulfraed Dennis Lee Bieber AF6VN
>   wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-08, 2qdxy4rzwzuui...@potatochowder.com 
<2qdxy4rzwzuui...@potatochowder.com> wrote:
> On 2022-06-09 at 04:15:46 +1000,
> Chris Angelico  wrote:
>
>> On Thu, 9 Jun 2022 at 04:14, <2qdxy4rzwzuui...@potatochowder.com> wrote:
>> >
>> > On 2022-06-09 at 03:18:56 +1000,
>> > Chris Angelico  wrote:
>> >
>> > > On Thu, 9 Jun 2022 at 03:15, <2qdxy4rzwzuui...@potatochowder.com> wrote:
>> > > >
>> > > > On 2022-06-08 at 08:07:40 -,
>> > > > De ongekruisigde  wrote:
>> > > >
>> > > > > Depending on the problem a regular expression may be the much simpler
>> > > > > solution. I love them for e.g. text parsing and use them all the 
>> > > > > time.
>> > > > > Unrivaled when e.g. parts of text have to be extracted, e.g. from 
>> > > > > lines
>> > > > > like these:
>> > > > >
>> > > > >   root:x:0:0:System 
>> > > > > administrator:/root:/run/current-system/sw/bin/bash
>> > > > >   dhcpcd:x:995:991::/var/empty:/run/current-system/sw/bin/nologin
>> > > > >   nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin
>> > > > >   avahi:x:997:996:avahi-daemon privilege separation 
>> > > > > user:/var/empty:/run/current-system/sw/bin/nologin
>> > > > >   sshd:x:998:993:SSH privilege separation 
>> > > > > user:/var/empty:/run/current-system/sw/bin/nologin
>> > > > >   geoclue:x:999:998:Geoinformation 
>> > > > > service:/var/lib/geoclue:/run/current-system/sw/bin/nologin
>> > > > >
>> > > > > Compare a regexp solution like this:
>> > > > >
>> > > > >   >>> g = 
>> > > > > re.search(r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$' , s)
>> > > > >   >>> print(g.groups())
>> > > > >   ('geoclue', 'x', '999', '998', 'Geoinformation service', 
>> > > > > '/var/lib/geoclue', '/run/current-system/sw/bin/nologin')
>> > > > >
>> > > > > to the code one would require to process it manually, with all the 
>> > > > > edge
>> > > > > cases. The regexp surely reads much simpler (?).
>> > > >
>> > > > Uh...
>> > > >
>> > > > >>> import pwd # https://docs.python.org/3/library/pwd.html
>> > > > >>> [x for x in pwd.getpwall() if x[0] == 'geoclue']
>> > > > [pwd.struct_passwd(pw_name='geoclue', pw_passwd='x', pw_uid=992, 
>> > > > pw_gid=992, pw_gecos='Geoinformation service', 
>> > > > pw_dir='/var/lib/geoclue', pw_shell='/sbin/nologin')]
>> > >
>> > > That's great if the lines are specifically coming from your system's
>> > > own /etc/passwd, but not so much if you're trying to compare passwd
>> > > files from different systems, where you simply have the files
>> > > themselves.
>> >
>> > In addition to pwent to get specific entries from the local password
>> > database, POSIX has fpwent to get a specific entry from a stream that
>> > looks like /etc/passwd.  So even POSIX agrees that if you think you have
>> > to process this data manually, you're doing it wrong.  Python exposes
>> > neither functon directly (at least not in the pwd module or the os
>> > module; I didn't dig around or check PyPI).
>> 
>> So.. we can go find some other way of calling fpwent, or we can
>> just parse the file ourselves. It's a very VERY simple format.
>
> If you insist:
>
> >>> s = 
> 'nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin'
> >>> print(s.split(':'))
> ['nm-iodine', 'x', '996', '57', '', '/var/empty', 
> '/run/current-system/sw/bin/nologin']
>
> Hesitantly, because this is the Python mailing list, I claim (a) ':' is
> simpler than r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$', and
> (b) string.split covers pretty much the same edge cases as re.search.

Ah, but you don't catch the be numeric of fields (0-based) 2 and 3! But
agreed, it's not the best of examples.


-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dennis Lee Bieber]

On Wed, 8 Jun 2022 01:53:26 + (UTC), Avi Gross 
declaimed the following:


>
>So is it necessary to insist on an exact pattern of two digits followed by a 
>space? 
>
>
>That would fail on "44 Minutes", "40 Oz. Dream", "50 Mission Cap", "50 Ways to 
>Say Goodbye", "99 Ways to Die" 
>
>It looks to me like you need to compare TWICE just in case. If it matches in 
>the original (perhaps with some normalization of case and whitespace, fine. If 
>not will they match if one or both have something to remove as a prefix such 
>as "02 ". And if you are comparing items where the same song is in two 
>different numeric sequences on different disks, ...

I suspect the OP really needs to extract the /track number/ from the
ID3 information, and (converting to a 2digit formatted string) see if the
file name begins with that track number... The format of the those
filenames appear to be those generated by some software when ripping CDs to
MP3s -- for example:

-=-=-
c:\Music\Roger Miller\All Time Greatest Hits>dir
 Volume in drive C is OS
 Volume Serial Number is 4ACC-3CB4

 Directory of c:\Music\Roger Miller\All Time Greatest Hits

04/11/2022  05:06 PM  .
04/11/2022  05:06 PM  ..
07/26/2018  11:20 AM 4,493,279 01 Dang Me.mp3
07/26/2018  11:20 AM 5,072,414 02 Chug-A-Lug.mp3
07/26/2018  11:20 AM 4,275,844 03 Do-Wacka-Do.mp3
07/26/2018  11:20 AM 4,284,208 04 In the Summertime.mp3
07/26/2018  11:20 AM 6,028,730 05 King of the Road.mp3
07/26/2018  11:20 AM 4,662,182 06 You Can't Roller Skate in a
Buffalo Herd.mp3
07/26/2018  11:20 AM 5,624,704 07 Engine, Engine #9.mp3
07/26/2018  11:20 AM 5,002,492 08 One Dyin' and a Buryin'.mp3
07/26/2018  11:21 AM 6,799,224 09 Last Word in Lonesome Is Me.mp3
07/26/2018  11:21 AM 5,637,230 10 Kansas City Star.mp3
07/26/2018  11:21 AM 4,656,910 11 England Swings.mp3
07/26/2018  11:21 AM 5,836,638 12 Husbands and Wives.mp3
07/26/2018  11:21 AM 5,470,216 13 I've Been a Long Time Leavin'.mp3
07/26/2018  11:21 AM 6,230,236 14 Walkin' in the Sunshine.mp3
07/26/2018  11:21 AM 6,416,060 15 Little Green Apples.mp3
07/26/2018  11:21 AM 9,794,442 16 Me and Bobby McGee.mp3
07/26/2018  11:22 AM 7,330,642 17 Where Have All the Average People
Gone.mp3
07/26/2018  11:22 AM 7,334,752 18 South.mp3
07/26/2018  11:22 AM 6,981,924 19 Tomorrow Night in Baltimore.mp3
07/26/2018  11:22 AM 9,353,872 20 River in the Rain.mp3
  20 File(s)121,285,999 bytes
   2 Dir(s)  295,427,198,976 bytes free

c:\Music\Roger Miller\All Time Greatest Hits>
-=-=-

Untested (especially the ID3 "variable" -- substitute variables as
needed to match the original code):

>>> id3Track = 2
>>> track_number = "%2.2d " % id3Track
>>> track_number
'02 '
>>> filename = "02 This is the life.mp3"
>>> if filename.startswith(track_number):
... nametitle = filename[3:]
... else:
... nametitle = filename
... 
>>> if nametitle.endswith(".mp3"):
... nametitle = nametitle[:-4]
... 
>>> nametitle
'This is the life'

Handling ASCII ' and " vs Unicode "smart" quotes is a different matter.

One may still run the risk of having a filename without a track number
BUT having a number that just manages to match the track number. To account
for that I'd suggest using the sequence:

*   Strip extension (if filename.lower().endswith(".mp3"): ...)
*   Handle any Unicode/ASCII quotes in both filename AND ID3 track title
*   Compare filename and title.
*   IF MATCHED -- done
*   IF NOT MATCHED
*   Format ID3 track number as shown above
*   Compare filename to (formatted track number + track 
title)
*   IF MATCHED -- done
*   IF NOT MATCHED
*   Log full filename and ID3 track 
title/track number to a
log for later examination.



-- 
Wulfraed Dennis Lee Bieber AF6VN
wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-08, 2qdxy4rzwzuui...@potatochowder.com 
<2qdxy4rzwzuui...@potatochowder.com> wrote:
> On 2022-06-08 at 08:07:40 -,
> De ongekruisigde  wrote:
>
>> Depending on the problem a regular expression may be the much simpler
>> solution. I love them for e.g. text parsing and use them all the time.
>> Unrivaled when e.g. parts of text have to be extracted, e.g. from lines
>> like these:
>> 
>>   root:x:0:0:System administrator:/root:/run/current-system/sw/bin/bash
>>   dhcpcd:x:995:991::/var/empty:/run/current-system/sw/bin/nologin
>>   nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin
>>   avahi:x:997:996:avahi-daemon privilege separation 
>> user:/var/empty:/run/current-system/sw/bin/nologin
>>   sshd:x:998:993:SSH privilege separation 
>> user:/var/empty:/run/current-system/sw/bin/nologin
>>   geoclue:x:999:998:Geoinformation 
>> service:/var/lib/geoclue:/run/current-system/sw/bin/nologin
>> 
>> Compare a regexp solution like this:
>> 
>>   >>> g = re.search(r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$' , s)
>>   >>> print(g.groups())
>>   ('geoclue', 'x', '999', '998', 'Geoinformation service', 
>> '/var/lib/geoclue', '/run/current-system/sw/bin/nologin')
>> 
>> to the code one would require to process it manually, with all the edge
>> cases. The regexp surely reads much simpler (?).
>
> Uh...
>
> >>> import pwd # https://docs.python.org/3/library/pwd.html
> >>> [x for x in pwd.getpwall() if x[0] == 'geoclue']
> [pwd.struct_passwd(pw_name='geoclue', pw_passwd='x', pw_uid=992, 
> pw_gid=992, pw_gecos='Geoinformation service', pw_dir='/var/lib/geoclue', 
> pw_shell='/sbin/nologin')]

Yeah... Well, it was just an example and it must be clear by now I'm not
a Python programmer.

-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[2QdxY4RzWzUUiLuE]

On 2022-06-09 at 04:15:46 +1000,
Chris Angelico  wrote:

> On Thu, 9 Jun 2022 at 04:14, <2qdxy4rzwzuui...@potatochowder.com> wrote:
> >
> > On 2022-06-09 at 03:18:56 +1000,
> > Chris Angelico  wrote:
> >
> > > On Thu, 9 Jun 2022 at 03:15, <2qdxy4rzwzuui...@potatochowder.com> wrote:
> > > >
> > > > On 2022-06-08 at 08:07:40 -,
> > > > De ongekruisigde  wrote:
> > > >
> > > > > Depending on the problem a regular expression may be the much simpler
> > > > > solution. I love them for e.g. text parsing and use them all the time.
> > > > > Unrivaled when e.g. parts of text have to be extracted, e.g. from 
> > > > > lines
> > > > > like these:
> > > > >
> > > > >   root:x:0:0:System 
> > > > > administrator:/root:/run/current-system/sw/bin/bash
> > > > >   dhcpcd:x:995:991::/var/empty:/run/current-system/sw/bin/nologin
> > > > >   nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin
> > > > >   avahi:x:997:996:avahi-daemon privilege separation 
> > > > > user:/var/empty:/run/current-system/sw/bin/nologin
> > > > >   sshd:x:998:993:SSH privilege separation 
> > > > > user:/var/empty:/run/current-system/sw/bin/nologin
> > > > >   geoclue:x:999:998:Geoinformation 
> > > > > service:/var/lib/geoclue:/run/current-system/sw/bin/nologin
> > > > >
> > > > > Compare a regexp solution like this:
> > > > >
> > > > >   >>> g = 
> > > > > re.search(r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$' , s)
> > > > >   >>> print(g.groups())
> > > > >   ('geoclue', 'x', '999', '998', 'Geoinformation service', 
> > > > > '/var/lib/geoclue', '/run/current-system/sw/bin/nologin')
> > > > >
> > > > > to the code one would require to process it manually, with all the 
> > > > > edge
> > > > > cases. The regexp surely reads much simpler (?).
> > > >
> > > > Uh...
> > > >
> > > > >>> import pwd # https://docs.python.org/3/library/pwd.html
> > > > >>> [x for x in pwd.getpwall() if x[0] == 'geoclue']
> > > > [pwd.struct_passwd(pw_name='geoclue', pw_passwd='x', pw_uid=992, 
> > > > pw_gid=992, pw_gecos='Geoinformation service', 
> > > > pw_dir='/var/lib/geoclue', pw_shell='/sbin/nologin')]
> > >
> > > That's great if the lines are specifically coming from your system's
> > > own /etc/passwd, but not so much if you're trying to compare passwd
> > > files from different systems, where you simply have the files
> > > themselves.
> >
> > In addition to pwent to get specific entries from the local password
> > database, POSIX has fpwent to get a specific entry from a stream that
> > looks like /etc/passwd.  So even POSIX agrees that if you think you have
> > to process this data manually, you're doing it wrong.  Python exposes
> > neither functon directly (at least not in the pwd module or the os
> > module; I didn't dig around or check PyPI).
> 
> So.. we can go find some other way of calling fpwent, or we can
> just parse the file ourselves. It's a very VERY simple format.

If you insist:

>>> s = 'nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin'
>>> print(s.split(':'))
['nm-iodine', 'x', '996', '57', '', '/var/empty', 
'/run/current-system/sw/bin/nologin']

Hesitantly, because this is the Python mailing list, I claim (a) ':' is
simpler than r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$', and
(b) string.split covers pretty much the same edge cases as re.search.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Chris Angelico]

On Thu, 9 Jun 2022 at 04:14, <2qdxy4rzwzuui...@potatochowder.com> wrote:
>
> On 2022-06-09 at 03:18:56 +1000,
> Chris Angelico  wrote:
>
> > On Thu, 9 Jun 2022 at 03:15, <2qdxy4rzwzuui...@potatochowder.com> wrote:
> > >
> > > On 2022-06-08 at 08:07:40 -,
> > > De ongekruisigde  wrote:
> > >
> > > > Depending on the problem a regular expression may be the much simpler
> > > > solution. I love them for e.g. text parsing and use them all the time.
> > > > Unrivaled when e.g. parts of text have to be extracted, e.g. from lines
> > > > like these:
> > > >
> > > >   root:x:0:0:System administrator:/root:/run/current-system/sw/bin/bash
> > > >   dhcpcd:x:995:991::/var/empty:/run/current-system/sw/bin/nologin
> > > >   nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin
> > > >   avahi:x:997:996:avahi-daemon privilege separation 
> > > > user:/var/empty:/run/current-system/sw/bin/nologin
> > > >   sshd:x:998:993:SSH privilege separation 
> > > > user:/var/empty:/run/current-system/sw/bin/nologin
> > > >   geoclue:x:999:998:Geoinformation 
> > > > service:/var/lib/geoclue:/run/current-system/sw/bin/nologin
> > > >
> > > > Compare a regexp solution like this:
> > > >
> > > >   >>> g = 
> > > > re.search(r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$' , s)
> > > >   >>> print(g.groups())
> > > >   ('geoclue', 'x', '999', '998', 'Geoinformation service', 
> > > > '/var/lib/geoclue', '/run/current-system/sw/bin/nologin')
> > > >
> > > > to the code one would require to process it manually, with all the edge
> > > > cases. The regexp surely reads much simpler (?).
> > >
> > > Uh...
> > >
> > > >>> import pwd # https://docs.python.org/3/library/pwd.html
> > > >>> [x for x in pwd.getpwall() if x[0] == 'geoclue']
> > > [pwd.struct_passwd(pw_name='geoclue', pw_passwd='x', pw_uid=992, 
> > > pw_gid=992, pw_gecos='Geoinformation service', pw_dir='/var/lib/geoclue', 
> > > pw_shell='/sbin/nologin')]
> >
> > That's great if the lines are specifically coming from your system's
> > own /etc/passwd, but not so much if you're trying to compare passwd
> > files from different systems, where you simply have the files
> > themselves.
>
> In addition to pwent to get specific entries from the local password
> database, POSIX has fpwent to get a specific entry from a stream that
> looks like /etc/passwd.  So even POSIX agrees that if you think you have
> to process this data manually, you're doing it wrong.  Python exposes
> neither functon directly (at least not in the pwd module or the os
> module; I didn't dig around or check PyPI).

So.. we can go find some other way of calling fpwent, or we can
just parse the file ourselves. It's a very VERY simple format.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[2QdxY4RzWzUUiLuE]

On 2022-06-09 at 03:18:56 +1000,
Chris Angelico  wrote:

> On Thu, 9 Jun 2022 at 03:15, <2qdxy4rzwzuui...@potatochowder.com> wrote:
> >
> > On 2022-06-08 at 08:07:40 -,
> > De ongekruisigde  wrote:
> >
> > > Depending on the problem a regular expression may be the much simpler
> > > solution. I love them for e.g. text parsing and use them all the time.
> > > Unrivaled when e.g. parts of text have to be extracted, e.g. from lines
> > > like these:
> > >
> > >   root:x:0:0:System administrator:/root:/run/current-system/sw/bin/bash
> > >   dhcpcd:x:995:991::/var/empty:/run/current-system/sw/bin/nologin
> > >   nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin
> > >   avahi:x:997:996:avahi-daemon privilege separation 
> > > user:/var/empty:/run/current-system/sw/bin/nologin
> > >   sshd:x:998:993:SSH privilege separation 
> > > user:/var/empty:/run/current-system/sw/bin/nologin
> > >   geoclue:x:999:998:Geoinformation 
> > > service:/var/lib/geoclue:/run/current-system/sw/bin/nologin
> > >
> > > Compare a regexp solution like this:
> > >
> > >   >>> g = re.search(r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$' 
> > > , s)
> > >   >>> print(g.groups())
> > >   ('geoclue', 'x', '999', '998', 'Geoinformation service', 
> > > '/var/lib/geoclue', '/run/current-system/sw/bin/nologin')
> > >
> > > to the code one would require to process it manually, with all the edge
> > > cases. The regexp surely reads much simpler (?).
> >
> > Uh...
> >
> > >>> import pwd # https://docs.python.org/3/library/pwd.html
> > >>> [x for x in pwd.getpwall() if x[0] == 'geoclue']
> > [pwd.struct_passwd(pw_name='geoclue', pw_passwd='x', pw_uid=992, 
> > pw_gid=992, pw_gecos='Geoinformation service', pw_dir='/var/lib/geoclue', 
> > pw_shell='/sbin/nologin')]
> 
> That's great if the lines are specifically coming from your system's
> own /etc/passwd, but not so much if you're trying to compare passwd
> files from different systems, where you simply have the files
> themselves.

In addition to pwent to get specific entries from the local password
database, POSIX has fpwent to get a specific entry from a stream that
looks like /etc/passwd.  So even POSIX agrees that if you think you have
to process this data manually, you're doing it wrong.  Python exposes
neither functon directly (at least not in the pwd module or the os
module; I didn't dig around or check PyPI).

IMO, higher level functions to process such data is way better than a
[insert your own adjective/expletive here] regular expression that
collects the pieces into numbered groups rather than labeled fields.
Readability counts.

Yes, absolutely, use a regular expression when all else fails.  Don't
forget to handle all the edge cases!  (I assume that sane OSes preclude
colons in paths that are likely to come up in the local password
database, but I don't know what happens, e.g., when there's a reason for
GECOS to contain a colon.)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Chris Angelico]

On Thu, 9 Jun 2022 at 03:15, <2qdxy4rzwzuui...@potatochowder.com> wrote:
>
> On 2022-06-08 at 08:07:40 -,
> De ongekruisigde  wrote:
>
> > Depending on the problem a regular expression may be the much simpler
> > solution. I love them for e.g. text parsing and use them all the time.
> > Unrivaled when e.g. parts of text have to be extracted, e.g. from lines
> > like these:
> >
> >   root:x:0:0:System administrator:/root:/run/current-system/sw/bin/bash
> >   dhcpcd:x:995:991::/var/empty:/run/current-system/sw/bin/nologin
> >   nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin
> >   avahi:x:997:996:avahi-daemon privilege separation 
> > user:/var/empty:/run/current-system/sw/bin/nologin
> >   sshd:x:998:993:SSH privilege separation 
> > user:/var/empty:/run/current-system/sw/bin/nologin
> >   geoclue:x:999:998:Geoinformation 
> > service:/var/lib/geoclue:/run/current-system/sw/bin/nologin
> >
> > Compare a regexp solution like this:
> >
> >   >>> g = re.search(r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$' , 
> > s)
> >   >>> print(g.groups())
> >   ('geoclue', 'x', '999', '998', 'Geoinformation service', 
> > '/var/lib/geoclue', '/run/current-system/sw/bin/nologin')
> >
> > to the code one would require to process it manually, with all the edge
> > cases. The regexp surely reads much simpler (?).
>
> Uh...
>
> >>> import pwd # https://docs.python.org/3/library/pwd.html
> >>> [x for x in pwd.getpwall() if x[0] == 'geoclue']
> [pwd.struct_passwd(pw_name='geoclue', pw_passwd='x', pw_uid=992, 
> pw_gid=992, pw_gecos='Geoinformation service', pw_dir='/var/lib/geoclue', 
> pw_shell='/sbin/nologin')]

That's great if the lines are specifically coming from your system's
own /etc/passwd, but not so much if you're trying to compare passwd
files from different systems, where you simply have the files
themselves.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[2QdxY4RzWzUUiLuE]

On 2022-06-08 at 08:07:40 -,
De ongekruisigde  wrote:

> Depending on the problem a regular expression may be the much simpler
> solution. I love them for e.g. text parsing and use them all the time.
> Unrivaled when e.g. parts of text have to be extracted, e.g. from lines
> like these:
> 
>   root:x:0:0:System administrator:/root:/run/current-system/sw/bin/bash
>   dhcpcd:x:995:991::/var/empty:/run/current-system/sw/bin/nologin
>   nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin
>   avahi:x:997:996:avahi-daemon privilege separation 
> user:/var/empty:/run/current-system/sw/bin/nologin
>   sshd:x:998:993:SSH privilege separation 
> user:/var/empty:/run/current-system/sw/bin/nologin
>   geoclue:x:999:998:Geoinformation 
> service:/var/lib/geoclue:/run/current-system/sw/bin/nologin
> 
> Compare a regexp solution like this:
> 
>   >>> g = re.search(r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$' , s)
>   >>> print(g.groups())
>   ('geoclue', 'x', '999', '998', 'Geoinformation service', 
> '/var/lib/geoclue', '/run/current-system/sw/bin/nologin')
> 
> to the code one would require to process it manually, with all the edge
> cases. The regexp surely reads much simpler (?).

Uh...

>>> import pwd # https://docs.python.org/3/library/pwd.html
>>> [x for x in pwd.getpwall() if x[0] == 'geoclue']
[pwd.struct_passwd(pw_name='geoclue', pw_passwd='x', pw_uid=992, 
pw_gid=992, pw_gecos='Geoinformation service', pw_dir='/var/lib/geoclue', 
pw_shell='/sbin/nologin')]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Barry Scott]

> On 7 Jun 2022, at 23:24, Dave  wrote:
> 
> Yes, it was probably just a typeo on my part.
> 
> I’ve now fixed the majority of cases but still got two strings that look 
> identical but fail to match, this time (again by 10cc), “I’m Mandy Fly Me”.
> 
> I’m putting money on it being a utf8 problem but I’m stuck on how to handle 
> it. It’s probably the single quote in I’m, although it has worked with other 
> songs.
> 
> Any ideas?

You can use difflib to give you a diff of the two strings:

:>>> print('\n'.join(difflib.unified_diff('abc', 'adc')))
---

+++

@@ -1,3 +1,3 @@

 a
-b
+d
 c
:>>>

The docs talk about lines, but difflib works on sequence. I use it a lot to find
differences within lines.

Barry



> 
> All the Best
> Cheers
> Dave
> 
> Here is the whole function/method or whatever it’s called in Python:
> 
> 
> #
> #   checkMusicFiles
> #
> 
> def checkMusicFiles(theBaseMusicLibraryFolder):
>myArtistDict = []
> 
> #
> #  Loop thru Artists Folder
> #
>myArtistsFoldlerList = getFolderList(theBaseMusicLibraryFolder)
>myArtistCount = 0
>for myArtistFolder in myArtistsFoldlerList:
>print('Artist: ' + myArtistFolder)
> #
> #  Loop thru Albums Folder
> #
>myAlbumList = getFolderList(theBaseMusicLibraryFolder + myArtistFolder)
>for myAlbum in myAlbumList:
>print('Album: ' + myAlbum)
> 
> #
> #  Loop thru Tracks (Files) Folder
> #
>myAlbumPath = theBaseMusicLibraryFolder + myArtistFolder + '/' + 
> myAlbum + '/'
>myFilesList = getFileList(myAlbumPath)
>for myFile in myFilesList:
>myFilePath = myAlbumPath + myFile
>myID3 = eyed3.load(myFilePath)
>if myID3 is None:
>continue
> 
>myArtistName = myID3.tag.artist
>if myArtistName is None:
>continue
> 
>myAlbumName = myID3.tag.album
>if myAlbumName is None:
>continue
> 
>myTitleName = myID3.tag.title
>if myTitleName is None:
>continue
> 
>myCompareFileName = myFile[0:-4]
>if myCompareFileName[0].isdigit() and 
> myCompareFileName[1].isdigit():
>myCompareFileName = myFile[3:-4]
> 
>if myCompareFileName != myTitleName:
>myLength1 = len(myCompareFileName)
>myLength2 = len(myTitleName)
>print('File Name Mismatch - Artist: [' + myArtistName + '] 
>  Album: ['+ myAlbumName + ']  Track: [' + myTitleName + ']  File: [' + 
> myCompareFileName + ']')
>if (myLength1 == myLength2):
>print('lengths match: ',myLength1)
>else:
>print('lengths mismatch: ',myLength1,'  ',myLength2)
> 
>print(' ')
> 
> 
> 
> 
>return myArtistsFoldlerList
> 
> 
> 
> 
> 
> 
>> On 8 Jun 2022, at 00:07, MRAB  wrote:
>> 
>> On 2022-06-07 21:23, Dave wrote:
>>> Thanks a lot for this! isDigit was the method I was looking for and 
>>> couldn’t find.
>>> I have another problem related to this, the following code uses the code 
>>> you just sent. I am getting a files ID3 tags using eyed3, this part seems 
>>> to work and I get expected values in this case myTitleName (Track name) is 
>>> set to “Deadlock Holiday” and myCompareFileName is set to “01 Deadlock 
>>> Holiday” (File Name with the Track number prepended). The is digit test 
>>> works and myCompareFileName is set to  “Deadlock Holiday”, so they should 
>>> match, right?
>> OT, but are you sure about that name? Isn't it "Dreadlock Holiday" (by 10cc)?
>> 
>> [snip]
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-08, Christian Gollwitzer  wrote:
> Am 07.06.22 um 23:01 schrieb Christian Gollwitzer:
>
>>> In [3]: re.sub(r'^\d+\s*', '', s) Out[3]: 'Trinket'
>>>
>
> that RE does match what you intended to do, but not exactly what you 
> wrote in the OP. that would be '^\d\d.'  start with exactly two digits 
> followed by any character.

Indeed but then I'd like '\d{2}' even better.


>   Christian


-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-08, Dave  wrote:
> I hate regEx and avoid it whenever possible, I’ve never found something that 
> was impossible to do without it.

I love regular expressions and use them where appropriate. Saves tons of
code and is often much more readable than the pages of code required to
do the same.

-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-08, dn  wrote:
> On 08/06/2022 10.18, De ongekruisigde wrote:
>> On 2022-06-08, Christian Gollwitzer  wrote:
>>> Am 07.06.22 um 21:56 schrieb Dave:
 It depends on the language I’m using, in Objective C, I’d use isNumeric, 
 just wanted to know what the equivalent is in Python.

>>>
>>> Your problem is also a typical case for regular expressions. You can 
>>> create an expression for "starts with any number of digits plus optional 
>>> whitespace" and then replace this with nothing:
>> 
>> Regular expressions are overkill for this and much slower than the
>> simple isdigit based solution.
>
> ...
>
>> Regular expressions are indeeed extremely powerful and useful but I tend
>> to avoid them when there's a (faster) normal solution.
>
> Yes, simple solutions are (likely) easier to read.

Depending on the problem a regular expression may be the much simpler
solution. I love them for e.g. text parsing and use them all the time.
Unrivaled when e.g. parts of text have to be extracted, e.g. from lines
like these:

  root:x:0:0:System administrator:/root:/run/current-system/sw/bin/bash
  dhcpcd:x:995:991::/var/empty:/run/current-system/sw/bin/nologin
  nm-iodine:x:996:57::/var/empty:/run/current-system/sw/bin/nologin
  avahi:x:997:996:avahi-daemon privilege separation 
user:/var/empty:/run/current-system/sw/bin/nologin
  sshd:x:998:993:SSH privilege separation 
user:/var/empty:/run/current-system/sw/bin/nologin
  geoclue:x:999:998:Geoinformation 
service:/var/lib/geoclue:/run/current-system/sw/bin/nologin

Compare a regexp solution like this:

  >>> g = re.search(r'([^:]*):([^:]*):(\d+):(\d+):([^:]*):([^:]*):(.*)$' , s)
  >>> print(g.groups())
  ('geoclue', 'x', '999', '998', 'Geoinformation service', '/var/lib/geoclue', 
'/run/current-system/sw/bin/nologin')

to the code one would require to process it manually, with all the edge
cases. The regexp surely reads much simpler (?).


> RegEx-s are more powerful (and well worth learning for this reason), but
> are only 'readable' to those who use them frequently.
>
> Has either of you performed a timeit comparison?

No need: the isdigit solution doesn't require the overhead of a regex
processor.

-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Avi Gross via Python-list]

Amazing how some people bring out the heavy artillery, first! LOL!

If the question was how to remove any initial digits and perhaps whitespace in 
a string, it is fairly easy to do without any functions to test if there are 
digits before the title. I mean look at initial characters and move forward if 
it is between '0' and '9' or a space. Duh!

Sure, a regular expression that matches anything following a run of digits and 
whitespace and before a ".MPG" or the end of the entry will be easy to extract 
and compare after removing any left/right whitespace in both things being 
compared and coercing both to the same case.

But the solution may be doomed to failure when it sees things like:
"100 Letters" 
"1+1" 
"10,000 hours"
"1 Trillion Dollar$" 
"2,000 Light Years From Home"


So is it necessary to insist on an exact pattern of two digits followed by a 
space? 


That would fail on "44 Minutes", "40 Oz. Dream", "50 Mission Cap", "50 Ways to 
Say Goodbye", "99 Ways to Die" 

It looks to me like you need to compare TWICE just in case. If it matches in 
the original (perhaps with some normalization of case and whitespace, fine. If 
not will they match if one or both have something to remove as a prefix such as 
"02 ". And if you are comparing items where the same song is in two different 
numeric sequences on different disks, ...




-----Original Message-
From: Christian Gollwitzer 
To: python-list@python.org
Sent: Tue, Jun 7, 2022 6:01 pm
Subject: Re: How to test characters of a string

Am 07.06.22 um 21:56 schrieb Dave:
> It depends on the language I’m using, in Objective C, I’d use isNumeric, just 
> wanted to know what the equivalent is in Python.
> 

Your problem is also a typical case for regular expressions. You can 
create an expression for "starts with any number of digits plus optional 
whitespace" and then replace this with nothing:

> chris@linux-tb9f:~> ipython
> Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC]
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.
> 
> In [1]: import re                                                             
>                                                                               
>                              
> 
> In [2]: s='05 Trinket'                                                        
>                                                                               
>                             
> 
> In [3]: re.sub(r'^\d+\s*', '', s)                                             
>                                                                               
>                              
> Out[3]: 'Trinket'
> 

If it doesn't match, it will do nothing:

> In [4]: s='Es geht los'                                                       
>                                                                               
>                              
> 
> In [5]: re.sub(r'^\d+\s*', '', s)                                             
>                                                                               
>                              
> Out[5]: 'Es geht los'

Some people on this list don't like regexes but for tasks like this they 
are made and working well.

^ is "starts with"
\d is any digit
\s is any space
+ is at least one
* is nothing or one of

Christian




-- 
https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[MRAB]

On 2022-06-07 23:24, Dave wrote:

Yes, it was probably just a typeo on my part.


You've misspelled "typo"!


I’ve now fixed the majority of cases but still got two strings that look 
identical but fail to match, this time (again by 10cc), “I’m Mandy Fly Me”.


Try printing the asciified string:

>>> print(ascii("I’m Mandy Fly Me"))
'I\u2019m Mandy Fly Me'

What you typed above has "smart quotes". Maybe that's also the problem 
in the program: straight single quote/apodtrophe vs smart single quote.



I’m putting money on it being a utf8 problem but I’m stuck on how to handle it. 
It’s probably the single quote in I’m, although it has worked with other songs.

Any ideas?

All the Best
Cheers
Dave

Here is the whole function/method or whatever it’s called in Python:


#
#   checkMusicFiles
#

def checkMusicFiles(theBaseMusicLibraryFolder):
 myArtistDict = []

#
#  Loop thru Artists Folder
#
 myArtistsFoldlerList = getFolderList(theBaseMusicLibraryFolder)
 myArtistCount = 0
 for myArtistFolder in myArtistsFoldlerList:
 print('Artist: ' + myArtistFolder)
#
#  Loop thru Albums Folder
#
 myAlbumList = getFolderList(theBaseMusicLibraryFolder + myArtistFolder)
 for myAlbum in myAlbumList:
 print('Album: ' + myAlbum)

#
#  Loop thru Tracks (Files) Folder
#
 myAlbumPath = theBaseMusicLibraryFolder + myArtistFolder + '/' + 
myAlbum + '/'
 myFilesList = getFileList(myAlbumPath)
 for myFile in myFilesList:
 myFilePath = myAlbumPath + myFile
 myID3 = eyed3.load(myFilePath)
 if myID3 is None:
 continue

 myArtistName = myID3.tag.artist
 if myArtistName is None:
 continue

 myAlbumName = myID3.tag.album
 if myAlbumName is None:
 continue

 myTitleName = myID3.tag.title
 if myTitleName is None:
 continue

 myCompareFileName = myFile[0:-4]
 if myCompareFileName[0].isdigit() and 
myCompareFileName[1].isdigit():
 myCompareFileName = myFile[3:-4]

 if myCompareFileName != myTitleName:
 myLength1 = len(myCompareFileName)
 myLength2 = len(myTitleName)
 print('File Name Mismatch - Artist: [' + myArtistName + '] 
 Album: ['+ myAlbumName + ']  Track: [' + myTitleName + ']  File: [' + 
myCompareFileName + ']')
 if (myLength1 == myLength2):
 print('lengths match: ',myLength1)
 else:
 print('lengths mismatch: ',myLength1,'  ',myLength2)

 print(' ')




 return myArtistsFoldlerList


"myArtistsFoldlerList"?

And so many variables starting with "my". Not wrong; just ... :-)








On 8 Jun 2022, at 00:07, MRAB  wrote:

On 2022-06-07 21:23, Dave wrote:

Thanks a lot for this! isDigit was the method I was looking for and couldn’t 
find.
I have another problem related to this, the following code uses the code you 
just sent. I am getting a files ID3 tags using eyed3, this part seems to work 
and I get expected values in this case myTitleName (Track name) is set to 
“Deadlock Holiday” and myCompareFileName is set to “01 Deadlock Holiday” (File 
Name with the Track number prepended). The is digit test works and 
myCompareFileName is set to  “Deadlock Holiday”, so they should match, right?

OT, but are you sure about that name? Isn't it "Dreadlock Holiday" (by 10cc)?

[snip]




--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dave]

I hate regEx and avoid it whenever possible, I’ve never found something that 
was impossible to do without it.

> On 8 Jun 2022, at 00:49, dn  wrote:
> 
> On 08/06/2022 10.18, De ongekruisigde wrote:
>> On 2022-06-08, Christian Gollwitzer  wrote:
>>> Am 07.06.22 um 21:56 schrieb Dave:
 It depends on the language I’m using, in Objective C, I’d use isNumeric, 
 just wanted to know what the equivalent is in Python.
 
>>> 
>>> Your problem is also a typical case for regular expressions. You can 
>>> create an expression for "starts with any number of digits plus optional 
>>> whitespace" and then replace this with nothing:
>> 
>> Regular expressions are overkill for this and much slower than the
>> simple isdigit based solution.
> 
> ...
> 
>> Regular expressions are indeeed extremely powerful and useful but I tend
>> to avoid them when there's a (faster) normal solution.
> 
> Yes, simple solutions are (likely) easier to read.
> 
> RegEx-s are more powerful (and well worth learning for this reason), but
> are only 'readable' to those who use them frequently.
> 
> Has either of you performed a timeit comparison?
> -- 
> Regards,
> =dn
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[dn]

On 08/06/2022 10.18, De ongekruisigde wrote:
> On 2022-06-08, Christian Gollwitzer  wrote:
>> Am 07.06.22 um 21:56 schrieb Dave:
>>> It depends on the language I’m using, in Objective C, I’d use isNumeric, 
>>> just wanted to know what the equivalent is in Python.
>>>
>>
>> Your problem is also a typical case for regular expressions. You can 
>> create an expression for "starts with any number of digits plus optional 
>> whitespace" and then replace this with nothing:
> 
> Regular expressions are overkill for this and much slower than the
> simple isdigit based solution.

...

> Regular expressions are indeeed extremely powerful and useful but I tend
> to avoid them when there's a (faster) normal solution.

Yes, simple solutions are (likely) easier to read.

RegEx-s are more powerful (and well worth learning for this reason), but
are only 'readable' to those who use them frequently.

Has either of you performed a timeit comparison?
-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dave]

rotfl! Nice one! 

> On 8 Jun 2022, at 00:24, 2qdxy4rzwzuui...@potatochowder.com wrote:
> 
> On 2022-06-07 at 23:07:42 +0100,
> Regarding "Re: How to test characters of a string,"
> MRAB  wrote:
> 
>> On 2022-06-07 21:23, Dave wrote:
>>> Thanks a lot for this! isDigit was the method I was looking for and 
>>> couldn’t find.
>>> 
>>> I have another problem related to this, the following code uses the code 
>>> you just sent. I am getting a files ID3 tags using eyed3, this part seems 
>>> to work and I get expected values in this case myTitleName (Track name) is 
>>> set to “Deadlock Holiday” and myCompareFileName is set to “01 Deadlock 
>>> Holiday” (File Name with the Track number prepended). The is digit test 
>>> works and myCompareFileName is set to  “Deadlock Holiday”, so they should 
>>> match, right?
>>> 
>> OT, but are you sure about that name? Isn't it "Dreadlock Holiday" (by
>> 10cc)?
> 
> Edsger Dijkstra originally wrote Deadlock Holiday for his band, The
> Semaphores.  10cc lost the race condition and had to change the lyrics.
> 
> Sorry.
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dave]

Yes, it was probably just a typeo on my part.

I’ve now fixed the majority of cases but still got two strings that look 
identical but fail to match, this time (again by 10cc), “I’m Mandy Fly Me”.

I’m putting money on it being a utf8 problem but I’m stuck on how to handle it. 
It’s probably the single quote in I’m, although it has worked with other songs.

Any ideas?

All the Best
Cheers
Dave

Here is the whole function/method or whatever it’s called in Python:


#
#   checkMusicFiles
#

def checkMusicFiles(theBaseMusicLibraryFolder):
myArtistDict = []

#
#  Loop thru Artists Folder
#
myArtistsFoldlerList = getFolderList(theBaseMusicLibraryFolder)
myArtistCount = 0
for myArtistFolder in myArtistsFoldlerList:
print('Artist: ' + myArtistFolder)
#
#  Loop thru Albums Folder
#
myAlbumList = getFolderList(theBaseMusicLibraryFolder + myArtistFolder)
for myAlbum in myAlbumList:
print('Album: ' + myAlbum)

#
#  Loop thru Tracks (Files) Folder
#
myAlbumPath = theBaseMusicLibraryFolder + myArtistFolder + '/' + 
myAlbum + '/'
myFilesList = getFileList(myAlbumPath)
for myFile in myFilesList:
myFilePath = myAlbumPath + myFile
myID3 = eyed3.load(myFilePath)
if myID3 is None:
continue

myArtistName = myID3.tag.artist
if myArtistName is None:
continue

myAlbumName = myID3.tag.album
if myAlbumName is None:
continue

myTitleName = myID3.tag.title
if myTitleName is None:
continue

myCompareFileName = myFile[0:-4]
if myCompareFileName[0].isdigit() and 
myCompareFileName[1].isdigit():
myCompareFileName = myFile[3:-4]

if myCompareFileName != myTitleName:
myLength1 = len(myCompareFileName)
myLength2 = len(myTitleName)
print('File Name Mismatch - Artist: [' + myArtistName + ']  
Album: ['+ myAlbumName + ']  Track: [' + myTitleName + ']  File: [' + 
myCompareFileName + ']')
if (myLength1 == myLength2):
print('lengths match: ',myLength1)
else:
print('lengths mismatch: ',myLength1,'  ',myLength2)

print(' ')




return myArtistsFoldlerList






> On 8 Jun 2022, at 00:07, MRAB  wrote:
> 
> On 2022-06-07 21:23, Dave wrote:
>> Thanks a lot for this! isDigit was the method I was looking for and couldn’t 
>> find.
>> I have another problem related to this, the following code uses the code you 
>> just sent. I am getting a files ID3 tags using eyed3, this part seems to 
>> work and I get expected values in this case myTitleName (Track name) is set 
>> to “Deadlock Holiday” and myCompareFileName is set to “01 Deadlock Holiday” 
>> (File Name with the Track number prepended). The is digit test works and 
>> myCompareFileName is set to  “Deadlock Holiday”, so they should match, right?
> OT, but are you sure about that name? Isn't it "Dreadlock Holiday" (by 10cc)?
> 
> [snip]
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[dn]

 It depends on the language I’m using, in Objective C, I’d use isNumeric, 
 just wanted to know what the equivalent is in Python.

 If you know the answer why don’t you just tell me and if you don’t, don’t 
 post!
>>>
>>> People ask home work questions here and we try to teach a student with 
>>> hints not finished answers.
>>> Your post was confused with a home work question.
>>
>> In the future, to make it look less like a homework question, show
>> your current code, which would provide context. Last I checked,
>> homework questions don't usually involve ID3 tags in MP3 files :)

Ah, so that's where I've seen it before!
(thanks for scratching my head @Chris - but watch-out for splinters!)

Yes, the problem has been used as a training exercise, eg same song but
in different albums/play-lists, different capitalisation, and such-like;
ie 'data cleaning' and harmonisation - good for use at the intersection
of Python and SQL (or NoSQL).


Knowing the background, and thus the particular need, would have saved a
lot of time - giving the answer as code (per one of the contributions)
would have taken considerably less effort than looking-up and citing the
docs.

Perhaps then, the 'learning-opportunity' is that when such questions pop
into one's mind, 'the docs' is *the* recommended first-call?


> The original question in this thread didn't say anything about MP3
> files.  Jumping to that conclusion from strings like '05 Trinket' was
> left as an exercise for the interested reader.  :-)

This reader's interest was to figure-out why "trinket" didn't refer to
some small decoration or 'bling', nor to a Python training tool
(https://trinket.io/), but to a music group/video series.
(even more-surprising: that this grey-beard recognised one of their tracks).


On the other side of the relationship, writers are expected to follow
the PSF Code of Conduct (https://www.python.org/psf/conduct/), eg
respect, acknowledgement, grace...

Such also encourages (positive) responses when asking future questions...


Now that you (@Dave) have revealed yourself as more than a raw-beginner,
and to have skills transferable to the Python world, it'll be great to
see you 'here', contributing to others' posts...
-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-08, Christian Gollwitzer  wrote:
> Am 07.06.22 um 21:56 schrieb Dave:
>> It depends on the language I’m using, in Objective C, I’d use isNumeric, 
>> just wanted to know what the equivalent is in Python.
>> 
>
> Your problem is also a typical case for regular expressions. You can 
> create an expression for "starts with any number of digits plus optional 
> whitespace" and then replace this with nothing:

Regular expressions are overkill for this and much slower than the
simple isdigit based solution.


>> chris@linux-tb9f:~> ipython
>> Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC]
>> Type 'copyright', 'credits' or 'license' for more information
>> IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.
>> 
>> In [1]: import re
>>  
>>
>> 
>> In [2]: s='05 Trinket'   
>>  
>>
>> 
>> In [3]: re.sub(r'^\d+\s*', '', s)
>>  
>>
>> Out[3]: 'Trinket'
>> 
>
> If it doesn't match, it will do nothing:
>
>> In [4]: s='Es geht los'  
>>  
>>
>> 
>> In [5]: re.sub(r'^\d+\s*', '', s)
>>  
>>
>> Out[5]: 'Es geht los'
>
> Some people on this list don't like regexes but for tasks like this they 
> are made and working well.

Regular expressions are indeeed extremely powerful and useful but I tend
to avoid them when there's a (faster) normal solution.


> ^ is "starts with"
> \d is any digit
> \s is any space
> + is at least one
> * is nothing or one of
>
> Christian

-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Christian Gollwitzer]

Am 07.06.22 um 23:01 schrieb Christian Gollwitzer:


In [3]: re.sub(r'^\d+\s*', '', s) Out[3]: 'Trinket'



that RE does match what you intended to do, but not exactly what you 
wrote in the OP. that would be '^\d\d.'  start with exactly two digits 
followed by any character.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Christian Gollwitzer]

Am 07.06.22 um 21:56 schrieb Dave:

It depends on the language I’m using, in Objective C, I’d use isNumeric, just 
wanted to know what the equivalent is in Python.



Your problem is also a typical case for regular expressions. You can 
create an expression for "starts with any number of digits plus optional 
whitespace" and then replace this with nothing:



chris@linux-tb9f:~> ipython
Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import re

In [2]: s='05 Trinket'   

In [3]: re.sub(r'^\d+\s*', '', s)
Out[3]: 'Trinket'




If it doesn't match, it will do nothing:

In [4]: s='Es geht los'  

In [5]: re.sub(r'^\d+\s*', '', s)
Out[5]: 'Es geht los'


Some people on this list don't like regexes but for tasks like this they 
are made and working well.


^ is "starts with"
\d is any digit
\s is any space
+ is at least one
* is nothing or one of

Christian




--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[2QdxY4RzWzUUiLuE]

On 2022-06-07 at 23:07:42 +0100,
Regarding "Re: How to test characters of a string,"
MRAB  wrote:

> On 2022-06-07 21:23, Dave wrote:
> > Thanks a lot for this! isDigit was the method I was looking for and 
> > couldn’t find.
> > 
> > I have another problem related to this, the following code uses the code 
> > you just sent. I am getting a files ID3 tags using eyed3, this part seems 
> > to work and I get expected values in this case myTitleName (Track name) is 
> > set to “Deadlock Holiday” and myCompareFileName is set to “01 Deadlock 
> > Holiday” (File Name with the Track number prepended). The is digit test 
> > works and myCompareFileName is set to  “Deadlock Holiday”, so they should 
> > match, right?
> > 
> OT, but are you sure about that name? Isn't it "Dreadlock Holiday" (by
> 10cc)?

Edsger Dijkstra originally wrote Deadlock Holiday for his band, The
Semaphores.  10cc lost the race condition and had to change the lyrics.

Sorry.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[MRAB]

On 2022-06-07 21:23, Dave wrote:

Thanks a lot for this! isDigit was the method I was looking for and couldn’t 
find.

I have another problem related to this, the following code uses the code you 
just sent. I am getting a files ID3 tags using eyed3, this part seems to work 
and I get expected values in this case myTitleName (Track name) is set to 
“Deadlock Holiday” and myCompareFileName is set to “01 Deadlock Holiday” (File 
Name with the Track number prepended). The is digit test works and 
myCompareFileName is set to  “Deadlock Holiday”, so they should match, right?

OT, but are you sure about that name? Isn't it "Dreadlock Holiday" (by 
10cc)?


[snip]
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[2QdxY4RzWzUUiLuE]

On 2022-06-08 at 07:29:03 +1000,
Chris Angelico  wrote:

> On Wed, 8 Jun 2022 at 07:24, Barry  wrote:
> >
> >
> >
> > > On 7 Jun 2022, at 22:04, Dave  wrote:
> > >
> > > It depends on the language I’m using, in Objective C, I’d use isNumeric, 
> > > just wanted to know what the equivalent is in Python.
> > >
> > > If you know the answer why don’t you just tell me and if you don’t, don’t 
> > > post!
> >
> > People ask home work questions here and we try to teach a student with 
> > hints not finished answers.
> > Your post was confused with a home work question.
> >
> 
> In the future, to make it look less like a homework question, show
> your current code, which would provide context. Last I checked,
> homework questions don't usually involve ID3 tags in MP3 files :)

The original question in this thread didn't say anything about MP3
files.  Jumping to that conclusion from strings like '05 Trinket' was
left as an exercise for the interested reader.  :-)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dave]

A, ok will do, was just trying to be a brief as possible, will post more 
fully in future.

> On 7 Jun 2022, at 23:29, Chris Angelico  wrote:
> 
> On Wed, 8 Jun 2022 at 07:24, Barry  wrote:
>> 
>> 
>> 
>>> On 7 Jun 2022, at 22:04, Dave  wrote:
>>> 
>>> It depends on the language I’m using, in Objective C, I’d use isNumeric, 
>>> just wanted to know what the equivalent is in Python.
>>> 
>>> If you know the answer why don’t you just tell me and if you don’t, don’t 
>>> post!
>> 
>> People ask home work questions here and we try to teach a student with hints 
>> not finished answers.
>> Your post was confused with a home work question.
>> 
> 
> In the future, to make it look less like a homework question, show
> your current code, which would provide context. Last I checked,
> homework questions don't usually involve ID3 tags in MP3 files :)
> 
> ChrisA
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Chris Angelico]

On Wed, 8 Jun 2022 at 07:24, Barry  wrote:
>
>
>
> > On 7 Jun 2022, at 22:04, Dave  wrote:
> >
> > It depends on the language I’m using, in Objective C, I’d use isNumeric, 
> > just wanted to know what the equivalent is in Python.
> >
> > If you know the answer why don’t you just tell me and if you don’t, don’t 
> > post!
>
> People ask home work questions here and we try to teach a student with hints 
> not finished answers.
> Your post was confused with a home work question.
>

In the future, to make it look less like a homework question, show
your current code, which would provide context. Last I checked,
homework questions don't usually involve ID3 tags in MP3 files :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Barry]

> On 7 Jun 2022, at 22:04, Dave  wrote:
> 
> It depends on the language I’m using, in Objective C, I’d use isNumeric, 
> just wanted to know what the equivalent is in Python.
> 
> If you know the answer why don’t you just tell me and if you don’t, don’t 
> post!

People ask home work questions here and we try to teach a student with hints 
not finished answers.
Your post was confused with a home work question.

Barry

> 
> 
>> On 7 Jun 2022, at 22:08, 2qdxy4rzwzuui...@potatochowder.com wrote:
>> 
>>> On 2022-06-07 at 21:35:43 +0200,
>>> Dave  wrote:
>>> 
>>> I’m new to Python and have a simple problem that I can’t seem to find
>>> the answer.
>> 
>>> I want to test the first two characters of a string to check if the
>>> are numeric (00 to 99) and if so remove the fist three chars from the
>>> string.
>> 
>>> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want
>>> “Trinket”. I can’t for the life of work out how to do it in Python?
>> 
>> How would you do it without Python?
>> 
>> Given that if the string is called x, then x[y] is the y'th character
>> (where what you would call "the first character," Python calls "the
>> zeroth character"), describe the steps you would take *as a person* (or
>> in some other programming language, if you know one) to carry out this
>> task.
>> 
>> Translating that algorithm to Python is the next step.  Perhaps
>> https://docs.python.org/3/library/string.html can help.
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dave]

Hi,

Found it! The files name had .mp3 at the end, the problem was being masked by 
null objects (or whatever) being returned by eyed3.

Checked for null objects and then stripped off the .mp3 and its mostly working 
now. I’ve got a few other eyed3 errors to do with null objects but I can sort 
those out tomorrow.

Thanks for your help - All the Best
Dave

> On 7 Jun 2022, at 23:01, Dave  wrote:
> 
> Hi,
> 
> No, I’ve checked leading/trailing whitespace, it seems to be related to the 
> variables that are returned from eyed3 in this case, for instance, I added a 
> check for None:
> myTitleName = myID3.tag.title
> if myTitleName is None:
>continue
> Seems like it can return a null object (or none?).
> 
> 
>> On 7 Jun 2022, at 22:35, De ongekruisigde 
>> > > wrote:
>> 
>> On 2022-06-07, Dave >  > >> wrote:
>>> Thanks a lot for this! isDigit was the method I was looking for and 
>>> couldn’t find.
>>> 
>>> I have another problem related to this, the following code uses the code 
>>> you just sent. I am getting a files ID3 tags using eyed3, this part seems 
>>> to work and I get expected values in this case myTitleName (Track name) is 
>>> set to “Deadlock Holiday” and myCompareFileName is set to “01 Deadlock 
>>> Holiday” (File Name with the Track number prepended). The is digit test 
>>> works and myCompareFileName is set to  “Deadlock Holiday”, so they should 
>>> match, right? 
>>> 
>>> However the if myCompareFileName != myTitleName always gives a mismatch! 
>>> What could cause two string that look the fail to not match properly?
>> 
>> Possibly leading or trailing spaces, or upper/lower case differences?
>> 
>> 
>>> myCompareFileName = myFile
>>> if myCompareFileName[0].isdigit() and myCompareFileName[1].isdigit():
>>>   myCompareFileName = myCompareFileName[3:]
>>> 
>>> if myCompareFileName != myTitleName:
>>>   print('File Name Mismatch - Artist: ',myArtistName,'  Album: 
>>> ',myAlbumName,'  Track:',myTitleName,'  File: ',myFile)
>>> Thanks a lot
>>> Dave
>>> 
 On 7 Jun 2022, at 21:58, De ongekruisigde 
  wrote:
 
 On 2022-06-07, Dave  wrote:
> Hi,
> 
> I’m new to Python and have a simple problem that I can’t seem to find the 
> answer.
> 
> I want to test the first two characters of a string to check if the are 
> numeric (00 to 99) and if so remove the fist three chars from the string. 
> 
> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want 
> “Trinket”. I can’t for the life of work out how to do it in Python?
 
 
 s[3:] if s[0:2].isdigit() else s
 
 
> All the Best
> Dave
> 
 
 -- 
  You're rewriting parts of Quake in *Python*?
  MUAHAHAHA
 -- 
 https://mail.python.org/mailman/listinfo/python-list
>>> 
>> 
>> 
>> -- 
>>  You're rewriting parts of Quake in *Python*?
>>  MUAHAHAHA
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list 
>>  
>> > >
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dave]

Hi,

No, I’ve checked leading/trailing whitespace, it seems to be related to the 
variables that are returned from eyed3 in this case, for instance, I added a 
check for None:
myTitleName = myID3.tag.title
if myTitleName is None:
continue
Seems like it can return a null object (or none?).

 
> On 7 Jun 2022, at 22:35, De ongekruisigde 
>  wrote:
> 
> On 2022-06-07, Dave  > wrote:
>> Thanks a lot for this! isDigit was the method I was looking for and couldn’t 
>> find.
>> 
>> I have another problem related to this, the following code uses the code you 
>> just sent. I am getting a files ID3 tags using eyed3, this part seems to 
>> work and I get expected values in this case myTitleName (Track name) is set 
>> to “Deadlock Holiday” and myCompareFileName is set to “01 Deadlock Holiday” 
>> (File Name with the Track number prepended). The is digit test works and 
>> myCompareFileName is set to  “Deadlock Holiday”, so they should match, 
>> right? 
>> 
>> However the if myCompareFileName != myTitleName always gives a mismatch! 
>> What could cause two string that look the fail to not match properly?
> 
> Possibly leading or trailing spaces, or upper/lower case differences?
> 
> 
>> myCompareFileName = myFile
>> if myCompareFileName[0].isdigit() and myCompareFileName[1].isdigit():
>>myCompareFileName = myCompareFileName[3:]
>> 
>> if myCompareFileName != myTitleName:
>>print('File Name Mismatch - Artist: ',myArtistName,'  Album: 
>> ',myAlbumName,'  Track:',myTitleName,'  File: ',myFile)
>> Thanks a lot
>> Dave
>> 
>>> On 7 Jun 2022, at 21:58, De ongekruisigde 
>>>  wrote:
>>> 
>>> On 2022-06-07, Dave  wrote:
 Hi,
 
 I’m new to Python and have a simple problem that I can’t seem to find the 
 answer.
 
 I want to test the first two characters of a string to check if the are 
 numeric (00 to 99) and if so remove the fist three chars from the string. 
 
 Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want 
 “Trinket”. I can’t for the life of work out how to do it in Python?
>>> 
>>> 
>>> s[3:] if s[0:2].isdigit() else s
>>> 
>>> 
 All the Best
 Dave
 
>>> 
>>> -- 
>>>  You're rewriting parts of Quake in *Python*?
>>>  MUAHAHAHA
>>> -- 
>>> https://mail.python.org/mailman/listinfo/python-list
>> 
> 
> 
> -- 
>  You're rewriting parts of Quake in *Python*?
>  MUAHAHAHA
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dave]

It depends on the language I’m using, in Objective C, I’d use isNumeric, just 
wanted to know what the equivalent is in Python.

If you know the answer why don’t you just tell me and if you don’t, don’t post!


> On 7 Jun 2022, at 22:08, 2qdxy4rzwzuui...@potatochowder.com wrote:
> 
> On 2022-06-07 at 21:35:43 +0200,
> Dave  wrote:
> 
>> I’m new to Python and have a simple problem that I can’t seem to find
>> the answer.
> 
>> I want to test the first two characters of a string to check if the
>> are numeric (00 to 99) and if so remove the fist three chars from the
>> string.
> 
>> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want
>> “Trinket”. I can’t for the life of work out how to do it in Python?
> 
> How would you do it without Python?
> 
> Given that if the string is called x, then x[y] is the y'th character
> (where what you would call "the first character," Python calls "the
> zeroth character"), describe the steps you would take *as a person* (or
> in some other programming language, if you know one) to carry out this
> task.
> 
> Translating that algorithm to Python is the next step.  Perhaps
> https://docs.python.org/3/library/string.html can help.
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-07, Dave  wrote:
> Thanks a lot for this! isDigit was the method I was looking for and couldn’t 
> find.
>
> I have another problem related to this, the following code uses the code you 
> just sent. I am getting a files ID3 tags using eyed3, this part seems to work 
> and I get expected values in this case myTitleName (Track name) is set to 
> “Deadlock Holiday” and myCompareFileName is set to “01 Deadlock Holiday” 
> (File Name with the Track number prepended). The is digit test works and 
> myCompareFileName is set to  “Deadlock Holiday”, so they should match, right? 
>
> However the if myCompareFileName != myTitleName always gives a mismatch! What 
> could cause two string that look the fail to not match properly?

Possibly leading or trailing spaces, or upper/lower case differences?


> myCompareFileName = myFile
> if myCompareFileName[0].isdigit() and myCompareFileName[1].isdigit():
> myCompareFileName = myCompareFileName[3:]
>
> if myCompareFileName != myTitleName:
> print('File Name Mismatch - Artist: ',myArtistName,'  Album: 
> ',myAlbumName,'  Track:',myTitleName,'  File: ',myFile)
> Thanks a lot
> Dave
>
>> On 7 Jun 2022, at 21:58, De ongekruisigde 
>>  wrote:
>> 
>> On 2022-06-07, Dave  wrote:
>>> Hi,
>>> 
>>> I’m new to Python and have a simple problem that I can’t seem to find the 
>>> answer.
>>> 
>>> I want to test the first two characters of a string to check if the are 
>>> numeric (00 to 99) and if so remove the fist three chars from the string. 
>>> 
>>> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want 
>>> “Trinket”. I can’t for the life of work out how to do it in Python?
>> 
>> 
>>  s[3:] if s[0:2].isdigit() else s
>> 
>> 
>>> All the Best
>>> Dave
>>> 
>> 
>> -- 
>>  You're rewriting parts of Quake in *Python*?
>>  MUAHAHAHA
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>


-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-07, Stefan Ram  wrote:
> Dave  writes:
>>Example: if "05 Trinket" I want "Trinket"
>
>   We're not supposed to write complete solutions, 

Okay, wasn't aware of this group policy; will keep it in mind.

-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[2QdxY4RzWzUUiLuE]

On 2022-06-07 at 21:35:43 +0200,
Dave  wrote:

> I’m new to Python and have a simple problem that I can’t seem to find
> the answer.

> I want to test the first two characters of a string to check if the
> are numeric (00 to 99) and if so remove the fist three chars from the
> string.

> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want
> “Trinket”. I can’t for the life of work out how to do it in Python?

How would you do it without Python?

Given that if the string is called x, then x[y] is the y'th character
(where what you would call "the first character," Python calls "the
zeroth character"), describe the steps you would take *as a person* (or
in some other programming language, if you know one) to carry out this
task.

Translating that algorithm to Python is the next step.  Perhaps
https://docs.python.org/3/library/string.html can help.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[Dave]

Thanks a lot for this! isDigit was the method I was looking for and couldn’t 
find.

I have another problem related to this, the following code uses the code you 
just sent. I am getting a files ID3 tags using eyed3, this part seems to work 
and I get expected values in this case myTitleName (Track name) is set to 
“Deadlock Holiday” and myCompareFileName is set to “01 Deadlock Holiday” (File 
Name with the Track number prepended). The is digit test works and 
myCompareFileName is set to  “Deadlock Holiday”, so they should match, right? 

However the if myCompareFileName != myTitleName always gives a mismatch! What 
could cause two string that look the fail to not match properly?
myCompareFileName = myFile
if myCompareFileName[0].isdigit() and myCompareFileName[1].isdigit():
myCompareFileName = myCompareFileName[3:]

if myCompareFileName != myTitleName:
print('File Name Mismatch - Artist: ',myArtistName,'  Album: 
',myAlbumName,'  Track:',myTitleName,'  File: ',myFile)
Thanks a lot
Dave

> On 7 Jun 2022, at 21:58, De ongekruisigde 
>  wrote:
> 
> On 2022-06-07, Dave  wrote:
>> Hi,
>> 
>> I’m new to Python and have a simple problem that I can’t seem to find the 
>> answer.
>> 
>> I want to test the first two characters of a string to check if the are 
>> numeric (00 to 99) and if so remove the fist three chars from the string. 
>> 
>> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want 
>> “Trinket”. I can’t for the life of work out how to do it in Python?
> 
> 
>  s[3:] if s[0:2].isdigit() else s
> 
> 
>> All the Best
>> Dave
>> 
> 
> -- 
>  You're rewriting parts of Quake in *Python*?
>  MUAHAHAHA
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[De ongekruisigde]

On 2022-06-07, Dave  wrote:
> Hi,
>
> I’m new to Python and have a simple problem that I can’t seem to find the 
> answer.
>
> I want to test the first two characters of a string to check if the are 
> numeric (00 to 99) and if so remove the fist three chars from the string. 
>
> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want 
> “Trinket”. I can’t for the life of work out how to do it in Python?


  s[3:] if s[0:2].isdigit() else s


> All the Best
> Dave
>

-- 
 You're rewriting parts of Quake in *Python*?
 MUAHAHAHA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test characters of a string

[dn]

On 08/06/2022 07.35, Dave wrote:
> Hi,
> 
> I’m new to Python and have a simple problem that I can’t seem to find the 
> answer.

> I want to test the first two characters of a string to check if the are 
> numeric (00 to 99) and if so remove the fist three chars from the string. 
> 
> Example: if “05 Trinket” I want “Trinket”, but “Trinket” I still want 
> “Trinket”. I can’t for the life of work out how to do it in Python?

This sounds like an assignment or quiz-question. We could provide an/the
answer - but then you wouldn't learn how to solve it for yourself...


There is a gentle introduction to "slicing" (taking the first two
characters) at
https://docs.python.org/3/tutorial/introduction.html?highlight=slicing

Characters can be turned into int[egers], as discussed at
https://docs.python.org/3/library/stdtypes.html#typesnumeric

Of course precautions must be taken in case the string is not an
integer, eg https://docs.python.org/3/tutorial/errors.html?highlight=except

Another approach might be to use isnumeric(),
https://docs.python.org/3/library/stdtypes.html?highlight=isnum


NB the first URL-pointer is a page in the Python Tutorial. Reading that
in its entirety may be a good investment of time!

Are you aware of the Python Tutor list?

-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test input via subprocess.Popen with data from file

[Tobiah]

Why not just have scripts that echo out the various sets of test
data you are interested in?  That way, Popen would
always be your interface and you wouldn't have to
make two cases in the consumer script.

In other words, make program that outputs test
data just like your main data source program.
Then the consumer would only have to work in one way.







On 3/10/22 04:16, Loris Bennett wrote:

Hi,

I have a command which produces output like the
following:

   Job ID: 9431211
   Cluster: curta
   User/Group: build/staff
   State: COMPLETED (exit code 0)
   Nodes: 1
   Cores per node: 8
   CPU Utilized: 01:30:53
   CPU Efficiency: 83.63% of 01:48:40 core-walltime
   Job Wall-clock time: 00:13:35
   Memory Utilized: 6.45 GB
   Memory Efficiency: 80.68% of 8.00 GB

I want to parse this and am using subprocess.Popen and accessing the
contents via Popen.stdout.  However, for testing purposes I want to save
various possible outputs of the command as text files and use those as
inputs.

What format should I use to pass data to the actual parsing function?

I could in both production and test convert the entire input to a string
and pass the string to the parsing method.

However, I could use something like

test_input_01 = subprocess.Popen(
 ["cat test_input_01.txt"],
 stdout=subprocess.PIPE,
 )
   
for the test input and then pass a Popen object to the parsing function.


Any comments on these alternative or suggestions for doing something
completely different?

Cheers,

Loris
  


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test input via subprocess.Popen with data from file

[Roel Schroeven]

Op 11/03/2022 om 10:11 schreef Roel Schroeven:

Op 10/03/2022 om 13:16 schreef Loris Bennett:

Hi,

I have a command which produces output like the
following:

   Job ID: 9431211
   Cluster: curta
   User/Group: build/staff
   State: COMPLETED (exit code 0)
   Nodes: 1
   Cores per node: 8
   CPU Utilized: 01:30:53
   CPU Efficiency: 83.63% of 01:48:40 core-walltime
   Job Wall-clock time: 00:13:35
   Memory Utilized: 6.45 GB
   Memory Efficiency: 80.68% of 8.00 GB

I want to parse this and am using subprocess.Popen and accessing the
contents via Popen.stdout.  However, for testing purposes I want to save
various possible outputs of the command as text files and use those as
inputs.

What format should I use to pass data to the actual parsing function?

Is this a command you run, produces that output, and then stops (as 
opposed to a long-running program that from time to time generates a 
bunch of output)?
Because in that case I would use subprocess.run() with 
capture_output=True instead of subprocess.Popen(). subprocess.run() 
returns a CompletedProcess instance wich has stdout and stderr members 
that contain the captured output as byte sequences or strings, 
depending on the parameters you passed.


So in your case I would simply read the content of each text file as a 
whole into a string, and use subprocess.run() to get the command's 
output also as a string. Then you can have a parse function that 
accepts such strings, and works exactly the same for the text files as 
for the command output. Your parse function can then use splitlines() 
to access the lines individually. The data size is very small so it's 
not a problem to have it all in memory at the same time (i.e. no need 
to worry about trying to stream it instead).



Very simple example:

    import subprocess
    from pprint import pprint

    def parse(state_data):
    lines = state_data.splitlines(keepends=False)
    state_dict = {}
    for line in lines:
    key, value = line.split(': ')
    state_dict[key] = value
    return state_dict

    def read_from_command():
    return subprocess.run(['./jobstate'], capture_output=True, 
check=True, encoding='UTF-8').stdout


    def read_from_file(fn):
    with open(fn, 'rt', encoding='UTF-8') as f:
    return f.read()

    pprint(parse(read_from_command()))
    pprint(parse(read_from_file('jobfile')))

--
"Iceland is the place you go to remind yourself that planet Earth is a
machine... and that all organic life that has ever existed amounts to a greasy
film that has survived on the exterior of that machine thanks to furious
improvisation."
-- Sam Hughes, Ra

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test input via subprocess.Popen with data from file

[Roel Schroeven]

Op 10/03/2022 om 13:16 schreef Loris Bennett:

Hi,

I have a command which produces output like the
following:

   Job ID: 9431211
   Cluster: curta
   User/Group: build/staff
   State: COMPLETED (exit code 0)
   Nodes: 1
   Cores per node: 8
   CPU Utilized: 01:30:53
   CPU Efficiency: 83.63% of 01:48:40 core-walltime
   Job Wall-clock time: 00:13:35
   Memory Utilized: 6.45 GB
   Memory Efficiency: 80.68% of 8.00 GB

I want to parse this and am using subprocess.Popen and accessing the
contents via Popen.stdout.  However, for testing purposes I want to save
various possible outputs of the command as text files and use those as
inputs.

What format should I use to pass data to the actual parsing function?

Is this a command you run, produces that output, and then stops (as 
opposed to a long-running program that from time to time generates a 
bunch of output)?
Because in that case I would use subprocess.run() with 
capture_output=True instead of subprocess.Popen(). subprocess.run() 
returns a CompletedProcess instance wich has stdout and stderr members 
that contain the captured output as byte sequences or strings, depending 
on the parameters you passed.


So in your case I would simply read the content of each text file as a 
whole into a string, and use subprocess.run() to get the command's 
output also as a string. Then you can have a parse function that accepts 
such strings, and works exactly the same for the text files as for the 
command output. Your parse function can then use splitlines() to access 
the lines individually. The data size is very small so it's not a 
problem to have it all in memory at the same time (i.e. no need to worry 
about trying to stream it instead).


--
"Don't Panic."
-- Douglas Adams, The Hitchhiker's Guide to the Galaxy

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test input via subprocess.Popen with data from file

[Dieter Maurer]

Loris Bennett wrote at 2022-3-11 07:40 +0100:
> ... I want to test the parsing ...
>Sorry if I was unclear but my question is:
>
>Given that the return value from Popen is a Popen object and given that
>the return value from reading a file is a single string or maybe a list
>of strings, what should the common format for the argument which is
>passed to the actual parsing function be?

What methods (of its input argument) does the parsing use?
If it uses `Popen` methods, then you pass a `POpen` object;
if it uses only (typical) file methods, then you pass a file object;
if it assumes its input to be a (line) interator, you pass
a (line) iterator (such as a "file" object).

I would design the parsing that it makes as few assumptions
about its input as possible -- to ease testing
and increase the chance for reuse.

That said, I would not design it to work with `Popen` objects
but likely to have a line iterator as input.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test input via subprocess.Popen with data from file

[Loris Bennett]

Dieter Maurer  writes:

> Loris Bennett wrote at 2022-3-10 13:16 +0100:
>>I have a command which produces output like the
>>following:
>>
>>  Job ID: 9431211
>>  Cluster: curta
>>  User/Group: build/staff
>>  State: COMPLETED (exit code 0)
>>  Nodes: 1
>>  Cores per node: 8
>>  CPU Utilized: 01:30:53
>>  CPU Efficiency: 83.63% of 01:48:40 core-walltime
>>  Job Wall-clock time: 00:13:35
>>  Memory Utilized: 6.45 GB
>>  Memory Efficiency: 80.68% of 8.00 GB
>>
>>I want to parse this and am using subprocess.Popen and accessing the
>>contents via Popen.stdout.  However, for testing purposes I want to save
>>various possible outputs of the command as text files and use those as
>>inputs.
>
> What do you want to test? the parsing? the "popen" interaction?
> You can separately test both tasks (I, at your place, would do this).

I just want to test the parsing.

> For the parsing test, it is not relevant that the actual text
> comes from an external process. You can directly read it from a file
> or have it in your text.

As mentioned in the original post, for the tests I indeed want to read
the input from files.

> In my view, you do not need a test for the `Popen` interaction:
> if it works once, it will work always.

Sorry if I was unclear but my question is:

Given that the return value from Popen is a Popen object and given that
the return value from reading a file is a single string or maybe a list
of strings, what should the common format for the argument which is
passed to the actual parsing function be?

Cheers,

Loris
-- 
This signature is currently under construction.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test input via subprocess.Popen with data from file

[Dieter Maurer]

Loris Bennett wrote at 2022-3-10 13:16 +0100:
>I have a command which produces output like the
>following:
>
>  Job ID: 9431211
>  Cluster: curta
>  User/Group: build/staff
>  State: COMPLETED (exit code 0)
>  Nodes: 1
>  Cores per node: 8
>  CPU Utilized: 01:30:53
>  CPU Efficiency: 83.63% of 01:48:40 core-walltime
>  Job Wall-clock time: 00:13:35
>  Memory Utilized: 6.45 GB
>  Memory Efficiency: 80.68% of 8.00 GB
>
>I want to parse this and am using subprocess.Popen and accessing the
>contents via Popen.stdout.  However, for testing purposes I want to save
>various possible outputs of the command as text files and use those as
>inputs.

What do you want to test? the parsing? the "popen" interaction?
You can separately test both tasks (I, at your place, would do this).

For the parsing test, it is not relevant that the actual text
comes from an external process. You can directly read it from a file
or have it in your text.

In my view, you do not need a test for the `Popen` interaction:
if it works once, it will work always.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for maildir 'folder' in Python?

[Barry]

> On 23 Jan 2022, at 17:08, Chris Green  wrote:
> 
> Barry Scott  wrote:
>> 
>> 
 On 22 Jan 2022, at 21:26, Chris Green  wrote:
>>> 
>>> I have a script that walks a quite deep tree of mail messages to find
>>> and archive old messages.  I'm trying to convert it from mbox to
>>> maildir (as I now store my mail in maildir format).
>>> 
>>> So I need to test whether a point I have reached in the hierarchy is a
>>> maildir mailbox or not.  Using mbox format it's easy because 'folders'
>>> are directories and mailboxes are files.  However with maildir the
>>> 'folders' have directories within them so the simple tree walking goes
>>> down a level too far and finds 'folders' which aren't mailboxes called
>>> 'cur', 'new' and 'tmp'.
>>> 
>>> Is there any 'ready made' way in python to tell whether a directory is
>>> a maildir mailbox?  If not I suppose I'll simply have to check if
>>> there are 'cur', 'new' and 'tmp' directories within the directory
>>> which may or may not be a maildir.
>> 
>> You do not need to walk the tree.
>> 
>> The structure is 
>>Maildir/cur, new, tmp - The INBOX
>>Maildir//cur, new, tmp
>> 
>> The encoding prefixed with a "." and uses a "." between folder name parts.
>> 
>> An example from my Maildir is ".Python.Users/" for a folder that client
>> shows as Python/Users
>> 
>> You can see this by doing:
>> 
>> ls -a ~/Maildir
>> 
> You're talking about one specific (and IMHO not very good) way of
> creating a maildir hierarchy.  Here's what I see on my system:

Oh right! That is imposed by dovecot that I use for the imap service.

Barry

>chris@esprimo$ cd mail
>/home/chris/mail
>chris@esprimo$ ls -a
>.  ..  Gm  In  Ju  Li  Tm  folder
>chris@esprimo$ ls -a folder
>. apexLodge entertainment  greville  houseHome  maxMum
> reshapers  tractor
>..archive   family hardware  internet   money 
> riding travel
>Scans boating   france healthisbd   pcc   
> software   ukraa
>Shopping  buySellAdmin  friendsholidays  jrml   personal  
> telecoms   vehicles
>chris@esprimo$ ls -a folder/Shopping
>.  cabin   cycling  garden maxine  pets 
> stationery
>.. car diy  giftsEtc   maxmum  photography  
> telecoms
>apexLodge  clothesEtc  electrical   grouponmedicineriding   
> toolstation
>boat   computerelectronics  houseHome  motorcycle  screwfix
>books  cpc food jrml   music   sportsEtc
> 
> No dot prefixes and my maildir hierarchy is a *real* directory hierarchy so
> all the other file management utilities on my system can be used easily.
> 
> 
> 
> -- 
> Chris Green
> ·
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for maildir 'folder' in Python?

[Chris Green]

Barry Scott  wrote:
> 
> 
> > On 22 Jan 2022, at 21:26, Chris Green  wrote:
> > 
> > I have a script that walks a quite deep tree of mail messages to find
> > and archive old messages.  I'm trying to convert it from mbox to
> > maildir (as I now store my mail in maildir format).
> > 
> > So I need to test whether a point I have reached in the hierarchy is a
> > maildir mailbox or not.  Using mbox format it's easy because 'folders'
> > are directories and mailboxes are files.  However with maildir the
> > 'folders' have directories within them so the simple tree walking goes
> > down a level too far and finds 'folders' which aren't mailboxes called
> > 'cur', 'new' and 'tmp'.
> > 
> > Is there any 'ready made' way in python to tell whether a directory is
> > a maildir mailbox?  If not I suppose I'll simply have to check if
> > there are 'cur', 'new' and 'tmp' directories within the directory
> > which may or may not be a maildir.
> 
> You do not need to walk the tree.
> 
> The structure is 
> Maildir/cur, new, tmp - The INBOX
> Maildir//cur, new, tmp
> 
> The encoding prefixed with a "." and uses a "." between folder name parts.
> 
> An example from my Maildir is ".Python.Users/" for a folder that client
> shows as Python/Users
> 
> You can see this by doing:
> 
>  ls -a ~/Maildir
> 
You're talking about one specific (and IMHO not very good) way of
creating a maildir hierarchy.  Here's what I see on my system:-

chris@esprimo$ cd mail
/home/chris/mail
chris@esprimo$ ls -a
.  ..  Gm  In  Ju  Li  Tm  folder
chris@esprimo$ ls -a folder
. apexLodge entertainment  greville  houseHome  maxMum
reshapers  tractor
..archive   family hardware  internet   money 
riding travel
Scans boating   france healthisbd   pcc   
software   ukraa
Shopping  buySellAdmin  friendsholidays  jrml   personal  
telecoms   vehicles
chris@esprimo$ ls -a folder/Shopping
.  cabin   cycling  garden maxine  pets 
stationery
.. car diy  giftsEtc   maxmum  photography  
telecoms
apexLodge  clothesEtc  electrical   grouponmedicineriding   
toolstation
boat   computerelectronics  houseHome  motorcycle  screwfix
books  cpc food jrml   music   sportsEtc

No dot prefixes and my maildir hierarchy is a *real* directory hierarchy so
all the other file management utilities on my system can be used easily.



-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for maildir 'folder' in Python?

[Chris Green]

Cameron Simpson  wrote:
> On 22Jan2022 21:26, Chris Green  wrote:
> >So I need to test whether a point I have reached in the hierarchy is a
> >maildir mailbox or not.  Using mbox format it's easy because 'folders'
> >are directories and mailboxes are files.  However with maildir the
> >'folders' have directories within them so the simple tree walking goes
> >down a level too far and finds 'folders' which aren't mailboxes called
> >'cur', 'new' and 'tmp'.
> >
> >Is there any 'ready made' way in python to tell whether a directory is
> >a maildir mailbox?  If not I suppose I'll simply have to check if
> >there are 'cur', 'new' and 'tmp' directories within the directory
> >which may or may not be a maildir.
> 
> I was hoping that the provided mailbox/Maildir class had something, but 
> apparently not.
> 
So was I! :-)  It seems strange that something isn't provided as it's
rather fundamental to handling maildir.


> So I suggest fetching my cs.mailutils module from PyPI and using its 
> ismaildir(path) function, which does exactly what you describe.
> 
Thanks Cameron.

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for maildir 'folder' in Python?

[Barry Scott]

> On 22 Jan 2022, at 21:26, Chris Green  wrote:
> 
> I have a script that walks a quite deep tree of mail messages to find
> and archive old messages.  I'm trying to convert it from mbox to
> maildir (as I now store my mail in maildir format).
> 
> So I need to test whether a point I have reached in the hierarchy is a
> maildir mailbox or not.  Using mbox format it's easy because 'folders'
> are directories and mailboxes are files.  However with maildir the
> 'folders' have directories within them so the simple tree walking goes
> down a level too far and finds 'folders' which aren't mailboxes called
> 'cur', 'new' and 'tmp'.
> 
> Is there any 'ready made' way in python to tell whether a directory is
> a maildir mailbox?  If not I suppose I'll simply have to check if
> there are 'cur', 'new' and 'tmp' directories within the directory
> which may or may not be a maildir.

You do not need to walk the tree.

The structure is 
Maildir/cur, new, tmp - The INBOX
Maildir//cur, new, tmp

The encoding prefixed with a "." and uses a "." between folder name parts.

An example from my Maildir is ".Python.Users/" for a folder that client
shows as Python/Users

You can see this by doing:

 ls -a ~/Maildir

Barry




> 
> -- 
> Chris Green
> ·
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for maildir 'folder' in Python?

[Cameron Simpson]

On 22Jan2022 21:26, Chris Green  wrote:
>So I need to test whether a point I have reached in the hierarchy is a
>maildir mailbox or not.  Using mbox format it's easy because 'folders'
>are directories and mailboxes are files.  However with maildir the
>'folders' have directories within them so the simple tree walking goes
>down a level too far and finds 'folders' which aren't mailboxes called
>'cur', 'new' and 'tmp'.
>
>Is there any 'ready made' way in python to tell whether a directory is
>a maildir mailbox?  If not I suppose I'll simply have to check if
>there are 'cur', 'new' and 'tmp' directories within the directory
>which may or may not be a maildir.

I was hoping that the provided mailbox/Maildir class had something, but 
apparently not.

So I suggest fetching my cs.mailutils module from PyPI and using its 
ismaildir(path) function, which does exactly what you describe.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Manfred Lotz]

On Wed, 17 Jun 2020 17:34:55 +0100
Tony Flury  wrote:

> On 24/04/2020 19:40, Manfred Lotz wrote:
> > I have a command like application which checks a directory tree for
> > certain things. If there are errors then messages will be written to
> > stdout.
> >
> > How to test this in the best way?
> >
> > One idea was for the error situations to write messages to files and
> > then later when running the tests to compare the error messages
> > output to the previously saved output.
> >
> > Is there anything better?
> >  
> 
> In a recent application that I wrote (where output to the console was 
> important), I tested it using the 'unittest' framework, and by
> patching sys.stderr to be a StringIO - that way my test case could
> inspect what was being output.
> 
> with patch('sys.stderr', StringIO()) as
> stderr:application.do_stuff()self.assertTrue(stderr.getvalue(),
> 'Woops - that didn\'t work')
> 
> I am not sure of the structure of your application, and whether you
> have a callable API that you can invoke.
> 

Thanks a lot. That's really great.

In the meantime I switched to pytest because things are so much more
convenient to setup. So, also thanks to the others who gave the links to
how this can be done with pytest.

-- 
Manfred

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Mats Wichmann]

On 5/1/20 2:34 PM, DL Neil via Python-list wrote:
>>> Given your replies, 'now' might be a good time to take a look at
>>> Pytest, and see how you could use it to help build better code - by
>>> building tested units/functions which are assembled into ever-larger
>>> tested-units... (there is a range of choice/other testing aids if
>>> Pytest doesn't take your fancy)
>>
>> I have to admit I chose unittest. Simply because it is in the standard
>> lbrary. As so many people seem to prefer pytest I should take a look at
>> it.
> 
> Not at all! This is a personal bias - I happen to have been using Pytest.

it doesn't have to be either-or; pytest can deal with unittest tests,
with some limitations. See here for some notes:

https://docs.pytest.org/en/stable/unittest.html

so you can try out what you have under pytest, and if liking it, can
begin to transition to more native pytest - or just keep it as the test
runner, or just stick with unittest.



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Stephen Rosen]

Worth noting: by assertTrue he probably meant assertEqual.
But I'd recommend using assertIn [1] if you're using unittest to check
output written to stdout/stderr.
That way, your tests are slightly more robust to changes in the exact
output.


pytest may also be helpful for this (or any!) type of testing.
Disclaimer/warning: pytest can be confusing even for experienced python
programmers because it does some fancy things.
But if you put in the time to learn it, it's very popular because of the
way it structures testsuites and code reuse (i.e. fixtures).

It can do a lot to help you, and provides output capturing out of the box
[2] as well as some handy tools for building temporary testing directories
[3].


[1]
https://docs.python.org/3.6/library/unittest.html#unittest.TestCase.assertIn
[2] https://docs.pytest.org/en/stable/capture.html
[3] https://docs.pytest.org/en/stable/tmpdir.html

On Fri, Jun 19, 2020 at 2:18 PM Terry Reedy  wrote:

> On 6/17/2020 12:34 PM, Tony Flury via Python-list wrote:
>
> > In a recent application that I wrote (where output to the console was
> > important), I tested it using the 'unittest' framework, and by patching
> > sys.stderr to be a StringIO - that way my test case could inspect what
> > was being output.
>
> Tony's code with hard returns added so that code lines remain separated
> instead of wrapping.
>
> with patch('sys.stderr', StringIO()) as stderr:
>application.do_stuff()  self.assertTrue(stderr.getvalue(),
>'Woops - that didn\'t work')
> This doc, worth reading more than once, is
> https://docs.python.org/3/library/unittest.mock.html#the-patchers
>
> --
> Terry Jan Reedy
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Terry Reedy]

On 6/17/2020 12:34 PM, Tony Flury via Python-list wrote:

In a recent application that I wrote (where output to the console was 
important), I tested it using the 'unittest' framework, and by patching 
sys.stderr to be a StringIO - that way my test case could inspect what 
was being output.


Tony's code with hard returns added so that code lines remain separated 
instead of wrapping.


with patch('sys.stderr', StringIO()) as stderr:
  application.do_stuff()  self.assertTrue(stderr.getvalue(), 
  'Woops - that didn\'t work')

This doc, worth reading more than once, is
https://docs.python.org/3/library/unittest.mock.html#the-patchers

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Tony Flury via Python-list]

On 24/04/2020 19:40, Manfred Lotz wrote:

I have a command like application which checks a directory tree for
certain things. If there are errors then messages will be written to
stdout.

How to test this in the best way?

One idea was for the error situations to write messages to files and
then later when running the tests to compare the error messages output
to the previously saved output.

Is there anything better?



In a recent application that I wrote (where output to the console was 
important), I tested it using the 'unittest' framework, and by patching 
sys.stderr to be a StringIO - that way my test case could inspect what 
was being output.


   with patch('sys.stderr', StringIO()) as
   stderr:application.do_stuff()self.assertTrue(stderr.getvalue(),
   'Woops - that didn\'t work')

I am not sure of the structure of your application, and whether you have a 
callable API that you can invoke.

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[DL Neil via Python-list]

Given your replies, 'now' might be a good time to take a look at
Pytest, and see how you could use it to help build better code - by
building tested units/functions which are assembled into ever-larger
tested-units... (there is a range of choice/other testing aids if
Pytest doesn't take your fancy)


I have to admit I chose unittest. Simply because it is in the standard
lbrary. As so many people seem to prefer pytest I should take a look at
it.


Not at all! This is a personal bias - I happen to have been using Pytest.

Your objective is learning to program in Python. 'Chopping and changing' 
between ancillary tools would be a distraction and time-sink. If you 
have already started learning unittest, keep going.


We all have to start 'somewhere'! At the learning stage, any one of 
these tools will help you with testing, and testing should improve your 
overall programming skills. Once your expertise, in both Python and the 
testing tool, matures; you will be better placed to survey the 
opportunities and to choose the best alternative for you and your 
applications...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Manfred Lotz]

On Mon, 27 Apr 2020 18:21:39 +1200
DL Neil  wrote:

...

> 
> Given your replies, 'now' might be a good time to take a look at
> Pytest, and see how you could use it to help build better code - by
> building tested units/functions which are assembled into ever-larger
> tested-units... (there is a range of choice/other testing aids if
> Pytest doesn't take your fancy)

I have to admit I chose unittest. Simply because it is in the standard
lbrary. As so many people seem to prefer pytest I should take a look at
it.

-- 
Manfred

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[DL Neil via Python-list]

If I have understood correctly, the objective is to check a dir-tree
to ensure that specific directory/file-permissions are in-effect/have
not been changed. The specifications come from a .JSON file and may
be over-ridden by command-line arguments. Correct?


Yes.




How to test this in the best way?


The best way to prepare for unit-testing is to have 'units' of code
(apologies!).

An effective guide is to break the code into functions and methods,
so that each performs exactly one piece/unit of work - and only the
one. A better guide is that if you cannot name the procedure using
one description and want to add an "and" an "or" or some other
conjunction, perhaps there should be more than one procedure!

For example:

  >for cat in cats:
  >  ...
  >  for d in scantree(cat.dir):
  >  # if `keep_fs` was specified then we must
  >  # make sure the file is on the same device
  >  if cat.keep_fs and devid != get_devid(d.path):
  >  continue
  >
  >  cat.check(d)



Above is part of the main() function. But I could make some part of
the main() function into its own function. Then the call to check a
directory could be a function argument. For testing I could reuse that
function by providing a different function for the check (now it is
cat.checkid(d) )


There is no $charge for the number of functions used. Don't hesitate!

Whilst I haven't heard the phrase used recently, we used to talk about 
systems- and program-design being a process of "step-wise 
decomposition". It was likened to peeling layers off an onion, or 
Russian nesting dolls - as one takes 'off' the outer layer, there is 
more inside.


In programming then, the main() calls a bunch of other functions. Each 
of those, calls its own set of sub-functions, and so-on. At first, we 
are talking 'higher levels' of control - and a good choice of function 
names summarises what's going-on/provides the 'plan of attack' and thus 
good "documentation"!


As control passes to the 'lower level' of functions, the level of detail 
increases. You might think of it like a microscope - the more powerful 
the magnification, the more detail is seen; but the field of view has 
narrowed/become more focussed.


The idea of having one function perform one (and only one) function is 
known as "encapsulation". Another term which was a new 'buzz-word', 
decades ago, is "modular programming" - instead of having one long 
program, splitting it up into functional components.


Yet another traditional phrase describing computer programs is: "input, 
process, output". This can also be applied, at the successive levels of 
detail, to functions - they 'take in' parameter-values, 'process' them 
in some way, and then return, or 'output' the result.


In a simplistic world, one function's output would become input to the 
next function. However, back in the real-world, it is more likely that 
several 'outputs' from earlier functions are used as 'input' by some 
successive procedure. Which is why we are paid 'the big bucks' (maybe).


Earlier functions should only influence or affect the behavior of later 
functions by passing data (output from one, becoming input to another). 
The complex ways in which functions can 'feed' or 'affect' each other is 
called "cohesion". If each function is dependent only upon its input 
data, this is held as an ideal. If a function is not independent, that 
is likely to be and/or become a source of friction and bugs. Again, in 
an ideal world...


There's a lot of follow-through, if you'd care to do some reading using 
such key-words...




If this were a function, how would you name it? First of all we are
processing every category, then we are scanning a dir-tree, and
finally we are doing something to each directory found.

If we split these into separate routines, eg (sub-setting the above):

  >  for d in scantree(cat.dir):
 do_something_with_directory( d )

and you devise a (much) more meaningful name than mine, it will
actually help readers (others - but also you!) to understand the
steps within the logic.

Now if we have a function which checks a single fileNM for
'whatever', the code will start something like:

def stat_file( fileNM ):
'''Gather stat information for nominated file.'''
etc

So, we can now write a test-function because we don't need any
"categories", we don't need all the dir-tree, and we don't need to
have performed a scan - all we need is a (valid and previously
inspected) file-path. Thus (using Pytest):

def test_stat_file( ... ):
'''Test file stat.'''
assert stat_file( "...file-path" ) == ...its
known-stat

def test_stat_dir( ... ):
'''Test file stat.'''
assert stat_file( "...dir-path" ) == ...its known-stat

There is no need to test for a non-existent file, if you are the only
user!

In case you 

Re: How to test?

[Manfred Lotz]

On Sun, 26 Apr 2020 15:26:58 +1200
DL Neil  wrote:

> On 25/04/20 7:53 PM, Manfred Lotz wrote:
> > On Sat, 25 Apr 2020 18:41:37 +1200
> > DL Neil  wrote:
> >   
> >> On 25/04/20 5:16 PM, Manfred Lotz wrote:  
> >>> On Fri, 24 Apr 2020 19:12:39 -0300
> >>> Cholo Lennon  wrote:
> >>>  
>  On 24/4/20 15:40, Manfred Lotz wrote:  
> > I have a command like application which checks a directory tree
> > for certain things. If there are errors then messages will be
> > written to stdout.  
> 
>  > What I do here specifically is to check directory trees' file
>  > objects for user and group ownerships as well as permissions
>  > according to a given policy. There is a general policy and there
>  > could be exceptions which specify a specific policy for a certain
>  > file.  
> 
> If I have understood correctly, the objective is to check a dir-tree
> to ensure that specific directory/file-permissions are in-effect/have
> not been changed. The specifications come from a .JSON file and may
> be over-ridden by command-line arguments. Correct?
> 

Yes.

> There must be a whole 'genre' of programs which inspect a
> directory-tree and doing 'something' to the files-contained. I had a
> few, and needing another, decided to write a generic 'scanner' which
> would then call a tailored-function to perform the particular
> 'something' - come to think of it, am not sure if that was ever quite
> finished. Sigh!
> 
> 
>  > One idea was for the error situations to write messages to
>  > files and then later when running the tests to compare the
>  > error messages output to the previously saved output.
>  >
>  > Is there anything better?  
> 
> The next specification appears to be that you want a list of files
> and their stats, perhaps reporting only any exceptions which weren't
> there 'last time'.
> 

I just want to report violations against the policy(ies) given in the
JSON file.

> In which case, an exception could be raised, or it might be simpler
> to call a reporting-function when a file should be 'reported'.
> 
> I'm still a little confused about the difference between 
> printing/logging/reporting some sort of exception, and the need to 
> compare with 'history'.
> 

There is no compare with history. I just want to see current violations
(I think I phrased things badly in my previous post so that it looked
like I want to see history.)

> 
> The problem with the "previously saved output" is the process of
> linking 'today's data' with that from the last run. I would use a
> database (but then, that's my background/bias) to store each file's
> stat.
> 

As said above I phrased things badly here in my previous post.


> Alternately, if you only need to store exceptions, and that number is 
> likely to be small, perhaps output only the exceptions from 'this
> run' to a .JSON/.yaml file - which would become input (and a dict for
> easy look-ups) next time?
> 
> 
>   Maybe I am wrong because I don't understand your scenario: If
>   your application is like a command, it has to return an error
>   code to the system, a distinct number for each error condition.
>   The error code is easier to test than the stdout/stderr.  
> 
> Either way, you might decrease the amount of data to be stored by 
> reducing the file's stat to a code.
> 
> 
> > How to test this in the best way?  
> 
> The best way to prepare for unit-testing is to have 'units' of code 
> (apologies!).
> 
> An effective guide is to break the code into functions and methods,
> so that each performs exactly one piece/unit of work - and only the
> one. A better guide is that if you cannot name the procedure using
> one description and want to add an "and" an "or" or some other
> conjunction, perhaps there should be more than one procedure!
> 
> For example:
> 
>  >for cat in cats:
>  >  ...
>  >  for d in scantree(cat.dir):
>  >  # if `keep_fs` was specified then we must
>  >  # make sure the file is on the same device
>  >  if cat.keep_fs and devid != get_devid(d.path):
>  >  continue
>  >
>  >  cat.check(d)  
> 

Above is part of the main() function. But I could make some part of
the main() function into its own function. Then the call to check a
directory could be a function argument. For testing I could reuse that
function by providing a different function for the check (now it is
cat.checkid(d) )




> If this were a function, how would you name it? First of all we are 
> processing every category, then we are scanning a dir-tree, and
> finally we are doing something to each directory found.
> 
> If we split these into separate routines, eg (sub-setting the above):
> 
>  >  for d in scantree(cat.dir):  
> do_something_with_directory( d )
> 
> and you devise a (much) more meaningful name than mine, it will
> actually help readers (others - but also you!) to understand the
> steps within the logic.
> 

Re: How to test?

[DL Neil via Python-list]

On 25/04/20 7:53 PM, Manfred Lotz wrote:

On Sat, 25 Apr 2020 18:41:37 +1200
DL Neil  wrote:


On 25/04/20 5:16 PM, Manfred Lotz wrote:

On Fri, 24 Apr 2020 19:12:39 -0300
Cholo Lennon  wrote:
   

On 24/4/20 15:40, Manfred Lotz wrote:

I have a command like application which checks a directory tree
for certain things. If there are errors then messages will be
written to stdout.


> What I do here specifically is to check directory trees' file objects
> for user and group ownerships as well as permissions according to a
> given policy. There is a general policy and there could be exceptions
> which specify a specific policy for a certain file.

If I have understood correctly, the objective is to check a dir-tree to 
ensure that specific directory/file-permissions are in-effect/have not 
been changed. The specifications come from a .JSON file and may be 
over-ridden by command-line arguments. Correct?


There must be a whole 'genre' of programs which inspect a directory-tree 
and doing 'something' to the files-contained. I had a few, and needing 
another, decided to write a generic 'scanner' which would then call a 
tailored-function to perform the particular 'something' - come to think 
of it, am not sure if that was ever quite finished. Sigh!



> One idea was for the error situations to write messages to files
> and then later when running the tests to compare the error
> messages output to the previously saved output.
>
> Is there anything better?

The next specification appears to be that you want a list of files and 
their stats, perhaps reporting only any exceptions which weren't there 
'last time'.


In which case, an exception could be raised, or it might be simpler to 
call a reporting-function when a file should be 'reported'.


I'm still a little confused about the difference between 
printing/logging/reporting some sort of exception, and the need to 
compare with 'history'.



The problem with the "previously saved output" is the process of linking 
'today's data' with that from the last run. I would use a database (but 
then, that's my background/bias) to store each file's stat.


Alternately, if you only need to store exceptions, and that number is 
likely to be small, perhaps output only the exceptions from 'this run' 
to a .JSON/.yaml file - which would become input (and a dict for easy 
look-ups) next time?



 Maybe I am wrong because I don't understand your scenario: If your
 application is like a command, it has to return an error code to
 the system, a distinct number for each error condition. The error
 code is easier to test than the stdout/stderr.

Either way, you might decrease the amount of data to be stored by 
reducing the file's stat to a code.




How to test this in the best way?


The best way to prepare for unit-testing is to have 'units' of code 
(apologies!).


An effective guide is to break the code into functions and methods, so 
that each performs exactly one piece/unit of work - and only the one. A 
better guide is that if you cannot name the procedure using one 
description and want to add an "and" an "or" or some other conjunction, 
perhaps there should be more than one procedure!


For example:

>for cat in cats:
>  ...
>  for d in scantree(cat.dir):
>  # if `keep_fs` was specified then we must
>  # make sure the file is on the same device
>  if cat.keep_fs and devid != get_devid(d.path):
>  continue
>
>  cat.check(d)

If this were a function, how would you name it? First of all we are 
processing every category, then we are scanning a dir-tree, and finally 
we are doing something to each directory found.


If we split these into separate routines, eg (sub-setting the above):

>  for d in scantree(cat.dir):
   do_something_with_directory( d )

and you devise a (much) more meaningful name than mine, it will actually 
help readers (others - but also you!) to understand the steps within the 
logic.


Now if we have a function which checks a single fileNM for 'whatever', 
the code will start something like:


def stat_file( fileNM ):
'''Gather stat information for nominated file.'''
etc

So, we can now write a test-function because we don't need any 
"categories", we don't need all the dir-tree, and we don't need to have 
performed a scan - all we need is a (valid and previously inspected) 
file-path. Thus (using Pytest):


def test_stat_file( ... ):
'''Test file stat.'''
assert stat_file( "...file-path" ) == ...its known-stat

def test_stat_dir( ... ):
'''Test file stat.'''
assert stat_file( "...dir-path" ) == ...its known-stat

There is no need to test for a non-existent file, if you are the only user!

In case you hadn't thought about it, make the test path, part of your 
test directory - not part of 'the 

Re: How to test?

[Cholo Lennon]

On 4/25/20 2:16 AM, Manfred Lotz wrote:

On Fri, 24 Apr 2020 19:12:39 -0300
Cholo Lennon  wrote:


On 24/4/20 15:40, Manfred Lotz wrote:

I have a command like application which checks a directory tree for
certain things. If there are errors then messages will be written to
stdout.

How to test this in the best way?

One idea was for the error situations to write messages to files and
then later when running the tests to compare the error messages
output to the previously saved output.

Is there anything better?

   


Maybe I am wrong because I don't understand your scenario: If your
application is like a command, it has to return an error code to the
system, a distinct number for each error condition. The error code is
easier to test than the stdout/stderr.



Yes, a different error code for each condition is good to test.
However, I need to test as well the correct file name and other values
belonging to a certain error condition.


So, I though that you were testing your application from an "external 
point of view", running it and validating its behavior. This is not the 
case... if you need such degree of control over its functionality, IMHO, 
you just need a unit test framework to validate every part of it.


If you still need to execute the application to test it, the best way, 
as other suggested, it is to log the working conditions. You can design 
very well the log information in order to simplify the parsing of it.


--
Cholo Lennon
Bs.As.
ARG
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Manfred Lotz]

On Sat, 25 Apr 2020 18:41:37 +1200
DL Neil  wrote:

> On 25/04/20 5:16 PM, Manfred Lotz wrote:
> > On Fri, 24 Apr 2020 19:12:39 -0300
> > Cholo Lennon  wrote:
> >   
> >> On 24/4/20 15:40, Manfred Lotz wrote:  
> >>> I have a command like application which checks a directory tree
> >>> for certain things. If there are errors then messages will be
> >>> written to stdout.
> >>>
> >>> How to test this in the best way?
> >>>
> >>> One idea was for the error situations to write messages to files
> >>> and then later when running the tests to compare the error
> >>> messages output to the previously saved output.
> >>>
> >>> Is there anything better?
> >>>
> >>>  
> >>
> >> Maybe I am wrong because I don't understand your scenario: If your
> >> application is like a command, it has to return an error code to
> >> the system, a distinct number for each error condition. The error
> >> code is easier to test than the stdout/stderr.
> >>  
> > 
> > Yes, a different error code for each condition is good to test.
> > However, I need to test as well the correct file name and other
> > values belonging to a certain error condition.  
> 
> It is frustrating to be shown only part of the information, and later
> be told our efforts aren't good-enough. 

I found the tips I got here quite good. Actually, each time when I had
a questions here I got competent answers.

> How about respecting our
> (donated) time, and posting some sample code that shows exactly
> what/where the problem lies?
> 

Sorry, perhaps I was too lazy to explain. Hope the following
explanation isn't too bad.


What I do here specifically is to check directory trees' file objects
for user and group ownerships as well as permissions according to a
given policy. There is a general policy and there could be exceptions
which specify a specific policy for a certain file.

The policy file is a JSON file and could have different categories.
Each category defines a policy for a certain directory tree. Comand
line args could overwrite directory names, as well as user and group.
Or if for example a directory is not specified in the JSON file I am
required to specify it via command line. Otherwise no check can take
place.

Here an example of a simple json file:

default":
  {
  "match_perms":  "644",
  "match_permsd":  "755",
  "match_permsx":  "755",
  "owner":  "manfred",
  "group":  "manfred"
  }
}

another simple example:

{
"default":
  {
"have_all_permsx" : "555",
"have_all_perms" : "444",
"have_all_permsd" : "555",
"owner":  "manfred",
"group":  "manfred"
  }
}


class Category contains a policy.

Here a simplified version of Category


class Category:
def __init__(self, cat, jsoncat, dirdict, ownerdict, groupdict):

self.cat = cat
self.jsoncat = jsoncat
self.keep_fs = jsoncat.get('keep_fs')
self.dir = jsoncat.get('dir')

if dirdict.get(cat):
self.dir = os.path.realpath(dirdict[cat])

self.policy = Policy(jsoncat, ownerdict, groupdict)

def check(self, fpath):
self.policy.check(fpath)

It is important to note that ownerdict and groupdict are  overwrites
from the command line, and could be empty if nothing was
specified.

In my main routine I do this

...
  for cat in cats:
... 
for d in scantree(cat.dir):
# if `keep_fs` was specified then we must
# make sure the file is on the same device
if cat.keep_fs and devid != get_devid(d.path):
continue

cat.check(d)

...


class Policy contains required user and group owners and a
list of permission objects for files, for executable files and for
directories. 


Those permissions are described like this

class BasePerms:
def __init__(self, perms: str):
"""Create an object holding a permission which is octal"""
self.perms = int(perms, 8)

class MatchPerms(BasePerms):
def check(self, fpath: str, mode: int):
mode = mode & 0o
if mode != self.perms:
print("%s perms differ, should have %s has %s" % (fpath, 
oct(self.perms), oct(mode)))

class HaveAllPerms(BasePerms):
def check(self, fpath: str, mode: int):
mode = mode & 0o
if (mode & self.perms) != self.perms:
print("%s: perms differ, should have all of %s has %s" % (fpath, 
oct(self.perms), oct(mode)))


After getting some ideas here I want to change my code to return an
error object in check() instead of doing a print(). Or return None
if all is ok. 

Then the error object could have a print-like method and a compare
method so that later on when doing a test I could compare two error
objects.  Perhaps I can combine it with callback. I have to play with
these things.


-- 
Manfred

















-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[DL Neil via Python-list]

On 25/04/20 5:16 PM, Manfred Lotz wrote:

On Fri, 24 Apr 2020 19:12:39 -0300
Cholo Lennon  wrote:


On 24/4/20 15:40, Manfred Lotz wrote:

I have a command like application which checks a directory tree for
certain things. If there are errors then messages will be written to
stdout.

How to test this in the best way?

One idea was for the error situations to write messages to files and
then later when running the tests to compare the error messages
output to the previously saved output.

Is there anything better?

   


Maybe I am wrong because I don't understand your scenario: If your
application is like a command, it has to return an error code to the
system, a distinct number for each error condition. The error code is
easier to test than the stdout/stderr.



Yes, a different error code for each condition is good to test.
However, I need to test as well the correct file name and other values
belonging to a certain error condition.


It is frustrating to be shown only part of the information, and later be 
told our efforts aren't good-enough. How about respecting our (donated) 
time, and posting some sample code that shows exactly what/where the 
problem lies?


From the above, I'm wondering if a custom exception might be applicable.

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Manfred Lotz]

On 24 Apr 2020 22:18:45 GMT
r...@zedat.fu-berlin.de (Stefan Ram) wrote:

> DL Neil  writes:
> >Python's logging library enables messages to be formatted
> >accordingly, and directed differently, depending upon 'level of
> >severity'. So, as well as the flexibility mentioned before, there is
> >an option to direct logging output to stdout/stderr as a
> >matter-of-course!  
> 
>   Here's some example code I wrote:
> 
> import logging
> 
> def setup_logging( logging ):
> """Setup a logger using the standard logging module,
> which needs to be passed as the argument"""
> logger = logging.getLogger()
> handler = logging.StreamHandler()
> formatter = logging.Formatter( '%(asctime)s %(name)-12s
> %(levelname)-8s %(message)s' ) handler.setFormatter( formatter )
> logger.addHandler( handler )
> logger.setLevel( logging.DEBUG )
> return logger
> 
> logger = setup_logging( logging )
> logger.debug( "Hi!" )
> 
>   It outputs (apparently to sys.stderr):
> 
> 2020-04-24 23:13:59,467 root DEBUGHi!
> 
> 

Yes, I know about this. But in this particular case I don't gain much
by using a logger. print() is good enough.

I will investigate yielding and callback to see which is best in my
particular case.




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Manfred Lotz]

On Fri, 24 Apr 2020 19:12:39 -0300
Cholo Lennon  wrote:

> On 24/4/20 15:40, Manfred Lotz wrote:
> > I have a command like application which checks a directory tree for
> > certain things. If there are errors then messages will be written to
> > stdout.
> > 
> > How to test this in the best way?
> > 
> > One idea was for the error situations to write messages to files and
> > then later when running the tests to compare the error messages
> > output to the previously saved output.
> > 
> > Is there anything better?
> > 
> >   
> 
> Maybe I am wrong because I don't understand your scenario: If your 
> application is like a command, it has to return an error code to the 
> system, a distinct number for each error condition. The error code is 
> easier to test than the stdout/stderr.
> 

Yes, a different error code for each condition is good to test.
However, I need to test as well the correct file name and other values
belonging to a certain error condition.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Cholo Lennon]

On 24/4/20 15:40, Manfred Lotz wrote:

I have a command like application which checks a directory tree for
certain things. If there are errors then messages will be written to
stdout.

How to test this in the best way?

One idea was for the error situations to write messages to files and
then later when running the tests to compare the error messages output
to the previously saved output.

Is there anything better?




Maybe I am wrong because I don't understand your scenario: If your 
application is like a command, it has to return an error code to the 
system, a distinct number for each error condition. The error code is 
easier to test than the stdout/stderr.


--
Cholo Lennon
Bs.As.
ARG

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[DL Neil via Python-list]

May I point-out that the above may not be the best approach. Rather
than using screen-prints to report errors, another method is to
utilise "logging" to collect such data - so that there is always a
formal record (regardless of user behavior). During 'production' the
information could be collected at some central 'location' for
inspection by competent staff. During 'development', it is possible,
by changing one line, to re-direct the log to wherever you would like
- including the above!



Logging wouldn't help here as the person running the program is
competent, and it likes to see what the errors are.


Perhaps it is a 'traditional view' that "logging" implies capture to 
some "log file".


Python's logging library enables messages to be formatted accordingly, 
and directed differently, depending upon 'level of severity'. So, as 
well as the flexibility mentioned before, there is an option to direct 
logging output to stdout/stderr as a matter-of-course!


(and thus, today's user can have output to the screen, but if another 
and less-capable user joins-the-club, his/her output could easily be 
re-directed. On the other hand: YAGNI!)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Manfred Lotz]

On Sat, 25 Apr 2020 08:25:00 +1200
DL Neil  wrote:

> On 25/04/20 6:40 AM, Manfred Lotz wrote:
> > I have a command like application which checks a directory tree for
> > certain things. If there are errors then messages will be written to
> > stdout.
> > 
> > How to test this in the best way?
> > 
> > One idea was for the error situations to write messages to files and
> > then later when running the tests to compare the error messages
> > output to the previously saved output.
> > 
> > Is there anything better?  
> 
> Yes, as well as reproducing the output on-screen, there are now three 
> ways to deal with stdout. The newest is "tee", which like the Linux 
> command of the same name, gives the best of both worlds - display and 
> 'capture'!
> 
> Capturing of the stdout/stderr output
> https://docs.pytest.org/en/latest/capture.html
> 
> 

This is interesing.

> May I point-out that the above may not be the best approach. Rather
> than using screen-prints to report errors, another method is to
> utilise "logging" to collect such data - so that there is always a
> formal record (regardless of user behavior). During 'production' the
> information could be collected at some central 'location' for
> inspection by competent staff. During 'development', it is possible,
> by changing one line, to re-direct the log to wherever you would like
> - including the above!
> 

Logging wouldn't help here as the person running the program is
competent, and it likes to see what the errors are.

-- 
Manfred





-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[Manfred Lotz]

On 24 Apr 2020 20:17:04 GMT
r...@zedat.fu-berlin.de (Stefan Ram) wrote:

> Manfred Lotz  writes:
> >I have a command like application which checks a directory tree for
> >certain things. If there are errors then messages will be written to
> >stdout.  
> 
>   Error messages should be written to sys.stderr.
> 

Yes. But for testing it doesn't matter if it is written to stdout or
stderr.


> >How to test this in the best way?  
> 
>   The functions that check the tree should not write to
>   the console. Instead, when an error occurs they 
> 
>   - raise an exception,

This would be bad. In my case I want to report all the errors to
somebody else. An exception means I don't see more errors which
may have occured.


>   - yield an error information, 

Hm, you mean the check function would be something like an iterator.
Sounds interesting.

>   - return an error information, or

I think this is harder to implement.

>   - call a callback with the error information.
> 
>   Their callers than can be either high-level console code,
>   that writes those information to a console, or test code.
> 

This is also very interesting.


So, I think yield or callback are the ideas I have to investigate.


Thanks a lot.

-- 
Manfred





-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test?

[DL Neil via Python-list]

On 25/04/20 6:40 AM, Manfred Lotz wrote:

I have a command like application which checks a directory tree for
certain things. If there are errors then messages will be written to
stdout.

How to test this in the best way?

One idea was for the error situations to write messages to files and
then later when running the tests to compare the error messages output
to the previously saved output.

Is there anything better?


Yes, as well as reproducing the output on-screen, there are now three 
ways to deal with stdout. The newest is "tee", which like the Linux 
command of the same name, gives the best of both worlds - display and 
'capture'!


Capturing of the stdout/stderr output
https://docs.pytest.org/en/latest/capture.html


May I point-out that the above may not be the best approach. Rather than 
using screen-prints to report errors, another method is to utilise 
"logging" to collect such data - so that there is always a formal record 
(regardless of user behavior). During 'production' the information could 
be collected at some central 'location' for inspection by competent 
staff. During 'development', it is possible, by changing one line, to 
re-direct the log to wherever you would like - including the above!


Python library: https://docs.python.org/3/library/logging.html
Logging (testing): https://docs.pytest.org/en/latest/logging.html
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[Souvik Dutta]

He is talking about this.
https://mail.python.org/mailman/listinfo/python-list
Scroll down to the bottom of the page and you will find overview of
all..

Souvik flutter dev

On Fri, Apr 24, 2020, 7:50 AM Deac-33 Lancaster  wrote:

> On Thursday, April 23, 2020 at 7:14:16 PM UTC-7, DL Neil wrote:
> > ... Is there a way to see all of the groups?
> >
> > Yes! Follow the link at the bottom of this email msg. Then follow the
> > link at the bottom of this list's web-page ...
> > --
> > Regards =dn
>
> Sorry, I don't see the link.
> -deac33
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[Deac-33 Lancaster]

On Thursday, April 23, 2020 at 7:14:16 PM UTC-7, DL Neil wrote:
> ... Is there a way to see all of the groups?
> 
> Yes! Follow the link at the bottom of this email msg. Then follow the 
> link at the bottom of this list's web-page ...
> -- 
> Regards =dn

Sorry, I don't see the link.
-deac33
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[DL Neil via Python-list]

... Is there a way to see all of the groups?

Yes! Follow the link at the bottom of this email msg. Then follow the 
link at the bottom of this list's web-page ...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[Deac-33 Lancaster]

On Thursday, April 23, 2020 at 7:11:47 PM UTC-7, DL Neil wrote:
> On 24/04/20 1:24 PM, Deac-33 Lancaster wrote:
> > I'm aware that you can find the type of a variable with
> > type(var)
> > 
> > But are there Boolean operators in Python3.8 to test the data type, e.g.
> >is_floate(var)
> >is_string(var)
> > etc. ?
> 
> There is also a 'pythonic' answer (what is the 'Python way'?) and that 
> is to proceed on the basis of "duck typing" and 'presumption'. The 
> latter known as EAFP ("It's Easier To Ask Forgiveness Than To Get 
> Permission"), eg:
> 
>  >>> n = 2
>  >>> d = 'two'
> 
> my training/first inclination has always been to check before use - in 
> this case, that both the numerator and denominator are numeric and that 
> the latter is non-zero - seeking to avoid:
> 
>  >>> n / d
> Traceback (most recent call last):
>File "", line 1, in 
> TypeError: unsupported operand type(s) for /: 'int' and 'str'
> 
> 
> Crash!
> 
> Conversely, here is the pythonic EAFP approach:
> 
>  >>> try:
> ... n / d
> ... except TypeError:
> ... print( "I'm sorry, Dave. I'm afraid I can't do that." )
> ...
> I'm sorry, Dave. I'm afraid I can't do that.
>  >>>
> 
> 
> and if you want to go all-in and try to break the laws of mathematics:
> 
>  >>> n = 2
>  >>> d = 0
> 
>  >>> try:
> ... n / d
> ... except TypeError:
> ... print( "I'm sorry, Dave. I'm afraid I can't do that." )
> ... except ZeroDivisionError:
> ... print( "Officer, it wasn't me - honest!" )
> ...
> Officer, it wasn't me - honest!
>  >>>
> 
> 
> or just-for-fun:
> 
>  >>> try:
> ... n / d
> ... except ( ZeroDivisionError, TypeError ):
> ... print( "I'm sorry, Dave. I'm afraid I can't do that." )
> ...
> I'm sorry, Dave. I'm afraid I can't do that.
> -- 
> Regards =dn


DL,
I love it.  I like the style as well.
thanks again,
-deac33
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[DL Neil via Python-list]

On 24/04/20 1:24 PM, Deac-33 Lancaster wrote:

I'm aware that you can find the type of a variable with
type(var)

But are there Boolean operators in Python3.8 to test the data type, e.g.
   is_floate(var)
   is_string(var)
etc. ?


There is also a 'pythonic' answer (what is the 'Python way'?) and that 
is to proceed on the basis of "duck typing" and 'presumption'. The 
latter known as EAFP ("It's Easier To Ask Forgiveness Than To Get 
Permission"), eg:


>>> n = 2
>>> d = 'two'

my training/first inclination has always been to check before use - in 
this case, that both the numerator and denominator are numeric and that 
the latter is non-zero - seeking to avoid:


>>> n / d
Traceback (most recent call last):
  File "", line 1, in 
TypeError: unsupported operand type(s) for /: 'int' and 'str'


Crash!

Conversely, here is the pythonic EAFP approach:

>>> try:
... n / d
... except TypeError:
... print( "I'm sorry, Dave. I'm afraid I can't do that." )
...
I'm sorry, Dave. I'm afraid I can't do that.
>>>


and if you want to go all-in and try to break the laws of mathematics:

>>> n = 2
>>> d = 0

>>> try:
... n / d
... except TypeError:
... print( "I'm sorry, Dave. I'm afraid I can't do that." )
... except ZeroDivisionError:
... print( "Officer, it wasn't me - honest!" )
...
Officer, it wasn't me - honest!
>>>


or just-for-fun:

>>> try:
... n / d
... except ( ZeroDivisionError, TypeError ):
... print( "I'm sorry, Dave. I'm afraid I can't do that." )
...
I'm sorry, Dave. I'm afraid I can't do that.
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[Deac-33 Lancaster]

On Thursday, April 23, 2020 at 6:47:04 PM UTC-7, DL Neil wrote:
> On 24/04/20 1:24 PM, Deac-33 Lancaster wrote:
> > I'm aware that you can find the type of a variable with
> > type(var)
> > 
> > But are there Boolean operators in Python3.8 to test the data type, e.g.
> >is_floate(var)
> >is_string(var)
> > etc. ?
> 
> You are close! https://docs.python.org/3/library/functions.html#isinstance
> 
> 
> > (If this is the wrong group for this question, what group should I use.)
> 
> We're happy to help. There is a Python-Tutor group which is a good 
> 'home' for beginners or new-converts to Python...
> -- 
> Regards =dn

DL Neil,
Much thanks, exactly what I was looking for.  
And I found and joined Python-Tutor.  Is there a way to see all of the groups?

Mucho thanks,
-deac33
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[Deac-33 Lancaster]

On Thursday, April 23, 2020 at 6:46:14 PM UTC-7, Alan Bawden wrote:
> Deac-33 Lancaster  writes:
> 
> > I'm aware that you can find the type of a variable with 
> >type(var)
> 
> (Strictly speaking, variables don't have types.  This gives you the type of
> the variable's current value.  But we know what you meant.)
> 
> > But are there Boolean operators in Python3.8 to test the data type, e.g.
> >   is_floate(var)
> >   is_string(var)
> > etc. ?
> 
> You should probably be using isinstance(), as in:
>   isinstance(var, float)
>   isinstance(var, str)
> 
> -- 
> Alan Bawden

Alan,
Much thanks for the response.   Yes, dynamic typing so it can change.  But this 
is what I was looking for, unable to find it in the docs, need more practice at 
that.  :-) 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[Alan Bawden]

Deac-33 Lancaster  writes:

> I'm aware that you can find the type of a variable with 
>type(var)

(Strictly speaking, variables don't have types.  This gives you the type of
the variable's current value.  But we know what you meant.)

> But are there Boolean operators in Python3.8 to test the data type, e.g.
>   is_floate(var)
>   is_string(var)
> etc. ?

You should probably be using isinstance(), as in:
  isinstance(var, float)
  isinstance(var, str)

-- 
Alan Bawden
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[Deac-33 Lancaster]

ChrisA,

Most awesome, thank you very much, I just couldn't find that in the docs.  
Still learning!

Much thanks,
-deac33
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[DL Neil via Python-list]

On 24/04/20 1:24 PM, Deac-33 Lancaster wrote:

I'm aware that you can find the type of a variable with
type(var)

But are there Boolean operators in Python3.8 to test the data type, e.g.
   is_floate(var)
   is_string(var)
etc. ?


You are close! https://docs.python.org/3/library/functions.html#isinstance



(If this is the wrong group for this question, what group should I use.)


We're happy to help. There is a Python-Tutor group which is a good 
'home' for beginners or new-converts to Python...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test the data type of a variable

[Chris Angelico]

On Fri, Apr 24, 2020 at 11:26 AM Deac-33 Lancaster  wrote:
>
> I'm aware that you can find the type of a variable with
>type(var)
>
> But are there Boolean operators in Python3.8 to test the data type, e.g.
>   is_floate(var)
>   is_string(var)
> etc. ?
>
> (If this is the wrong group for this question, what group should I use.)
>

Yep! You can ask questions like this:

isinstance(var, int)
isinstance(var, str)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for type or instance of dict_values?

[fabian . becker87]

Very late to the party, but I just encountered a very similar problem and found 
a solution:

```
import collections

obj = {"foo": "bar"}
isinstance(obj.values(), collections.abc.ValuesView) # => True
```

Hope that helps someone out there :)

On Thursday, November 17, 2016 at 9:09:23 AM UTC-8, Terry Reedy wrote:
> On 11/17/2016 9:57 AM, Thorsten Kampe wrote:
> 
> > The code in question is part of an attempt to get the dimensions of
> > multi-dimensional lists, the `isinstance` is there in order to
> > exclude strings.
> 
> You can do the exclusion directly.
> 
> >
> > """
> > def dim(seq):
> > dimension = []
> > while isinstance(seq, (list, tuple)):
> 
>  while not isinstance(seq, str) # or (str, bytes, ...)
> 
> > dimension.append(len(seq))
> > try:
> > seq = seq[0]
> > except IndexError:  # sequence is empty
> > break
> > return dimension
> > """
> >
> > Thorsten
> >
> 
> 
> -- 
> Terry Jan Reedy
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for type or instance of dict_values?

[Terry Reedy]

On 11/17/2016 9:57 AM, Thorsten Kampe wrote:


The code in question is part of an attempt to get the dimensions of
multi-dimensional lists, the `isinstance` is there in order to
exclude strings.


You can do the exclusion directly.



"""
def dim(seq):
dimension = []
while isinstance(seq, (list, tuple)):


while not isinstance(seq, str) # or (str, bytes, ...)


dimension.append(len(seq))
try:
seq = seq[0]
except IndexError:  # sequence is empty
break
return dimension
"""

Thorsten




--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for type or instance of dict_values?

[Thorsten Kampe]

* Peter Otten (Thu, 17 Nov 2016 13:38:26 +0100)
> 
> Thorsten Kampe wrote:
> 
> > How can I test for type or instance of dictviews like dict_values?
> 
> Why do you want to?

Thanks, for the `collections.abc.ValuesView` tip.

The code in question is part of an attempt to get the dimensions of 
multi-dimensional lists, the `isinstance` is there in order to 
exclude strings.

"""
def dim(seq):
dimension = []
while isinstance(seq, (list, tuple)):
dimension.append(len(seq))
try:
seq = seq[0]
except IndexError:  # sequence is empty
break
return dimension
"""

Thorsten

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to test for type or instance of dict_values?

[Peter Otten]

Thorsten Kampe wrote:

> How can I test for type or instance of dictviews like dict_values?

Why do you want to?
 
> `isinstance({}.values, dict_values)` gives
> `NameError: name 'dict_values' is not defined`

You can "fix" this with

>>> dict_values = type({}.values())

or, depending on the use case, use another test:

>>> isinstance({}.values(), collections.abc.ValuesView)
True

> """
 type({}.values())
> 
> """
> 
> Thorsten


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to test attribute existence of feedparser objects

[xDog Walker]

On Thursday 2011 December 08 01:34, HansPeter wrote:
 Hi,

 While using the feedparser library for downloading RSS feeds some of
 the blog entries seem to have no title.

  File build\bdist.win32\egg\feedparser.py, line 382, in __getattr__
 AttributeError: object has no attribute 'title'

 Is there a way to test the existence of an attribute?


From the Fine Manual for feedparser 5.1:

Testing for Existence¶

Feeds in the real world may be missing elements, even elements that are 
required by the specification. You should always test for the existence of an 
element before getting its value. Never assume an element is present.

Use standard Python dictionary functions such as has_key to test whether an 
element exists.
Testing if elements are present¶

 import feedparser
 d = feedparser.parse('http://feedparser.org/docs/examples/atom10.xml')
 d.feed.has_key('title')
True
 d.feed.has_key('ttl')
False
 d.feed.get('title', 'No title')
u'Sample feed'
 d.feed.get('ttl', 60)
60


-- 
I have seen the future and I am not in it.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to test attribute existence of feedparser objects

[Steven D'Aprano]

On Sat, 10 Dec 2011 09:19:31 -0800, xDog Walker wrote:

 Use standard Python dictionary functions such as has_key to test whether
 an element exists.

Idiomatic Python code today no longer uses has_key.

# Was:
d.feed.has_key('title')

# Now preferred
'title' in d.feed



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to test attribute existence of feedparser objects

[Chris Rebert]

On Thu, Dec 8, 2011 at 1:34 AM, HansPeter hanspeter.sl...@gmail.com wrote:
 Hi,

 While using the feedparser library for downloading RSS feeds some of
 the blog entries seem to have no title.

  File build\bdist.win32\egg\feedparser.py, line 382, in __getattr__
 AttributeError: object has no attribute 'title'

 Is there a way to test the existence of an attribute?

 I can use an exception but like below to see whether it exists but
 this is a clumsy way since the function has to return the title.

hasattr(obj, attr_name)
See docs.python.org/dev/library/functions.html#hasattr

That said, sounds like it won't make much difference in the particular
case you mention.
Also, never use a bare except: clause, unless you know what you're
doing and have a really good reason. Do except AttributeError in
this case.

Cheers,
Chris
--
http://rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Mathias Lafeldt]

On Sat, Oct 15, 2011 at 1:44 AM, MrPink tdsimp...@gmail.com wrote:

 Is there a function in Python that can be used to test if the value in
 a string is an integer?  I had to make one up for myself and it looks
 like this:

 def isInt(s):
    try:
        i = int(s)
        return True
    except ValueError:
        return False

According to [1], there're more Exceptions to test for:

try:
int(s)
return True
except (TypeError, ValueError, OverflowError): # int conversion failed
return False

[1] http://jaynes.colorado.edu/PythonIdioms.html, idiom Catch errors
rather than avoiding them to avoid cluttering your code with special
cases

-Mathias
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Noah Hall]

On Sat, Oct 15, 2011 at 12:44 AM, MrPink tdsimp...@gmail.com wrote:

 Is there a function in Python that can be used to test if the value in
 a string is an integer?  I had to make one up for myself and it looks
 like this:

 def isInt(s):
    try:
        i = int(s)
        return True
    except ValueError:
        return False


There's the isdigit method, for example -

 str = 1324325
 str.isdigit()
True
 str = 1232.34
 str.isdigit()
False
 str = I am a string, not an int!
 str.isdigit()
False
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Ian Kelly]

On Mon, Oct 17, 2011 at 2:44 PM, Noah Hall enali...@gmail.com wrote:
 There's the isdigit method, for example -

 str = 1324325
 str.isdigit()
 True
 str = 1232.34
 str.isdigit()
 False
 str = I am a string, not an int!
 str.isdigit()
 False

That works for non-negative base-10 integers.  But:

 -1234.isdigit()
False

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Roy Smith]

In article mailman.2051.1318881724.27778.python-l...@python.org,
 Mathias Lafeldt mathias.lafe...@googlemail.com wrote:

 According to [1], there're more Exceptions to test for:
 
 try:
 int(s)
 return True
 except (TypeError, ValueError, OverflowError): # int conversion failed
 return False


I don't think I would catch TypeError here.  It kind of depends on how 
isInt() is defined.  Is it:

def isInt(s):
  Return True if s is a string representing an integer

or is it:

def isInt(s):
  Return True if s (which must be a string) represents an integer

If the latter, then passing a non-string violates the contract, and the 
function should raise TypeError.  If the former, then you could make 
some argument for catching the TypeError and returning False, but I 
think the second version is what most people have in mind for isInt().

Can you even get an OverflowError any more in a modern Python?

 
int('9')
9L
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Chris Kaynor]

Python 2.6 running on Windows 7:
 99.0**99**99
OverflowError: (34, 'Result too large')
Traceback (most recent call last):
  File stdin-inspect, line 1, in module
OverflowError: (34, 'Result too large')

However, from the documentation:
Because of the lack of standardization of floating point exception
handling in C, most floating point operations also aren’t checked.
(http://docs.python.org/library/exceptions.html#exceptions.OverflowError)

Chris

On Mon, Oct 17, 2011 at 5:33 PM, Roy Smith r...@panix.com wrote:

 In article mailman.2051.1318881724.27778.python-l...@python.org,
  Mathias Lafeldt mathias.lafe...@googlemail.com wrote:

  According to [1], there're more Exceptions to test for:
 
  try:
      int(s)
      return True
  except (TypeError, ValueError, OverflowError): # int conversion failed
      return False


 I don't think I would catch TypeError here.  It kind of depends on how
 isInt() is defined.  Is it:

 def isInt(s):
  Return True if s is a string representing an integer

 or is it:

 def isInt(s):
  Return True if s (which must be a string) represents an integer

 If the latter, then passing a non-string violates the contract, and the
 function should raise TypeError.  If the former, then you could make
 some argument for catching the TypeError and returning False, but I
 think the second version is what most people have in mind for isInt().

 Can you even get an OverflowError any more in a modern Python?

 
 int('9')
 9L
 --
 http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Ian Kelly]

On Mon, Oct 17, 2011 at 6:40 PM, Chris Kaynor ckay...@zindagigames.com wrote:
 Python 2.6 running on Windows 7:
 99.0**99**99
 OverflowError: (34, 'Result too large')
 Traceback (most recent call last):
   File stdin-inspect, line 1, in module
 OverflowError: (34, 'Result too large')

 However, from the documentation:
 Because of the lack of standardization of floating point exception
 handling in C, most floating point operations also aren’t checked.
 (http://docs.python.org/library/exceptions.html#exceptions.OverflowError)

I think what Roy meant was can you even get an OverflowError from
calling int() any more, to which I think the answer is no, since in
modern Pythons int() will auto-promote to a long, and in Python 3
they're even the same thing.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Yingjie Lan]

- Original Message -
 From: Noah Hall enali...@gmail.com
 To: MrPink tdsimp...@gmail.com
 Cc: python-list@python.org
 Sent: Tuesday, October 18, 2011 4:44 AM
 Subject: Re: How to test if object is an integer?

 There's the isdigit method, for example -
 
  str = 1324325
  str.isdigit()
 True
  str = 1232.34
  str.isdigit()
 False
  str = I am a string, not an int!
  str.isdigit()
 False


There are some corner cases to be considered with this approach:
1. negative integers: '-3'
2. strings starting with '0': '03'
3. strings starting with one '+': '+3'
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Steven D'Aprano]

On Mon, 17 Oct 2011 18:59:44 -0600, Ian Kelly wrote:

 On Mon, Oct 17, 2011 at 6:40 PM, Chris Kaynor ckay...@zindagigames.com
 wrote:
 Python 2.6 running on Windows 7:
 99.0**99**99
 OverflowError: (34, 'Result too large') Traceback (most recent call
 last):
   File stdin-inspect, line 1, in module
 OverflowError: (34, 'Result too large')

 However, from the documentation:
 Because of the lack of standardization of floating point exception
 handling in C, most floating point operations also aren’t checked.
 (http://docs.python.org/library/
exceptions.html#exceptions.OverflowError)
 
 I think what Roy meant was can you even get an OverflowError from
 calling int() any more, to which I think the answer is no, since in
 modern Pythons int() will auto-promote to a long, and in Python 3
 they're even the same thing.


You can still get an OverflowError:

 inf = float('inf')
 int(inf)
Traceback (most recent call last):
  File stdin, line 1, in module
OverflowError: cannot convert float infinity to integer


and similarly for Decimal('inf') as well.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Terry Reedy]

On 10/14/2011 9:51 PM, Ben Finney wrote:

Terry Reedytjre...@udel.edu  writes:


On 10/14/2011 9:05 PM, Chris Angelico wrote:



That tests if the object is already an int; the OP asked if a string
contains an integer.


The misleading subject line did not. It should have been How to test
if a string contains an integer?



Which would still be misleading :-)

Even better is “How to test whether a string is a valid representation
of an integer?”


I agree, but that is more than I would ask of a newbie, whereas asking 
people to ask the same question in subject line and text, even if the 
question is inadequate, is reasonable.



I say that's better because it gets to the relevant point of asking
*which* representations you want to test for – what qualifies as valid
for your particular use case, and what does not. There's no single right
answer; the programmer must choose exactly what they want to test for.


Yes. Even the wrong subject line question is ambiguous, as any of int, 
bool, float, complex, decimal.Decimal, and fractions.Fraction can have 
an integer value, as might user class instances, and, of course, 
depending on context, bytes and strings.


--
Terry Jan Reedy


--
http://mail.python.org/mailman/listinfo/python-list

Re: How to test if object is an integer?

[Chris Angelico]

On Sat, Oct 15, 2011 at 10:44 AM, MrPink tdsimp...@gmail.com wrote:
 Is there a function in Python that can be used to test if the value in
 a string is an integer?  I had to make one up for myself and it looks
 like this:

 def isInt(s):
    try:
        i = int(s)
        return True
    except ValueError:
        return False

There's some ambiguity in the definition of is an integer. For
instance, is 0x100 an integer? Is 0800? If your definition of is
an integer is can be passed to int() without triggering an
exception (which is probably the most useful), then your above code
is about perfect. The only change I'd make is to not have an isInt
function at all, but simply to try/except at the point where you need
to make the conversion.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list