Re: Checking for an exception

[Cameron Simpson]

On 25Jun2017 13:47, Ben Finney  wrote:

Steve D'Aprano  writes:

[…] the result of passing a non-exception to raise is to raise an
exception, so I cannot trivially distinguish between "caller passes an
exception" and "caller passes a non-exception" (result is still an
exception).


Yes, hence my characterising this problem as the caller's problem.

I'd say: document the expectation that the value will be an exception,
use it based on that specification, and let the caller deal with the
consequences of violating that expectation.


I'm a "fail early" kind of guy, and to me Steve's approach is in the same 
spirit as raising ValueError when a function is handed invalid arguments.


Particularly if the mistake is easy to make, having one's attention brought to 
it immediately (at "declaration" time, since Steve's example is a decorator), 
seems very desirable.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: environment variable

[Cameron Simpson]

On 24Jun2017 21:31, Gene Heskett  wrote:

On Saturday 24 June 2017 21:03:18 Steve D'Aprano wrote:

On Sun, 25 Jun 2017 08:39 am, Gene Heskett wrote:
> On Saturday 24 June 2017 16:49:25 Smith wrote:
>> Hello to all,
>> I wanted to ask you how I could delete a line of an environment
>> variable (PATH)

[...]
> export PATH=
> but be prepared to type the full path to anything you want to run.
> It a PITA, or a cast iron bitch or whatever definition fits.

Are you serious? The OP asked how to delete *one line* from an
environment variable, and you respond by deleting them all?


$PATH is a single line in the env. The OP asked how to delete it, so I
told him. If he had asked how to get rid of a single piece of the $PATH,
then my answer would have been longer and more precise.


English is clearly the OP's second language, and his post was pretty clear that 
he did indeed want to remove a single item from the things named in $PATH. He 
even provided example code showing exactly what he wanted to remove, and which 
demonstrated that he knew it was a colon separated string, which he was 
presenting as lines for purposes of clarity.


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Checking for an exception

[Ben Finney]

Steve D'Aprano  writes:

> […] the result of passing a non-exception to raise is to raise an
> exception, so I cannot trivially distinguish between "caller passes an
> exception" and "caller passes a non-exception" (result is still an
> exception).

Yes, hence my characterising this problem as the caller's problem.

I'd say: document the expectation that the value will be an exception,
use it based on that specification, and let the caller deal with the
consequences of violating that expectation.

-- 
 \“I went to court for a parking ticket; I pleaded insanity. I |
  `\   said ‘Your Honour, who in their right mind parks in the passing |
_o__)   lane?’” —Steven Wright |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: environment variable

[Gene Heskett]

On Saturday 24 June 2017 21:03:18 Steve D'Aprano wrote:

> On Sun, 25 Jun 2017 08:39 am, Gene Heskett wrote:
> > On Saturday 24 June 2017 16:49:25 Smith wrote:
> >> Hello to all,
> >> I wanted to ask you how I could delete a line of an environment
> >> variable (PATH)
>
> [...]
>
> > export PATH=
> >
> > but be prepared to type the full path to anything you want to run. 
> > It a PITA, or a cast iron bitch or whatever definition fits.
>
> Are you serious? The OP asked how to delete *one line* from an
> environment variable, and you respond by deleting them all?
>
$PATH is a single line in the env. The OP asked how to delete it, so I 
told him. If he had asked how to get rid of a single piece of the $PATH, 
then my answer would have been longer and more precise.

> "The ashtray in my car is full, how do I clean it?"

Take it out and dump it. Put it back in.  Or leave it out, the car will 
not care, it will still haul your butt as long as theres oil in the 
engine and gas in the tank.

> "Throw the car away and walk everywhere."

This is one of those posts that I should have ignored as this list has an 
abundance of standup comics anyway.
>
>
>
>
> --
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and
> sure enough, things got worse.


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking for an exception

[Steve D'Aprano]

On Sun, 25 Jun 2017 10:49 am, Ben Finney wrote:

> Steve D'Aprano  writes:
> 
>> What's the right/best way to test whether an object is an exception
>> ahead of time? (That is, without trying to raise from it.)
> 
> This being Python, it is Easier to Ask for Forgiveness than for
> Permission.

Sometimes...


> The corollary of that is, if you try to ask permission first (to Look
> Before You Leap), it will likely not be as easy as simply using the
> object as you intend to use it.
> 
> So, EAFP would suggest just raising the object:
> 
> raise obj

Unfortunately it's not that simple, as the result of passing a non-exception to
raise is to raise an exception, so I cannot trivially distinguish
between "caller passes an exception" and "caller passes a non-exception"
(result is still an exception).

I could write something like:

def is_exception(obj):
try:
raise obj  # This unconditionally raises.
except TypeError:
# Either obj is an instance or subclass of TypeError,
# or it's some arbitrary non-exception object.
if isinstance(obj, TypeError):
return True
try:
return issubclass(obj, TypeError)
except TypeError:
return False
except:
# Any exception other than TypeError means obj is that exception.
return True
else:
# In Python 2.4 and 2.5 you can (but shouldn't) raise strings.
assert isinstance(obj, str)
return False


but I don't think that's much of an improvement :-)




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking for an exception

[Steve D'Aprano]

On Sun, 25 Jun 2017 09:37 am, Cameron Simpson wrote:

> On 24Jun2017 20:31, Steve D'Aprano  wrote:
>>What's the right/best way to test whether an object is an exception ahead of
>>time? (That is, without trying to raise from it.)
>>
>>I have:
>>
>>return (isinstance(obj, type) and issubclass(obj, BaseException)
>>or isinstance(obj, BaseException))
> 
> I haven't a better idea.
> 
> Are you supporting Python 2 here, where one can raise bare exceptions instead
> of instances?

Both Python 2 and Python 3 support raising either exception classes or exception
instances:

raise ValueError
raise ValueError('message')

both work. (If you use a type alone, the raise statement automatically
instantiates it, otherwise it just uses the instance you give.)


> Also, do you need the "isinstance(obj, type)" precursor to the 
> issubclass?

Yes, because annoyingly issubclass raises if you pass something which isn't a
type, instead of just returning False:


py> issubclass(99, int)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: issubclass() arg 1 must be a class



> Might you be better with:
> 
>   return ( issubclass(obj, BaseException)
>if isinstance(obj, type)
>else isinstance(obj, BaseException)
>  )
> 
> ?

I didn't think of that. If I were just supporting Python 2.7 or 3.6, I think I
would prefer that, but for my sins I'm supporting 2.4 :-(


> Curious: why do you need to test this? Some function which may return a
> "value" or an exception?

I have a decorator which converts exceptions in the decorated function from one
type to another. It requires as two arguments:

- the exception to be caught: an exception type, or a tuple of exception types

- the exception to be re-raised: an exception type, or an exception instance


If the caller provides bad arguments to the decorator, I want to raise
*immediately*, not when the decorated function is called.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: environment variable

[Steve D'Aprano]

On Sun, 25 Jun 2017 08:39 am, Gene Heskett wrote:

> On Saturday 24 June 2017 16:49:25 Smith wrote:
> 
>> Hello to all,
>> I wanted to ask you how I could delete a line of an environment
>> variable (PATH)
[...]

> export PATH=
> 
> but be prepared to type the full path to anything you want to run.  It a
> PITA, or a cast iron bitch or whatever definition fits.

Are you serious? The OP asked how to delete *one line* from an environment
variable, and you respond by deleting them all?

"The ashtray in my car is full, how do I clean it?"

"Throw the car away and walk everywhere."




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[Steve D'Aprano]

On Sun, 25 Jun 2017 07:17 am, Peter Otten wrote:

> Then I'd fix the name manually...

The file name isn't broken.


What's broken is parts of the OP's code which assumes that non-ASCII file names
are broken...



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking for an exception

[Ben Finney]

Steve D'Aprano  writes:

> What's the right/best way to test whether an object is an exception
> ahead of time? (That is, without trying to raise from it.)

This being Python, it is Easier to Ask for Forgiveness than for
Permission.

The corollary of that is, if you try to ask permission first (to Look
Before You Leap), it will likely not be as easy as simply using the
object as you intend to use it.

So, EAFP would suggest just raising the object:

raise obj

and thereby reveal the bug in the *caller's* code if it tries to pass a
not-exception object for that purpose.

> return (isinstance(obj, type) and issubclass(obj, BaseException)
> or isinstance(obj, BaseException))
>
> Any better ideas?

It's clumsy and noisy, but I think that's the cost of trying to Look
Before You Leap in code.

-- 
 \ “My girlfriend has a queen sized bed; I have a court jester |
  `\   sized bed. It's red and green and has bells on it, and the ends |
_o__) curl up.” —Steven Wright |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking for an exception

[Cameron Simpson]

On 24Jun2017 20:31, Steve D'Aprano  wrote:

What's the right/best way to test whether an object is an exception ahead of
time? (That is, without trying to raise from it.)

I have:

return (isinstance(obj, type) and issubclass(obj, BaseException)
   or isinstance(obj, BaseException))


I haven't a better idea.

Are you supporting Python 2 here, where one can raise bare exceptions instead 
of instances? Also, do you need the "isinstance(obj, type)" precursor to the 
issubclass?


Might you be better with:

 return ( issubclass(obj, BaseException)
  if isinstance(obj, type)
  else isinstance(obj, BaseException)
)

?

Curious: why do you need to test this? Some function which may return a "value" 
or an exception?


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: environment variable

[Gene Heskett]

On Saturday 24 June 2017 16:49:25 Smith wrote:

> Hello to all,
> I wanted to ask you how I could delete a line of an environment
> variable (PATH)
>
> ~/Scaricati/pycharm-2017.1.4/bin$ echo $PATH | sed s/:/'\n'/g
> /usr/local/sbin
> /usr/local/bin
> /usr/sbin
> /usr/bin
> /sbin
> /bin
> /usr/games
> /usr/local/games
> /snap/bin
> /path/to/my/program  ---> linea da eliminare
> /home/dbruno/Scaricati/pycharm-2017.1
>
> Do I unset PATH delete all content?
> I have not tried yet
>
> I downloaded pycharm and wanted to make it executable from the command
> line without accessing the bin folder and launch the bash script:
>
> dbruno@dbruno:~/Scaricati/pycharm-2017.1.4$ ls -la
> total 48
> drwxrwxr-x 10 dbruno dbruno 4096 giu 24 10:23 .
> drwxr-xr-x 18 dbruno dbruno 4096 giu 24 10:23 ..
> drwxrwxr-x  2 dbruno dbruno 4096 giu 24 10:25 bin
> -rw-r--r--  1 dbruno dbruno   14 giu 13 14:48 build.txt
> drwxrwxr-x  2 dbruno dbruno 4096 giu 24 10:23 debug-eggs
> drwxrwxr-x  2 dbruno dbruno 4096 giu 24 10:23 help
> drwxrwxr-x 17 dbruno dbruno 4096 giu 24 10:23 helpers
> -rw-r--r--  1 dbruno dbruno 1887 giu 13 14:48 Install-Linux-tar.txt
> drwxrwxr-x  4 dbruno dbruno 4096 giu 24 10:23 jre64
> drwxrwxr-x  4 dbruno dbruno 4096 giu 24 10:23 lib
> drwxrwxr-x  2 dbruno dbruno 4096 giu 24 10:23 license
> drwxrwxr-x 52 dbruno dbruno 4096 giu 24 10:23 plugin
>
> :~/Scaricati/pycharm-2017.1.4/bin$ ls -la
>
> total 7120
> drwxrwxr-x  2 dbruno dbruno4096 giu 24 10:25 .
> drwxrwxr-x 10 dbruno dbruno4096 giu 24 10:23 ..
> -rw-rw-r--  1 dbruno dbruno   0 giu 24 20:18 0M?+
> -rwxr-xr-x  1 dbruno dbruno 221 giu 13 14:48 format.sh
> -rwxr-xr-x  1 dbruno dbruno   23072 giu 13 14:48 fsnotifier
> -rwxr-xr-x  1 dbruno dbruno   29648 giu 13 14:48 fsnotifier64
> -rwxr-xr-x  1 dbruno dbruno   26453 giu 13 14:48 fsnotifier-arm
> -rw-r--r--  1 dbruno dbruno   10804 giu 13 14:48 idea.properties
> -rwxr-xr-x  1 dbruno dbruno 272 giu 13 14:48 inspect.sh
> -rw-r--r--  1 dbruno dbruno 3449944 giu 13 14:48
> libyjpagent-linux64.so -rw-r--r--  1 dbruno dbruno 3679036 giu 13
> 14:48 libyjpagent-linux.so -rw-r--r--  1 dbruno dbruno2236 giu 13
> 14:48 log.xml
> -rwxr-xr-x  1 dbruno dbruno 410 giu 13 14:48 printenv.py
> -rw-r--r--  1 dbruno dbruno 329 giu 13 14:48 pycharm64.vmoptions
> -rw-r--r--  1 dbruno dbruno   10281 giu 13 14:48 pycharm.png
> -rwxr-xr-x  1 dbruno dbruno6860 giu 13 14:48 pycharm.sh
> -rw-r--r--  1 dbruno dbruno 337 giu 13 14:48 pycharm.vmoptions
> -rwxr-xr-x  1 dbruno dbruno 590 giu 13 14:48 restart.py
>
> You can help me ?
>
> Thank you

export PATH=

but be prepared to type the full path to anything you want to run.  It a 
PITA, or a cast iron bitch or whatever definition fits.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: environment variable

[Michael F. Stemper]

On 2017-06-24 15:49, Smith wrote:

Hello to all,
I wanted to ask you how I could delete a line of an environment variable 
(PATH)


~/Scaricati/pycharm-2017.1.4/bin$ echo $PATH | sed s/:/'\n'/g
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/usr/games
/usr/local/games
/snap/bin
/path/to/my/program  ---> linea da eliminare
/home/dbruno/Scaricati/pycharm-2017.1

Do I unset PATH delete all content?


That would not be a good idea.

Although this seems to be more of a bash question than a python
question, I'll take a crack at it.

The first question is: do you want to eliminate that directory from your
$PATH forever and ever, or do you just want to eliminate it in some
shell instances?

If the first is true, find out where it's being added to your $PATH,
such as ~/.profile or ~/.bashrc and comment out that line.

If the second is true, you could clear it out in real time, with a
little something like:
  . export PATH=`echo $PATH | sed 's,:/path/to/my/program,,'`

This has not been tested, but is reasonably close to what's needed. Note
that you need the period "." at the beginning to make sure that this
happens in the current shell.


I downloaded pycharm and wanted to make it executable from the command 
line without accessing the bin folder and launch the bash script:


But, here's the big question. How does having "/path/to/my/program" in
your $PATH variable prevent you from doing that? The simple answer is
that it doesn't.

You might want to ADD something to your $PATH, such as
"~/Scaricati/pycharm-2017.1.4/bin", but taking something out will not
do what you want.

Another approach would be to ADD something to your .bash_aliases file,
such as:

  pych()
  {
~/Scaricati/pycharm-2017.1.4/bin/pycharm.sh $*
  }

Then, you can type "pych" at the command line, and bash will interpret
it as the above command, including arguments.

Two words of caution here:
1. I haven't tested this code, either.
3. If pycharm does not know its location (which is not necessarily
   the current directory), it might not be able to find the many
   accessory files that are there.


--
Michael F. Stemper
I feel more like I do now than I did when I came in.
--
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[Peter Otten]

Rod Person wrote:

> On Sat, 24 Jun 2017 21:28:45 +0200
> Peter Otten <__pete...@web.de> wrote:
> 
>> Rod Person wrote:
>> 
>> > Hi,
>> > 
>> > I'm working on a program that will walk a file system and clean the
>> > id3 tags of mp3 and flac files, everything is working great until
>> > the follow file is found
>> > 
>> > '06 - Todd's Song (Post-Spiderland Song in Progress).flac'
>> > 
>> > for some reason that I can't understand os.walk() returns this file
>> > name as
>> > 
>> > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in
>> > Progress).flac'
>> > 
>> > which then causes more hell than a little bit for me. I'm not
>> > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
>> > about it.
>> 
>> >>> b"\xe2\x80\x99".decode("utf-8")
>> '’'
>> >>> unicodedata.name(_)
>> 'RIGHT SINGLE QUOTATION MARK'
>> 
>> So it's '’' rather than "'".
>> 
>> > The script is Python 3, the file system it is running on is a hammer
>> > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS
>> > which runs some kind of Linux so it probably ext3/4. The files came
>> > from various system (Mac, Windows, FreeBSD).
>> 
>> There seems to be a mismatch between the assumed and the actual file
>> system encoding somewhere in this mix. Is this the only glitch or are
>> there similar problems with other non-ascii characters?
>> 
> 
> This is the only glitch as in file names so far.
> 

Then I'd fix the name manually...

-- 
https://mail.python.org/mailman/listinfo/python-list

environment variable

[Smith]

Hello to all,
I wanted to ask you how I could delete a line of an environment variable 
(PATH)


~/Scaricati/pycharm-2017.1.4/bin$ echo $PATH | sed s/:/'\n'/g
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/usr/games
/usr/local/games
/snap/bin
/path/to/my/program  ---> linea da eliminare
/home/dbruno/Scaricati/pycharm-2017.1

Do I unset PATH delete all content?
I have not tried yet

I downloaded pycharm and wanted to make it executable from the command 
line without accessing the bin folder and launch the bash script:


dbruno@dbruno:~/Scaricati/pycharm-2017.1.4$ ls -la
total 48
drwxrwxr-x 10 dbruno dbruno 4096 giu 24 10:23 .
drwxr-xr-x 18 dbruno dbruno 4096 giu 24 10:23 ..
drwxrwxr-x  2 dbruno dbruno 4096 giu 24 10:25 bin
-rw-r--r--  1 dbruno dbruno   14 giu 13 14:48 build.txt
drwxrwxr-x  2 dbruno dbruno 4096 giu 24 10:23 debug-eggs
drwxrwxr-x  2 dbruno dbruno 4096 giu 24 10:23 help
drwxrwxr-x 17 dbruno dbruno 4096 giu 24 10:23 helpers
-rw-r--r--  1 dbruno dbruno 1887 giu 13 14:48 Install-Linux-tar.txt
drwxrwxr-x  4 dbruno dbruno 4096 giu 24 10:23 jre64
drwxrwxr-x  4 dbruno dbruno 4096 giu 24 10:23 lib
drwxrwxr-x  2 dbruno dbruno 4096 giu 24 10:23 license
drwxrwxr-x 52 dbruno dbruno 4096 giu 24 10:23 plugin

:~/Scaricati/pycharm-2017.1.4/bin$ ls -la
total 7120
drwxrwxr-x  2 dbruno dbruno4096 giu 24 10:25 .
drwxrwxr-x 10 dbruno dbruno4096 giu 24 10:23 ..
-rw-rw-r--  1 dbruno dbruno   0 giu 24 20:18 0M?+
-rwxr-xr-x  1 dbruno dbruno 221 giu 13 14:48 format.sh
-rwxr-xr-x  1 dbruno dbruno   23072 giu 13 14:48 fsnotifier
-rwxr-xr-x  1 dbruno dbruno   29648 giu 13 14:48 fsnotifier64
-rwxr-xr-x  1 dbruno dbruno   26453 giu 13 14:48 fsnotifier-arm
-rw-r--r--  1 dbruno dbruno   10804 giu 13 14:48 idea.properties
-rwxr-xr-x  1 dbruno dbruno 272 giu 13 14:48 inspect.sh
-rw-r--r--  1 dbruno dbruno 3449944 giu 13 14:48 libyjpagent-linux64.so
-rw-r--r--  1 dbruno dbruno 3679036 giu 13 14:48 libyjpagent-linux.so
-rw-r--r--  1 dbruno dbruno2236 giu 13 14:48 log.xml
-rwxr-xr-x  1 dbruno dbruno 410 giu 13 14:48 printenv.py
-rw-r--r--  1 dbruno dbruno 329 giu 13 14:48 pycharm64.vmoptions
-rw-r--r--  1 dbruno dbruno   10281 giu 13 14:48 pycharm.png
-rwxr-xr-x  1 dbruno dbruno6860 giu 13 14:48 pycharm.sh
-rw-r--r--  1 dbruno dbruno 337 giu 13 14:48 pycharm.vmoptions
-rwxr-xr-x  1 dbruno dbruno 590 giu 13 14:48 restart.py

You can help me ?

Thank you
--
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[MRAB]

On 2017-06-24 20:47, Rod Person wrote:

On Sat, 24 Jun 2017 13:28:55 -0600
Michael Torrie  wrote:


On 06/24/2017 12:57 PM, Rod Person wrote:
> Hi,
> 
> I'm working on a program that will walk a file system and clean the

> id3 tags of mp3 and flac files, everything is working great until
> the follow file is found
> 
> '06 - Todd's Song (Post-Spiderland Song in Progress).flac'
> 
> for some reason that I can't understand os.walk() returns this file

> name as
> 
> '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in
> Progress).flac'  


That's basically a UTF-8 string there:

$ python3
>>> a= b'06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in  
Progress).flac'
>>> print (a.decode('utf-8'))  
06 - Todd’s Song (Post-Spiderland Song in Progress).flac
>>>  


The NAS is just happily reading the UTF-8 bytes and passing them on
the wire.

> which then causes more hell than a little bit for me. I'm not
> understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
> about it.  


It's clearly not an apostrophe in the original filename, but probably
U+2019 (’)

> The script is Python 3, the file system it is running on is a hammer
> filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS
> which runs some kind of Linux so it probably ext3/4. The files came
> from various system (Mac, Windows, FreeBSD).  


It's the file serving protocol that dictates how filenames are
transmitted. In your case it's probably smb. smb (samba) is just
passing the native bytes along from the file system.  Since you know
the native file system is just UTF-8, you can just decode every
filename from utf-8 bytes into unicode.


This is the impression that I was under, my unicode is that strong, so
maybe my understand is off...but I tried.

file_name = file_name.decode('utf-8', 'ignore')

but when I get to my logging code:

logfile.write(file_name)

that throws the error:
UnicodeEncodeError: 'ascii' codec can't encode characters in
position 39-41: ordinal not in range(128)


Your logfile was opened with the 'ascii' encoding, so you can't write 
anything outside the ASCII range.


Open it with the 'utf-8' encoding instead.
--
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[Andre Müller]

Can os.fsencode and os.fsdecode help? I've seen it somewhere.
I've never used it.

To fix encodings, sometimes I use the module ftfy

Greetings
Andre
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[Rod Person]

On Sat, 24 Jun 2017 13:28:55 -0600
Michael Torrie  wrote:

> On 06/24/2017 12:57 PM, Rod Person wrote:
> > Hi,
> > 
> > I'm working on a program that will walk a file system and clean the
> > id3 tags of mp3 and flac files, everything is working great until
> > the follow file is found
> > 
> > '06 - Todd's Song (Post-Spiderland Song in Progress).flac'
> > 
> > for some reason that I can't understand os.walk() returns this file
> > name as
> > 
> > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in
> > Progress).flac'  
> 
> That's basically a UTF-8 string there:
> 
> $ python3
> >>> a= b'06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in  
> Progress).flac'
> >>> print (a.decode('utf-8'))  
> 06 - Todd’s Song (Post-Spiderland Song in Progress).flac
> >>>  
> 
> The NAS is just happily reading the UTF-8 bytes and passing them on
> the wire.
> 
> > which then causes more hell than a little bit for me. I'm not
> > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
> > about it.  
> 
> It's clearly not an apostrophe in the original filename, but probably
> U+2019 (’)
> 
> > The script is Python 3, the file system it is running on is a hammer
> > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS
> > which runs some kind of Linux so it probably ext3/4. The files came
> > from various system (Mac, Windows, FreeBSD).  
> 
> It's the file serving protocol that dictates how filenames are
> transmitted. In your case it's probably smb. smb (samba) is just
> passing the native bytes along from the file system.  Since you know
> the native file system is just UTF-8, you can just decode every
> filename from utf-8 bytes into unicode.

This is the impression that I was under, my unicode is that strong, so
maybe my understand is off...but I tried.

file_name = file_name.decode('utf-8', 'ignore')

but when I get to my logging code:

logfile.write(file_name)

that throws the error:
UnicodeEncodeError: 'ascii' codec can't encode characters in
position 39-41: ordinal not in range(128)


-- 
Rod

http://www.rodperson.com

Who at Clitorius fountain thirst remove 
Loath Wine and, abstinent, meer Water love.

 - Ovid
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[Rod Person]

On Sat, 24 Jun 2017 21:28:45 +0200
Peter Otten <__pete...@web.de> wrote:

> Rod Person wrote:
> 
> > Hi,
> > 
> > I'm working on a program that will walk a file system and clean the
> > id3 tags of mp3 and flac files, everything is working great until
> > the follow file is found
> > 
> > '06 - Todd's Song (Post-Spiderland Song in Progress).flac'
> > 
> > for some reason that I can't understand os.walk() returns this file
> > name as
> > 
> > '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in
> > Progress).flac'
> > 
> > which then causes more hell than a little bit for me. I'm not
> > understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
> > about it.  
> 
> >>> b"\xe2\x80\x99".decode("utf-8")  
> '’'
> >>> unicodedata.name(_)  
> 'RIGHT SINGLE QUOTATION MARK'
> 
> So it's '’' rather than "'".
> 
> > The script is Python 3, the file system it is running on is a hammer
> > filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS
> > which runs some kind of Linux so it probably ext3/4. The files came
> > from various system (Mac, Windows, FreeBSD).  
> 
> There seems to be a mismatch between the assumed and the actual file
> system encoding somewhere in this mix. Is this the only glitch or are
> there similar problems with other non-ascii characters?
> 

This is the only glitch as in file names so far.

-- 
Rod

http://www.rodperson.com

Who at Clitorius fountain thirst remove 
Loath Wine and, abstinent, meer Water love.

 - Ovid
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[Peter Otten]

Rod Person wrote:

> Hi,
> 
> I'm working on a program that will walk a file system and clean the id3
> tags of mp3 and flac files, everything is working great until the
> follow file is found
> 
> '06 - Todd's Song (Post-Spiderland Song in Progress).flac'
> 
> for some reason that I can't understand os.walk() returns this file
> name as
> 
> '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac'
> 
> which then causes more hell than a little bit for me. I'm not
> understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
> about it.

>>> b"\xe2\x80\x99".decode("utf-8")
'’'
>>> unicodedata.name(_)
'RIGHT SINGLE QUOTATION MARK'

So it's '’' rather than "'".

> The script is Python 3, the file system it is running on is a hammer
> filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which
> runs some kind of Linux so it probably ext3/4. The files came from
> various system (Mac, Windows, FreeBSD).

There seems to be a mismatch between the assumed and the actual file system 
encoding somewhere in this mix. Is this the only glitch or are there similar 
problems with other non-ascii characters?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[Michael Torrie]

On 06/24/2017 12:57 PM, Rod Person wrote:
> Hi,
> 
> I'm working on a program that will walk a file system and clean the id3
> tags of mp3 and flac files, everything is working great until the
> follow file is found
> 
> '06 - Todd's Song (Post-Spiderland Song in Progress).flac'
> 
> for some reason that I can't understand os.walk() returns this file
> name as
> 
> '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac'

That's basically a UTF-8 string there:

$ python3
>>> a= b'06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in
Progress).flac'
>>> print (a.decode('utf-8'))
06 - Todd’s Song (Post-Spiderland Song in Progress).flac
>>>

The NAS is just happily reading the UTF-8 bytes and passing them on the
wire.

> which then causes more hell than a little bit for me. I'm not
> understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
> about it.

It's clearly not an apostrophe in the original filename, but probably
U+2019 (’)

> The script is Python 3, the file system it is running on is a hammer
> filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which
> runs some kind of Linux so it probably ext3/4. The files came from
> various system (Mac, Windows, FreeBSD).

It's the file serving protocol that dictates how filenames are
transmitted. In your case it's probably smb. smb (samba) is just passing
the native bytes along from the file system.  Since you know the native
file system is just UTF-8, you can just decode every filename from utf-8
bytes into unicode.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[MRAB]

On 2017-06-24 19:57, Rod Person wrote:

Hi,

I'm working on a program that will walk a file system and clean the id3
tags of mp3 and flac files, everything is working great until the
follow file is found

'06 - Todd's Song (Post-Spiderland Song in Progress).flac'

for some reason that I can't understand os.walk() returns this file
name as

'06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac'

which then causes more hell than a little bit for me. I'm not
understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
about it.

The script is Python 3, the file system it is running on is a hammer
filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which
runs some kind of Linux so it probably ext3/4. The files came from
various system (Mac, Windows, FreeBSD).


If you treat it as a bytestring b'\xe2\x80\x99' and decode it:

>>> c = b'\xe2\x80\x99'.decode('utf-8')
>>> ascii(c)
"'\\u2019'"
>>> import unicodedata
>>> unicodedata.name(c)
'RIGHT SINGLE QUOTATION MARK'

It's not an apostrophe, it's '\u2019' ('\N{RIGHT SINGLE QUOTATION MARK}').

It looks like the filename is encoded as UTF-8, but Python thinks that 
the filesystem encoding is something like Latin-1.

--
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[John Ladasky]

On Saturday, June 24, 2017 at 12:07:05 PM UTC-7, Rod Person wrote:
> Hi,
> 
> I'm working on a program that will walk a file system and clean the id3
> tags of mp3 and flac files, everything is working great until the
> follow file is found
> 
> '06 - Todd's Song (Post-Spiderland Song in Progress).flac'
> 
> for some reason that I can't understand os.walk() returns this file
> name as
> 
> '06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac'
> 
> which then causes more hell than a little bit for me. I'm not
> understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
> about it.

That's a "right single quotation mark" character in Unicode.

http://unicode.scarfboy.com/?s=E28099

Something in your code is choosing to interpret the text variable as an 
old-fashioned byte array of characters, where every character is represented by 
a single byte.  That works as long as the file name only uses characters from 
the old ASCII set, but there are only 128 of those.

> The script is Python 3, the file system it is running on is a hammer
> filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which
> runs some kind of Linux so it probably ext3/4. The files came from
> various system (Mac, Windows, FreeBSD).

Since you are working in Python3, you have the ability to call the .encode() 
and .decode() methods to translate between Unicode and byte character arrays 
(which you still need on occasion).


> 
> -- 
> Rod
> 
> http://www.rodperson.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: os.walk the apostrophe and unicode

[alister]

On Sat, 24 Jun 2017 14:57:21 -0400, Rod Person wrote:

> \xe2\x80\x99,

because the file name has been created using "Right single quote" instead 
of apostrophe, the glyphs look identical in many fonts.




-- 
"If you understand what you're doing, you're not learning anything."
-- A. L.
-- 
https://mail.python.org/mailman/listinfo/python-list

os.walk the apostrophe and unicode

[Rod Person]

Hi,

I'm working on a program that will walk a file system and clean the id3
tags of mp3 and flac files, everything is working great until the
follow file is found

'06 - Todd's Song (Post-Spiderland Song in Progress).flac'

for some reason that I can't understand os.walk() returns this file
name as

'06 - Todd\xe2\x80\x99s Song (Post-Spiderland Song in Progress).flac'

which then causes more hell than a little bit for me. I'm not
understand why apostrophe(') becomes \xe2\x80\x99, or what I can do
about it.

The script is Python 3, the file system it is running on is a hammer
filesystem on DragonFlyBSD. The audio files reside on a QNAP NAS which
runs some kind of Linux so it probably ext3/4. The files came from
various system (Mac, Windows, FreeBSD).


-- 
Rod

http://www.rodperson.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: New to Python - Career question

[J. Clarke]

In article , 
larry.mart...@gmail.com says...
> 
> On Tue, Jun 6, 2017 at 6:37 PM, Marko Rauhamaa  wrote:
> > pta...@gmail.com:
> >
> >> New to Python and have been at it for about a month now. I'm doing
> >> well and like it very much. Considering a career change down the road
> >> and have been wondering... What are the job prospects for a middle age
> >> entry level programmer. Just trying to get a better understanding
> >> where I stand career wise. Appreciate all feed back. Thanks!
> >
> > Different employers hire differently. I have hired several people for my
> > employer, and age has never been a concern. Python is also an important
> > tool where I work.
> >
> > However, the problem in our field is that you have to be quite good to
> > be truly useful. Unfortunately, it seems that only a minority with a
> > formal degree are good enough. On the other hand, I work with some great
> > software developers who don't have a degree at all.
> >
> > One good way to become a good developer and also test oneself is to pick
> > a free software project online a become a contributor. Your commit log
> > entries on GitHub advertise you much better than any pretty-printed
> > résumé.
> 
> I work 70 or more hours a week and have a family and hobbies and a
> personal life. I do not have time to contribute to any open source
> projects. And because I cannot show anything on GitHub I have often
> been summarily disqualified for even getting an interview. (I have
> donated money to open source projects.)

If you're looking to change careers to programmer and you have hobbies that 
don't include programming, reconsider the change.  
-- 
https://mail.python.org/mailman/listinfo/python-list

Fw: Unable to convert pandas object to string

[Albert-Jan Roskam]

From: Albert-Jan Roskam 
Sent: Saturday, June 24, 2017 11:26:26 AM
To: Paul Barry
Subject: Re: Unable to convert pandas object to string

(sorry for top posting)

Try using fillna('') to convert np.nan into empty strings. df['desc'] = 
df.desc.fillna(''). Btw, np.object already is what best approximates str. I 
wish np.object had its own sentinel value for missing data instead of np.nan, 
which is a float.

From: Python-list  on 
behalf of Paul Barry 
Sent: Saturday, June 24, 2017 9:44:54 AM
To: Bhaskar Dhariyal
Cc: python-list@python.org
Subject: Re: Unable to convert pandas object to string

Any chance you could post one line of data so we can see what we have to
work with?

Also - have you taken a look at Jake VanderPlas's notebooks? There's lot of
help with pandas to be found there:
https://github.com/jakevdp/PythonDataScienceHandbook

Paul.

On 24 June 2017 at 10:32, Bhaskar Dhariyal 
wrote:

> 
> Int64Index: 171594 entries, 0 to 63464
> Data columns (total 7 columns):
> project_id  171594 non-null object
> desc171594 non-null object
> goal171594 non-null float64
> keywords171594 non-null object
> diff_creat_laun 171594 non-null int64
> diff_laun_status171594 non-null int64
> diff_status_dead171594 non-null int64
> dtypes: float64(1), int64(3), object(3)
>
> not able to convert desc and keywords to string for preprocessing.
> Tried astype(str). Please help
> --
> https://mail.python.org/mailman/listinfo/python-list
>



--
Paul Barry, t: @barrypj  - w:
http://paulbarry.itcarlow.ie - e: paul.ba...@itcarlow.ie
Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland.
--
https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to ensure user calls methods in correct order?

[Thomas Nyberg]

On 06/24/2017 11:53 AM, Thomas Jollans wrote:
> If the data is modified in-place, maybe it makes sense to to use a class
> like the one you have, but then it's a bit bizarre to make your methods
> create an artificial side effect (self._a_dirty) - why don't you simply
> check for the actual effect of a(), whatever it is. If a() did some
> calculations and added a column to your dataset with the results, check
> for the existence of that column in b()! If calling b() invalidates some
> calculation results generated in c(), delete them! (In the functional
> setup above, b() would refuse to process a dataset that it has already
> processed)
> 
> -- Thomas
> 
> 
Thanks I like this a lot! This does fit in well with my setup and I
think it probably is the cleanest approach. It also makes it pretty
obvious to tell where in the process you are by just looking at the data
(though this is more of a convenience for myself than anyone else since
the internal steps are hidden).

Thanks a lot!

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking for an exception

[Steve D'Aprano]

On Sat, 24 Jun 2017 09:23 pm, mbyrne...@gmail.com wrote:

> On Saturday, June 24, 2017 at 11:31:11 AM UTC+1, Steve D'Aprano wrote:
>> What's the right/best way to test whether an object is an exception ahead of
>> time? (That is, without trying to raise from it.)
>> 
>> I have:
>> 
>> return (isinstance(obj, type) and issubclass(obj, BaseException)
>> or isinstance(obj, BaseException))
>> 
>> 
>> Any better ideas?


> Would something along these lines help?
> 
> import exceptions

py> import exceptions
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named 'exceptions'


What module are you importing?



> if 'ArithmeticError' in dir(exceptions):
> print 'is an exception'


I don't understand that. I'm not looking for strings which happen to match the
names of built-in exceptions, I'm looking to test for any object which is an
exception. To be an exception, you must inherit from BaseException.

(String exceptions did work until Python 2.5, but I don't care about them.)



> try:
> if exceptions.__getattribute__('ArithmeticError'):

You shouldn't call __getattribute__ directly -- that's reserved for Python's
use. You should call getattr() with an optional default value:

if getattr(exceptions, 'ArithmeticError', False):
print 'is an exception'
else:
print 'is an exception'


But in any case, no, I don't want to look up a string in some list of
exceptions. That won't test for exceptions that aren't in the list.

class MyCustomException(LookupError):
pass




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency

[mbyrnepr2]

On Thursday, June 22, 2017 at 12:16:28 PM UTC+1, kishan.samp...@gmail.com wrote:
> I want to write a common file in which It can add the frequency by adding 
> multiple csv file and if the same words are repeated in python then it should 
> add the frequency in the common file can any one help me please
> 
> 
> import re
> import operator
> import string
> 
> class words:
> def __init__(self,fh):
> self.fh = fh
> def read(self):
> for line in fh:
> yield line.split()
> 
> if __name__ == "__main__":
> frequency = {}
> document_text = open('data_analysis.csv', 'r')
> common1_file = open("common_file1.csv", "r")
> 
> text_string = document_text.read().lower()
> match_pattern = re.findall(r'\b[a-z]{3,15}\b', text_string)
> 
> text_string_one = common1_file.read().lower()
> match_pattern_one = re.findall(r'\b[a-z]{3,15}\b', text_string_one)
> #print("match_pattern"+(str(match_pattern)))
> for word in match_pattern:
> for word1 in match_pattern_one:
> count = frequency.get(word,0)
> count1 = frequency.get(word1,0)
> if word1 == word:
> frequency[word] = count + count1
> else:
> frequency[word] = count 
> 
> 
> frequency_list = frequency.keys()
> text_file = open("common_file1.csv", "w")
> for words in frequency_list:
> data = (words, frequency[words])
> print (data)
> #text_file = open("common_file1.csv", "w")
> #for i in data:
> #store_fre = (str(data)+"\n")
> text_file.write(str(data)+"\n")
> 
> 
> text_file.close()
> 
> 
> this is my code written by me til now but not getting satisfied results

Dictionary 'frequency' is updated only with values of 0.

If the aim is to get a count of occurrences for each word 
where the word exists in both input files, you could replace this:

for word in match_pattern: 
for word1 in match_pattern_one: 
count = frequency.get(word,0) 
count1 = frequency.get(word1,0) 
if word1 == word: 
frequency[word] = count + count1 
else: 
frequency[word] = count 

with this:

all_words = match_pattern + match_pattern_one
word_set = set(match_pattern) & set(match_pattern_one)
while word_set:
word = word_set.pop()
count = all_words.count(word)
frequency[word] = count

Other observations:
- Reading from and writing to the csv files is not utilsing the csv format
- The regex may be too restrictive and not all expected words extracted
- The output is written to one of the input files, overwriting the original 
content of the input file
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unable to convert pandas object to string

[Paul Barry]

Hi Bhaskar.

Please see attached PDF of a small Jupyter notebook.  As you'll see, the
data in the fields you mentioned are *already* strings.  What is it you are
trying to do here?

Paul.

On 24 June 2017 at 10:51, Bhaskar Dhariyal 
wrote:

> ​
>  train.csv
> 
> ​here it is thanks for quick reply
>
> On Sat, Jun 24, 2017 at 3:14 PM, Paul Barry 
> wrote:
>
>> Any chance you could post one line of data so we can see what we have to
>> work with?
>>
>> Also - have you taken a look at Jake VanderPlas's notebooks? There's lot
>> of help with pandas to be found there: https://github.com/jake
>> vdp/PythonDataScienceHandbook
>>
>> Paul.
>>
>> On 24 June 2017 at 10:32, Bhaskar Dhariyal 
>> wrote:
>>
>>> 
>>> Int64Index: 171594 entries, 0 to 63464
>>> Data columns (total 7 columns):
>>> project_id  171594 non-null object
>>> desc171594 non-null object
>>> goal171594 non-null float64
>>> keywords171594 non-null object
>>> diff_creat_laun 171594 non-null int64
>>> diff_laun_status171594 non-null int64
>>> diff_status_dead171594 non-null int64
>>> dtypes: float64(1), int64(3), object(3)
>>>
>>> not able to convert desc and keywords to string for preprocessing.
>>> Tried astype(str). Please help
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>>>
>>
>>
>>
>> --
>> Paul Barry, t: @barrypj  - w:
>> http://paulbarry.itcarlow.ie - e: paul.ba...@itcarlow.ie
>> Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland.
>>
>
>


-- 
Paul Barry, t: @barrypj  - w:
http://paulbarry.itcarlow.ie - e: paul.ba...@itcarlow.ie
Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland.
-- 
https://mail.python.org/mailman/listinfo/python-list

how to write add frequency in particular file by reading a csv file and then making a new file of multiple csv file by adding frequency

[Mark Byrne]

A problem (possibly the problem) is the lines which use the get function:
count = frequency.get(word,0)

Since the dictionary is empty at the start of the loop, the get function is
passing a value of 0 to count and count1.
The subsequent updates to the dictionary are applying a value of zero

As a result, the file being written to contains count values of 0 for each
word.

Possible fix is to replace this:

count = frequency.get(word,0)
count1 = frequency.get(word1,0)
if word1 == word:
frequency[word] = count + count1
else:
frequency[word] = count

with this:

if word1 == word:
if word in frequency:
frequency[word] += 1
else:
frequency[word] = 1
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Checking for an exception

[mbyrnepr2]

On Saturday, June 24, 2017 at 11:31:11 AM UTC+1, Steve D'Aprano wrote:
> What's the right/best way to test whether an object is an exception ahead of
> time? (That is, without trying to raise from it.)
> 
> I have:
> 
> return (isinstance(obj, type) and issubclass(obj, BaseException)
> or isinstance(obj, BaseException))
> 
> 
> Any better ideas?
> 
> 
> 
> -- 
> Steve
> “Cheer up,” they said, “things could be worse.” So I cheered up, and sure
> enough, things got worse.

Would something along these lines help?

import exceptions

if 'ArithmeticError' in dir(exceptions):
print 'is an exception'

try:
if exceptions.__getattribute__('ArithmeticError'):
print 'is an exception'
except AttributeError:
print 'is not an exception'

try:
getattr(exceptions, 'ArithmeticError')
print 'is not an exception'
except AttributeError:
print 'is not an exception'
-- 
https://mail.python.org/mailman/listinfo/python-list

Checking for an exception

[Steve D'Aprano]

What's the right/best way to test whether an object is an exception ahead of
time? (That is, without trying to raise from it.)

I have:

return (isinstance(obj, type) and issubclass(obj, BaseException)
or isinstance(obj, BaseException))


Any better ideas?



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to ensure user calls methods in correct order?

[Thomas Jollans]

On 22/06/17 17:31, Thomas Nyberg wrote:
> Thanks for the response! I see the reasoning, but I can't entirely
> square it with what I'm thinking. Probably it's either because I was
> hiding some of my intention (sorry if I wasted your time due to lack of
> details) or that I'm naive and regardless of intention doing something
> silly.
> 
> On 06/22/2017 04:40 PM, Steve D'Aprano wrote:
>> On Thu, 22 Jun 2017 11:53 pm, Thomas Nyberg wrote:
>>
>> Don't do that. It's fragile and an anti-pattern. Your methods have too much
>> coupling. If c() relies on b() being called first, then either b() or c()
>> aren't good methods. They don't do enough:
>>
>> - calling b() alone doesn't do enough, so you have to call c() next to get 
>> the
>> job done;
>>
>> - calling c() alone doesn't do enough, because it relies on b() being called
>> first.
>>
>>
>> There are a very few exceptions to this rule of thumb, such as opening
>> connections to databases or files or similar. They are mostly enshrined from
>> long practice, or justified by low-level APIs (that's how file systems and
>> databases work, and it's not practical to change that). But you should try 
>> very
>> hard to avoid creating new examples.
>>
>> Of course, I'm talking about your *public* methods. Private methods can be a 
>> bit
>> more restrictive, since it's only *you* who suffers if you do it wrong.
>>
>> Ideally, your methods should be written in a functional style with as little
>> shared state as possible. (Shared state introduces coupling between 
>> components,
>> and excessive coupling is *the* ultimate evil in programming.)
>>  
> 
> This makes perfect sense in general. An important detail I left out
> earlier was what exactly I meant by "users" of my code. Really all I
> meant was myself calling this code as a library and avoiding making
> mistakes. Currently what I'm doing is kind of a data storage, data
> processing/conversion, data generation pipeline. This different
> variables at the different steps in the pipeline are specified in a
> script. The functions in this script have _no coupling at all_ for all
> the reasons you stated. In fact, the only reason I'm introducing a class
> is because I'm trying to force their ordering in the way I described.
> 
> The ultimate goal here is to instead put a basic http server wrapper
> around the whole process. The reason I want to do this is to allow
> others (internal to the company) to make changes in the process
> interactively at different steps in the process (the steps are "natural"
> for the problem at hand) without editing the python scripts explicitly.
> 
> Of course I could force them to execute everything in one go, but due to
> the time required for the different steps, it's nicer to allow for some
> eye-balling of results (and possible changes and re-running) before
> continuing. Before this the main way of enforcing this was all done by
> the few of use editing the scripts, but now I'd like to make it an error
> for corrupted data to propagate down through the pipeline. Having a
> run_all() method or something similar will definitely exist (usually
> once you figure out the "right" parameters you don't change them much on
> re-runs), but having things broken apart is still very nice.
> 
> Would you still say this is a bad way of going about it? Thanks for the
> feedback. It's very helpful.


If a() does some processing, and then b() does something else to the
result of a(), then the natural way of calling the functions is probably
c(b(a(initial_data))), rather than a sequence of method or function
calls that hide some internal state. If the user wants to jump in and
look at what's going on, they can:

a_result = a()
# peruse the data
b_result = b(a_result)
# have some coffee
c_result = c(b_result)
# and so on.

If the data is modified in-place, maybe it makes sense to to use a class
like the one you have, but then it's a bit bizarre to make your methods
create an artificial side effect (self._a_dirty) - why don't you simply
check for the actual effect of a(), whatever it is. If a() did some
calculations and added a column to your dataset with the results, check
for the existence of that column in b()! If calling b() invalidates some
calculation results generated in c(), delete them! (In the functional
setup above, b() would refuse to process a dataset that it has already
processed)

-- Thomas


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Unable to convert pandas object to string

[Paul Barry]

Any chance you could post one line of data so we can see what we have to
work with?

Also - have you taken a look at Jake VanderPlas's notebooks? There's lot of
help with pandas to be found there:
https://github.com/jakevdp/PythonDataScienceHandbook

Paul.

On 24 June 2017 at 10:32, Bhaskar Dhariyal 
wrote:

> 
> Int64Index: 171594 entries, 0 to 63464
> Data columns (total 7 columns):
> project_id  171594 non-null object
> desc171594 non-null object
> goal171594 non-null float64
> keywords171594 non-null object
> diff_creat_laun 171594 non-null int64
> diff_laun_status171594 non-null int64
> diff_status_dead171594 non-null int64
> dtypes: float64(1), int64(3), object(3)
>
> not able to convert desc and keywords to string for preprocessing.
> Tried astype(str). Please help
> --
> https://mail.python.org/mailman/listinfo/python-list
>



-- 
Paul Barry, t: @barrypj  - w:
http://paulbarry.itcarlow.ie - e: paul.ba...@itcarlow.ie
Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland.
-- 
https://mail.python.org/mailman/listinfo/python-list

Unable to convert pandas object to string

[Bhaskar Dhariyal]

Int64Index: 171594 entries, 0 to 63464
Data columns (total 7 columns):
project_id  171594 non-null object
desc171594 non-null object
goal171594 non-null float64
keywords171594 non-null object
diff_creat_laun 171594 non-null int64
diff_laun_status171594 non-null int64
diff_status_dead171594 non-null int64
dtypes: float64(1), int64(3), object(3)

not able to convert desc and keywords to string for preprocessing.
Tried astype(str). Please help
-- 
https://mail.python.org/mailman/listinfo/python-list