Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Alan Gauld via Tutor]

On 10/01/18 20:20, eryk sun wrote:

> ... And working with COM via ctypes is also complex, which is why
> comtypes exists.

Or easier still Pythonwin (aka PyWin32).
I far prefer pythonwin over ctypes for any kind of COM work.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[eryk sun]

On Wed, Jan 10, 2018 at 12:59 PM, Albert-Jan Roskam
 wrote:
>
> I tried:
 from os.path import _getfullpathname
 _getfullpathname(r"H:")
> 'h:\\path\\to\\folder'
 import os
 os.getcwd()
> 'h:\\path\\to\\folder'
>
> I expected h:\ to be \\server\share\foo.

You called _getfullpathname (WinAPI GetFullPathName), not
_getfinalpathname (WinAPI GetFinalPathNameByHandle). GetFullPathName
works on the path as a string without touching the filesystem.
GetFinalPathNameByHandle reconstructs a path when given a handle to a
file or directory.

> The fact that the current working directory was returned was even more 
> unexpected.

"H:" or "H:relative/path" is relative to the working directory on
drive H:. The process only has one working directory, but
GetFullPathName also checks for environment variables such as "=H:".
The C runtime's _chdir function sets these magic variables, as does
Python's os.chdir function (we don't call C _chdir). WinAPI
SetCurrentDirectory does not set them. For example:

>>> os.chdir('Z:/Temp')
>>> win32api.GetEnvironmentVariable('=Z:')
'Z:\\Temp'
>>> os.path._getfullpathname('Z:relative')
'Z:\\Temp\\relative'

> Why would anybody *want* the drive letters? They are only useful because (a) 
> they save on
> keystrokes (b) they bypass the annoying limitation of the cd command on 
> windows, ie. it
> does not work with UNC paths.

Windows itself has no problem using a UNC path as the working
directory. That's a limit of the CMD shell.

SUBST drives can be used to access long paths. Since the substitution
occurs in the kernel, it avoids the MAX_PATH 260-character limit. Of
course, you can also use junctions and symlinks to access long paths,
or in Windows 10 simply enable long-path support.

> I know net use, pushd, subst. I use 'net use' for more or less permanent 
> drives and
> pushd/popd to get a temporary drive, available letter (cd nuisance).

`net.exe use` and CMD's PUSHD command (with a UNC path) both call
WinAPI WNetAddConnection2 to create a mapped network drive. The
difference is that net.exe can supply alternate credentials and create
a persistent mapping, while PUSHD uses the current user's credentials
and creates a non-persistent mapping.

If your account gets logged on with a UAC split token, the standard
and elevated tokens actually have separate logon sessions with
separate local-device mappings. You can enable a policy to link the
two logon sessions. Set a DWORD value of 1 named
"EnableLinkedConnections" in the key
"HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System", and
reboot.

subst.exe creates substitute paths using WinAPI DefineDosDevice.
Unlike the WNet API, this function doesn't use MPR (multiple provider
router) to create a direct link for the network provider (e.g.
\Device\LanmanRedirectory); doesn't create a linked connection when
EnableLinkedConnections is defined; and can't create a persistent
drive with stored credentials (though you can use an account logon
script for this). On the plus side, a drive mapped via subst.exe can
target any path.

> Interesting code! I have used the following, which uses SHGetFolderPath, ie. 
> without 'Known'.
> from win32com.shell import shell, shellcon
> desktop = shell.SHGetFolderPath(0, shellcon.CSIDL_DESKTOP, 0, 0)

SHGetFolderPath is usually fine, but still, it's outdated and
deprecated. win32com.shell doesn't wrap SHGetKnownFolderPath for some
reason, but you can still use the new known-folder API without ctypes.
Just create a KnownFolderManager instance. For example:

import pythoncom
from win32com.shell import shell

kfmgr = pythoncom.CoCreateInstance(shell.CLSID_KnownFolderManager, None,
pythoncom.CLSCTX_INPROC_SERVER, shell.IID_IKnownFolderManager)

desktop_path = kfmgr.GetFolder(shell.FOLDERID_Desktop).GetPath()

This doesn't work as conveniently for getting known folders of other
users. While the high-level SHGetKnownFolderPath function takes care
of loading the user profile and impersonating, we have to do this
ourselves when using a KnownFolderManager instance.

That said, to correct my previous post, you have to be logged on with
SeTcbPrivilege access (e.g. a SYSTEM service) to get and set other
users' known folders without their password. (If you have the password
you can use a regular logon instead of an S4U logon, and that works
fine.)

> Working with ctypes.wintypes is quite complex!

I wouldn't say ctypes is complex in general. But calling LsaLogonUser
is complex due to all of the structs that include variable-sized
buffers. And working with COM via ctypes is also complex, which is why
comtypes exists.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Albert-Jan Roskam]

From: eryk sun 
Sent: Wednesday, January 10, 2018 3:56 AM
To: tutor@python.org
Cc: Albert-Jan Roskam
Subject: Re: [Tutor] Why does os.path.realpath('test_main.py') give different 
results for unittest than for testing statement in interpreter?
  

On Tue, Jan 9, 2018 at 2:48 PM, Albert-Jan Roskam
 wrote:
>
>> I think that it would be a great enhancement if os.realpath would return the 
>> UNC path if
>> given a mapped drive in Windows, if needed as extended path (prefixed with 
>> "\\?\UNC\").
>> That's something I use all the time, unlike symlinks, in Windows.
>
>pathlib can do this for you, or call os.path._getfinalpathname. 

I tried: 
>>> from os.path import _getfullpathname
>>> _getfullpathname(r"H:")
'h:\\path\\to\\folder'
>>> import os
>>> os.getcwd()
'h:\\path\\to\\folder'

I expected h:\ to be \\server\share\foo. The fact that the current working 
directory was returned was even more unexpected.

>>I recently helped someone that wanted the reverse, to map the resolved
>>UNC path back to a logical drive:

Oh dear. Why would anybody *want* the drive letters? They are only useful 
because (a) they save on keystrokes (b) they bypass the annoying limitation of 
the cd command on windows, ie. it does not work with UNC paths. 
Driveletter-->UNC conversion is useful when e.g. logging file paths. I do 
wonder whether the method used to assign the drive letter matters with the . I 
know net use, pushd, subst. I use 'net use' for more or less permanent drives 
and pushd/popd to get a temporary drive, available letter (cd nuisance).

>We can't assume in general that a user's special folders (e.g.
>Desktop, Documents) are in the default location relative to the
>profile directory. Almost all of them are relocatable. There are shell
>APIs to look up the current locations, such as SHGetKnownFolderPath.
>This can be called with ctypes [1]. For users other than the current
>user, it requires logging on and impersonating the user, which
>requires administrator access.
>
>[1]: https://stackoverflow.com/a/33181421/205580

Interesting code! I have used the following, which uses SHGetFolderPath, ie. 
without 'Known'.
from win32com.shell import shell, shellcon
desktop = shell.SHGetFolderPath(0, shellcon.CSIDL_DESKTOP, 0, 0)

Working with ctypes.wintypes is quite complex!


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[eryk sun]

On Tue, Jan 9, 2018 at 2:48 PM, Albert-Jan Roskam
 wrote:
>
> I think that it would be a great enhancement if os.realpath would return the 
> UNC path if
> given a mapped drive in Windows, if needed as extended path (prefixed with 
> "\\?\UNC\").
> That's something I use all the time, unlike symlinks, in Windows.

pathlib can do this for you, or call os.path._getfinalpathname. I
recently helped someone that wanted the reverse, to map the resolved
UNC path back to a logical drive:

https://bugs.python.org/issue32442

> And I would also welcome a convenience function in the os.path module that 
> does
> expanduser, expandvars, normpath, realpath, and maybe more.

pathlib implements expanduser, but it's still lame on Windows for a
user other than the current user. It assumes all user profiles are in
the same directory instead of directly getting the user's profile
path. Anyway, expanduser is of limited use.

We can't assume in general that a user's special folders (e.g.
Desktop, Documents) are in the default location relative to the
profile directory. Almost all of them are relocatable. There are shell
APIs to look up the current locations, such as SHGetKnownFolderPath.
This can be called with ctypes [1]. For users other than the current
user, it requires logging on and impersonating the user, which
requires administrator access.

[1]: https://stackoverflow.com/a/33181421/205580

You can log a user on without a password by calling LsaLogonUser to
request an MSV1 S4U (service for user) logon. The access token will
only be identification level, unless the script is running as a SYSTEM
service. But identification level is enough to query the location of a
user's known folders. I don't recommended calling LsaLogonUser via
ctypes if you have the option to write an extension module in C/C++,
but I did manage to get it working with ctypes [2]. For example:

>>> from security_lsa import logon_msv1_s4u
>>> from knownfolders import FOLDERID, get_known_folder_path

>>> logon_info = logon_msv1_s4u('Administrator')
>>> get_known_folder_path(FOLDERID.Desktop, logon_info.Token)
'C:\\Users\\Administrator\\Desktop'

[2]: https://stackoverflow.com/a/4322/205580

Also, please don't use expanduser to enable applications to spam the
profile directory with configuration files and directories. Use the
local and roaming application data directories.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Albert-Jan Roskam]

From: Tutor  on behalf of 
Steven D'Aprano 
Sent: Tuesday, January 9, 2018 8:47 AM
To: tutor@python.org
Subject: Re: [Tutor] Why does os.path.realpath('test_main.py') give different 
results for unittest than for testing statement in interpreter?

  
> The Python 3.5 source code for os.path.realpath under Windows looks like 
> this:
>
># realpath is a no-op on systems without islink support
> realpath = abspath

Déjà-vu [1], but I think that it would be a great enhancement if os.realpath 
would return the UNC path if given a mapped drive in Windows, if needed as 
extended path (prefixed with "\\?\UNC\"). That's something I use all the time, 
unlike symlinks, in Windows. 

And I would also welcome a convenience function in the os.path module that does 
expanduser, expandvars, normpath, realpath, and maybe more.

I haven't used pathlib yet. Will os.path development some day be frozen so 
pathlib becomes the standard? It seems to be nicer.

[1] http://python.6.x6.nabble.com/Tutor-getUncPath-mappedDrive-td4652304.html
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Steven D'Aprano]

On Mon, Jan 08, 2018 at 05:08:49PM -0600, boB Stepp wrote:
> Hi Steve,
> 
> On Mon, Jan 8, 2018 at 12:47 AM, Steven D'Aprano  wrote:
> 
> > As I actually explained in my previous email, the one which has
> > disappeared into the aether, ...
> 
> Do you still have the email you sent?  I would like to read it if you
> don't mind sending it to me.

Mystery solved. I've been sending to the wrong email address...

Fixed now, and re-sent to the list.


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[eryk sun]

On Sun, Jan 7, 2018 at 10:51 AM, Albert-Jan Roskam
 wrote:
>
> On Jan 7, 2018 09:08, Steven D'Aprano  wrote:
>>
>> realpath() returns the canonical path of the given filename. It doesn't
>> try to locate some actual existing file.
>
> I always thought that os.path.realpath is the Python equivalent of Linux
> realpath/readlink

In POSIX, os.path.realpath resolves a path that contains symlinks by
calling os.readlink in a loop until all links are resolved.

In Windows, os.path.realpath is an alias for os.path.abspath. This
decision predates NTFS support for junctions (2000) and symbolic links
(Vista). It's simply the case that no one has gotten around to
implementing realpath for Windows.

In Windows, os.path.abspath calls os.path._getfullpathname (WinAPI
GetFullPathName). Other than resolving the working directory, this is
a string transformation to return a canonical, fully-qualified path.
It doesn't touch the filesystem. Note that in the case of classic DOS
devices, the canonical path is a local-device path (i.e. "\\.\"
prefix). For example:

>>> print(os.path.abspath('C:/path/to/nul'))
\\.\nul

> maybe also when it's a hard link - in this case the function actually also 
> does
> something in Windows

A hard link shouldn't be resolved to another path by os.path.realpath.
It's already a real path.

Cross-platform Python 3 code that needs to resolve symbolic links (or
junctions) can use pathlib.Path.resolve(). For example:

>>> pathlib.Path('C:/Documents and Settings').resolve()
WindowsPath('C:/Users')

On Windows this relies on os.path._getfinalpathname (WinAPI
GetFinalPathNameByHandle). This function returns the final opened
filename, so the OS does all of the work to resolve the final path. In
non-strict mode pathlib uses a loop to reduce the path when it can't
be resolved to an existing file. (There are open issues for edge cases
in this code, but it should work as intended with regular files in
accessible directories.)

For those who are curious, WinAPI GetFinalPathNameByHandle is called
with a handle for a File object that's opened via CreateFile. It gets
the NT name of the volume (e.g, "\Device\HarddiskVolume1") and the
filesystem path (e.g. "\Users"). Then it asks the mount-point manager
to map this NT name back to a DOS name such as drive "C:"; or an NTFS
volume mountpoint such as "C:\Mount\VolumeName"; or a "Volume{GUID}"
name. (In the latter case, Python's _getfinalpathname and pathlib have
bugs and need improvement.)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Steven D'Aprano]

On Mon, Jan 08, 2018 at 12:39:07AM +, Alan Gauld via Tutor wrote:

> To be fair realpath() does do a tiny bit of magic in that it
> checks if the file (or any part of the path) is a link (aka
> shortcut) and tries to give you the actual path to the
> original file - ie it follows the link.

As I actually explained in my previous email, the one which has 
disappeared into the aether, realpath resolves symbolic links and 
replaces them by the real path they represent.

But that's only on Unix/Linux/Mac, on Windows it does not.

Symbolic links, however, are *not* Windows shortcuts. Windows shortcuts 
are just a special sort of file, one which the Windows file explorer 
treats as an alias to another file or folder.

Symbolic links on Unix are a feature of the file system itself, not a 
kind of file.

What's the difference?

With a shortcut, if you tell Python or some other program to open a 
shortcut, it will treat it as an ordinary file (because that's what it 
is!) and read from the shortcut file itself. Only programs like Windows 
Explorer which have been specially programmed to follow shortcuts will 
do so.

But with a symbolic link, programs don't need to know anything about 
them. If you have a link:

/A/link ---> /B/C/file

then *any* program will treat opening /A/link as if it were /B/C/file.

In other words:

- under Windows, a shortcut is a regular file, unless the programmer
  takes special action to treat it as an alias;

- under Unix, a symbolic link is an alias, unless the programmer 
  takes special action to treat it as a symbolic link.


(Aside: starting with Windows Vista, NTFS also supports symbolic links, 
like Unix. But few Windows users know this, or use them.)




-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Alan Gauld via Tutor]

On 08/01/18 00:17, boB Stepp wrote:

>> The os.path module is mostly a string manipulation toolkit.
> 
> This is the critical piece that was eluding me.  I assumed that the
> actual file system was being queried, checked and expanded as needed.

To be fair realpath() does do a tiny bit of magic in that it
checks if the file (or any part of the path) is a link (aka
shortcut) and tries to give you the actual path to the
original file - ie it follows the link. I'm not sure how
effective it is, I've only played with it but never used
it in earnest.

But in general I tend not to rely on os.path doing anything
remotely clever.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[boB Stepp]

On Sun, Jan 7, 2018 at 8:32 AM, Alan Gauld via Tutor  wrote:
> On 07/01/18 09:07, boB Stepp wrote:
>> clarify this?  What is the methodology that os.path.realpath(path) is
>> actually following to yield a particular path name?  And why does it
>> not care if path refers to a real file or not?
>
> The os.path module is mostly a string manipulation toolkit.

This is the critical piece that was eluding me.  I assumed that the
actual file system was being queried, checked and expanded as needed.

> It checks that a string looks like a path and it de/constructs
> paths from strings. But it does not pay muchy(any?) attention
> to the filesystem. It doesn't care if the paths it deals
> with are real paths or not, its purely manipulating the
> strings according to the syntax rules for paths.

I was assuming it was computing the actual path to the file given as
its argument.  Steve's comments make much more sense now as well as
the "canonical" aspect.

>> class TestOpenFileStrIntsToList(unittest.TestCase):
>> """Tests for the function open_file_str_ints_to_list."""
>>
>> def test_open_file_str_ints_to_list(self):
>> """Ensure expected list of string integers is returned."""
>>
>> path = os.path.dirname(
>> os.path.realpath(__file__)) + '/test_data.dat'
>> print('realpath =', os.path.realpath(__file__))
>
> Yes, thats it although you could test using isfile() or
> better still use a try/except around the open.
>
> Also you should strictly use os.path.join() to create the path

Duly noted on os.path.join().  I have not gotten to thinking about the
UI yet, and don't mind if a file not found error is generated here in
the testing code as it will let me know I've screwed something up.
This is strictly a play project to record my manual games of
solitaire.  I doubt it will be useful to anyone else.  Once I get to
coding the UI I might wrap this in a try/except if I feel I would gain
any benefit from handling the exception.

>> BTW, I am having trouble coming up with a good name for a string that
>> looks like an integer that I plan on using int() later to convert the
>> string to an integer.  "String integer" is my best thought at this
>> time.
>
> What's it for? Where did it come from?
> type based names are rarely good, better to describe why/where.
>
> input_Id, read_Id, stored_Id etc
>
> Or substitute Ref, or Count or whatever for Id.
>
> Then the real int var name becomes the second part:
>
> Id = int(input_Id)
>
> or
>
> count = int(stored_count)

The above has just now led me to a Homer Simpson "Doh!" moment.  When
I open the file and covert it to a list of "string" integers I
currently have:

with open(path) as infile:
   string_integers = [line.strip() for line in infile]

and then run the list through another function to get an integer list.

Why on earth didn't I just write:

with open(path) as infile:
   total_scores = [int(line.strip()) for line in infile]

which directly creates what I want directly?  [Please don't answer
this question!  ~(:>)) ]


-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[boB Stepp]

On Sun, Jan 7, 2018 at 4:51 AM, Albert-Jan Roskam
 wrote:
>
> On Jan 7, 2018 09:08, Steven D'Aprano  wrote:

>> realpath() returns the canonical path of the given filename. It doesn't
>> try to locate some actual existing file.
>
> I always thought that os.path.realpath is the Python equivalent of Linux 
> realpath/readlink (http://man7.org/linux/man-pages/man3/realpath.3.html). And 
> that, thus, there's only a difference between input and output when the input 
> is a symlink (and maybe also when it's a hard link - in this case the 
> function actually also does something in Windows). But then, I don't really 
> know the meaning of the word "canonical", to tell you the truth (maybe #4 in 
> http://www.dictionary.com/browse/canonical)

My hangup was mostly that I was assuming that an actually existing
path to a real file was implied.  Apparently that is not the case.  So
Alan's explanation has cleared that up for me (I think!).

I agree with you that the 4th definition you cite:

"4. Mathematics. (of an equation, coordinate, etc.) in simplest or
standard form."

is the most sensible definition in this context with Steve's caveats.

-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Alan Gauld via Tutor]

On 07/01/18 09:07, boB Stepp wrote:
> clarify this?  What is the methodology that os.path.realpath(path) is
> actually following to yield a particular path name?  And why does it
> not care if path refers to a real file or not?

The os.path module is mostly a string manipulation toolkit.

It checks that a string looks like a path and it de/constructs
paths from strings. But it does not pay muchy(any?) attention
to the filesystem. It doesn't care if the paths it deals
with are real paths or not, its purely manipulating the
strings according to the syntax rules for paths.

The exceptions are the test functions like isfile(), isdir() etc.

> What I *really* want to do is locate and open a file "test_data.dat"
> that is in the same directory as the file with the tests,

For that you need to do some work with listdir or glob.
You might need to do a os.getcwd() but I suspect it will
return the folder where you started te program rather than
the folder the file lives in. For that you may need to
use the __file__ attribute of the test module. So
something like:

dir = os.path.dirname(__file__)  # assuming this is in the test module
fn = os.path.join(dir,fname)
if os.path.isfile(fn)   # check the file actually exists
   f = open(fn)


> class TestOpenFileStrIntsToList(unittest.TestCase):
> """Tests for the function open_file_str_ints_to_list."""
> 
> def test_open_file_str_ints_to_list(self):
> """Ensure expected list of string integers is returned."""
> 
> path = os.path.dirname(
> os.path.realpath(__file__)) + '/test_data.dat'
> print('realpath =', os.path.realpath(__file__))

Yes, thats it although you could test using isfile() or
better still use a try/except around the open.

Also you should strictly use os.path.join() to create the path

> BTW, I am having trouble coming up with a good name for a string that
> looks like an integer that I plan on using int() later to convert the
> string to an integer.  "String integer" is my best thought at this
> time.

What's it for? Where did it come from?
type based names are rarely good, better to describe why/where.

input_Id, read_Id, stored_Id etc

Or substitute Ref, or Count or whatever for Id.

Then the real int var name becomes the second part:

Id = int(input_Id)

or

count = int(stored_count)

> Is this way of determining the directory which
> test_main.py is running from bullet proof?

If its really in the same folder then I believe
so - or as bullet proof as it gets. Maybe if you
create a shortcut to the real folder etc you might
need to revert to realpath.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Albert-Jan Roskam]

On Jan 7, 2018 09:08, Steven D'Aprano  wrote:
>
> On Sun, Jan 07, 2018 at 12:49:59AM -0600, boB Stepp wrote:
>
> > Win7, Python 3.6.2
> >
> > If I run a unit test with the following embedded:
> >
> > print('realpath =', os.path.realpath('test_main.py'))
> >
> > I get the following in my test output (Only relevant line is shown):
> >
> > Ensure expected list of string integers is returned. ...
> > realpath = c:\Projects\solitaire_scorekeeper\test_main.py
>
>
> realpath() returns the canonical path of the given filename. It doesn't
> try to locate some actual existing file.

I always thought that os.path.realpath is the Python equivalent of Linux 
realpath/readlink (http://man7.org/linux/man-pages/man3/realpath.3.html). And 
that, thus, there's only a difference between input and output when the input 
is a symlink (and maybe also when it's a hard link - in this case the function 
actually also does something in Windows). But then, I don't really know the 
meaning of the word "canonical", to tell you the truth (maybe #4 in 
http://www.dictionary.com/browse/canonical)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[boB Stepp]

On Sun, Jan 7, 2018 at 2:05 AM, Steven D'Aprano  wrote:
> On Sun, Jan 07, 2018 at 12:49:59AM -0600, boB Stepp wrote:
>
>> Win7, Python 3.6.2
>>
>> If I run a unit test with the following embedded:
>>
>> print('realpath =', os.path.realpath('test_main.py'))
>>
>> I get the following in my test output (Only relevant line is shown):
>>
>> Ensure expected list of string integers is returned. ...
>> realpath = c:\Projects\solitaire_scorekeeper\test_main.py
>
>
> realpath() returns the canonical path of the given filename. It doesn't
> try to locate some actual existing file. The fact that there is a file
> called "test_main.py" located here:
>
>> In actuality "test_main.py" is located at
>> "c:\Projects\solitaire_scorekeeper\tests\test_main.py"
>
> is irrelevent. You asked for the "real" (canonical) path name to the
> relative pathname "test_main.py". That means in the current directory,
> which apparently is C:\Projects\solitaire_scorekeeper. Hence you get the
> result C:\Projects\solitaire_scorekeeper\test_main.py even though the
> "real file" is one directory further in.

After some searching I have yet to locate a definition of "canonical
path" that makes sense to me.  The dictionary definition of canonical
does not seem to be very helpful in understanding this.  Can you
clarify this?  What is the methodology that os.path.realpath(path) is
actually following to yield a particular path name?  And why does it
not care if path refers to a real file or not?

What I *really* want to do is locate and open a file "test_data.dat"
that is in the same directory as the file with the tests,
"tests/test_main.py".  I asked a related question a while back but I
now suspect I did not fully get everything.  I have recently been
experimenting with the os and os.path modules and thought I had found
the magic bullet with os.path.realpath().  Anyway, based on that
earlier related question and your answer tonight I have modified my
test class to the following:

class TestOpenFileStrIntsToList(unittest.TestCase):
"""Tests for the function open_file_str_ints_to_list."""

def test_open_file_str_ints_to_list(self):
"""Ensure expected list of string integers is returned."""

path = os.path.dirname(
os.path.realpath(__file__)) + '/test_data.dat'
print('realpath =', os.path.realpath(__file__))
expected_list = ['-3', '-2', '-1', '0', '1', '2', '3']
self.assertEqual(open_file_str_ints_to_list(path), expected_list)

The test_data file is just the expected_list values, one value per
line in the file.  The function being tested is:

def open_file_str_ints_to_list(path):
"""Given a path to the file, open this file containing one string integer
per line, and convert it to a list of string integers, which is then
returned by this function."""

with open(path) as infile:
string_integers = [line.strip() for line in infile]

return string_integers

BTW, I am having trouble coming up with a good name for a string that
looks like an integer that I plan on using int() later to convert the
string to an integer.  "String integer" is my best thought at this
time.

Anyway, the print() statement in the test case is giving me the
correct result.  Is this way of determining the directory which
test_main.py is running from bullet proof?


-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Steven D'Aprano]

On Sun, Jan 07, 2018 at 12:49:59AM -0600, boB Stepp wrote:

> Win7, Python 3.6.2
> 
> If I run a unit test with the following embedded:
> 
> print('realpath =', os.path.realpath('test_main.py'))
> 
> I get the following in my test output (Only relevant line is shown):
> 
> Ensure expected list of string integers is returned. ... 
> realpath = c:\Projects\solitaire_scorekeeper\test_main.py


realpath() returns the canonical path of the given filename. It doesn't 
try to locate some actual existing file. The fact that there is a file 
called "test_main.py" located here:

> In actuality "test_main.py" is located at
> "c:\Projects\solitaire_scorekeeper\tests\test_main.py"

is irrelevent. You asked for the "real" (canonical) path name to the 
relative pathname "test_main.py". That means in the current directory, 
which apparently is C:\Projects\solitaire_scorekeeper. Hence you get the 
result C:\Projects\solitaire_scorekeeper\test_main.py even though the 
"real file" is one directory further in.


> If I open the interpreter from the directory
> "c:\Projects\solitaire_scorekeeper\tests\" I get what I expect:

Indeed, because now the current directory is different.

I think you may be confused because you think realpath() has something 
to do with actual existing files. It doesn't. Remember that realpath() 
isn't just called on existing files, it may be called on arbitrary file 
names which don't exist. Its more like os.path.abspath() except that if 
the path contains symbolic links (not likely on Windows, but possible) 
they will be resolved to their real path.


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor