Re: [Tutor] What is the best way for a program suite to know where it is installed?

[Cameron Simpson]

On 22Oct2018 19:26, boB Stepp  wrote:

On Mon, Oct 22, 2018 at 9:47 AM Mats Wichmann  wrote:

As you've concluded by now, this isn't a completely trivial exercise,
and you'll possibly get asked the question "why do you think you need to
know that?".


Actually no one has yet asked me this question, and you have me
intrigued.  Importing the various program modules/module contents is
no issue.  Where I believe I need to know the paths to things are to
get to data folders, config files, and occasionally utility programs
that I have written that are on my hard drive, but not copied to my
current program suite.


Data folders inbuilt to your app? (I'm going to call it an "app" purely 
for convenience.) Not sure there's a universally good answer for this.  
You can include data files in your package I think (haven't tried this, 
as far as PyPI/pip goes), which means you could locate them from your 
source code.


Happy to hear more on this from others.

Config files:

I'm coming to be of the opinion, based on a couple of things I'm 
writiing at the moment, that apps should (a) come with a builtin default 
config (possibly computed form the environment the find themselves 
executed in) and that (b) that config should be expressed in a class 
instance which can transcribe itself to a file in the right format.


Why?

It solves the "where is it" problem: it is in the code, as some function 
which constructs the app's config instance in the absense of a config 
file.


It solves the "how does the user get their first config file for later 
hacking/customisation"? If the config file is not in the well known 
place (~/.your-apprc or ~/.config/something, etc) then having made your 
default config you can transcribe it to the default place. Now the user 
has a file they can modify.


It also solves the "incomplete config file" problem: if you can 
construct a default config, you can usually handle an incomplete config 
as "construct the default, then alter based on the contents of the 
config file". Which lets the user keep a tiny config file modifying only 
the stuff which needs tweaking.


Utilities:

I my opinion, unless they shift with you app it is the end user's job to 
have these in the execution path ($PATH on UNIX). If you app/package 
provides them you need a bin directory, with the same issues as your 
data files.


Cheers,
Cameron Simpson 
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What is the best way for a program suite to know where it is installed?

[eryk sun]

On 10/22/18, boB Stepp  wrote:
>
> Importing the various program modules/module contents is
> no issue.  Where I believe I need to know the paths to things are to
> get to data folders, config files, and occasionally utility programs
> that I have written that are on my hard drive, but not copied to my
> current program suite.

It's common to use __file__ for a portable application, except you
should use sys.executable if it's a frozen executable, i.e. if
sys.frozen exists.

For an installed application, limit this approach to the application's
immutable resources. Don't use the installation directory to store
modifiable data. You may not have write access (e.g. a system
installation and the user lacks root/admin access, or it's an
immutable directory or read-only disk mount).
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

[William Ray Wing via Tutor]

> On Oct 22, 2018, at 8:30 PM, boB Stepp  wrote:
> 
> On Mon, Oct 22, 2018 at 11:57 AM Mats Wichmann  wrote:
>> 
>> On 10/22/18 8:24 AM, boB Stepp wrote:
>>> Forwarding to the Tutor list.  Herr Maier offers a good idea that
>>> would take away much of a remaining issue -- the name "Temporary".  I
>>> need to look into the functools library to see what "partial" does.
>> 
>> 
>> if you really don't care what the file is called because you will
>> maintain a map which leads you to the filename, you might as well use a
>> uuid.
> 
> Wouldn't I have to write a check to ensure such a named file (However
> unlikely that would be.) did not exist before creating it?  And if
> yes, would not that get into the same potential security issue that
> cause tempfile.mktemp() to be deprecated?
> 

The whole point of UUIDs is to make the probability of a UUID collision so 
infinitesimally small as to make that hypothetical collision simply not worth 
worrying about, even when they are created on different systems.  Since a big 
chunk of a UUID is a high precision time stamp, any UUIDs created on a single 
system are pretty much guaranteed to be unique.

Bill 

> -- 
> boB
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

[boB Stepp]

On Mon, Oct 22, 2018 at 11:57 AM Mats Wichmann  wrote:
>
> On 10/22/18 8:24 AM, boB Stepp wrote:
> > Forwarding to the Tutor list.  Herr Maier offers a good idea that
> > would take away much of a remaining issue -- the name "Temporary".  I
> > need to look into the functools library to see what "partial" does.
>
>
> if you really don't care what the file is called because you will
> maintain a map which leads you to the filename, you might as well use a
> uuid.

Wouldn't I have to write a check to ensure such a named file (However
unlikely that would be.) did not exist before creating it?  And if
yes, would not that get into the same potential security issue that
cause tempfile.mktemp() to be deprecated?


-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What is the best way for a program suite to know where it is installed?

[boB Stepp]

On Mon, Oct 22, 2018 at 9:47 AM Mats Wichmann  wrote:
>
> On 10/20/18 9:00 PM, boB Stepp wrote:

> > So far the best method I've come up with is to make use of "__file__"
> > for the initiating program file.  But from past discussions I am not
> > certain this is the *best* way.  Is the *best* way going to get me
> > into best techniques for installing and distributing programs?  If so,
> > this strikes me as a huge mess to dive into!
> >
> > TIA!
> >
>
> As you've concluded by now, this isn't a completely trivial exercise,
> and you'll possibly get asked the question "why do you think you need to
> know that?".

Actually no one has yet asked me this question, and you have me
intrigued.  Importing the various program modules/module contents is
no issue.  Where I believe I need to know the paths to things are to
get to data folders, config files, and occasionally utility programs
that I have written that are on my hard drive, but not copied to my
current program suite.  Unless there is some built-into-Python-easy
way to find such things, I do not see any other alternative than
determining the path to the desired files.  Is there a better way?

> Probably though this is considered one of the more reliable ways:
>
> import os
>
> if __name__ == '__main__':
> script_path = os.path.dirname(os.path.realpath(__file__))
> print("Main module at", script_path)

Despite the past thread I mentioned in this thread's originating post,
I have switched to using pathlib.Path for doing this sort of thing.
Something I read on the main list in the past year I found convincing,
though I now don't remember now why I was so convinced.


-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] organisation of folder structure of Python'sinstallation directory

[Mats Wichmann]

On 10/22/18 9:58 AM, Malhar Lathkar wrote:
> Is it possible to install just python runtime environment only similar to
> JRE?

You could probably arrange it if you build your own, but there doesn't
seem much point.  I'm looking at a Windows system here with several
Python versions installed; for a recent Python, it takes 126,500kb of
which the two pieces related to compilation, libs and Include, take
2500kb - less than 2% of the total.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] organisation of folder structure of Python'sinstallation directory

[Malhar Lathkar]

Is it possible to install just python runtime environment only similar to
JRE?

On Sun, 21 Oct 2018, 20:12 Mats Wichmann,  wrote:

> [image: Boxbe]  Mats Wichmann (
> m...@wichmann.us) is not on your Guest List
> 
> | Approve sender
> 
> | Approve domain
> 
> On 10/21/18 12:52 AM, Malhar Lathkar wrote:
> > Hi all.
> >
> > I am trying to understand how Python's installation directory is
> organized.
> > I find lib as well as libs folder. what's the difference? I couldn't find
> > exact explanation of purpose of various other sub folders like include,
> > scripts, share etc. Can you help me understand this? Please share
> > documentation link if any.
> >
> > thanks.
> >
>
> on a Windows installation, Lib is the python code for the standard
> library, while DLLs contains any backing binary objects for the standard
> library; meanwhile libs is what is necessary to to link with the python
> libraries themselves, as when developing a C-language extension. include
> is the header files used for compiling such extensions - so these two
> are not part of the Python runtime, but have this special purpose.
> (note sometimes when you "python -m pip install" it may try to compile
> some code, so this is not always just for you yourself to write
> extensions).
>
> The rest should be relatively straightforward.
>
> The layout can be quite different on other systems. A /usr/share
> directory on Linux is used here for shareable, non-binary pieces like
> documentation and license files. The python layout on Linux is normally
> packaged along the lines of the Filesystem Hierarchy Standard:
>
> https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

[Mats Wichmann]

On 10/22/18 8:24 AM, boB Stepp wrote:
> Forwarding to the Tutor list.  Herr Maier offers a good idea that
> would take away much of a remaining issue -- the name "Temporary".  I
> need to look into the functools library to see what "partial" does.


if you really don't care what the file is called because you will
maintain a map which leads you to the filename, you might as well use a
uuid.

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] What is the best way for a program suite to know where it is installed?

[Mats Wichmann]

On 10/20/18 9:00 PM, boB Stepp wrote:
> I was just re-reading the entire thread at
> https://www.mail-archive.com/tutor@python.org/msg77864.html
> And I have asked similar questions previous to that thread.  I still
> do not have a satisfactory answer for the subject line's question that
> both makes sense to me and seems to be good programming practice.  And
> that works for any OS that the program suite is installed under.
> 
> A constantly recurring situation I am having is a programming project
> with a nested directory structure.  Usually I will have "tests" and
> "data" folders nested under the top level project folder.  There may
> be others as well depending on the complexity of the project I am
> attempting.  As far as I can tell, I cannot make any assumptions about
> what the current working directory is when a user starts the program.
> So how can one of my programs *know* what the absolute path is to,
> say, the top level of the program suite?  If I could always reliably
> determine this by my program's code, every other location could be
> determined from the known nested structure of the program suite.  [But
> I have a dim recollection that either Ben or Cameron thought relying
> on the "known nested structure" is brittle at best.]
> 
> So far the best method I've come up with is to make use of "__file__"
> for the initiating program file.  But from past discussions I am not
> certain this is the *best* way.  Is the *best* way going to get me
> into best techniques for installing and distributing programs?  If so,
> this strikes me as a huge mess to dive into!
> 
> TIA!
> 

As you've concluded by now, this isn't a completely trivial exercise,
and you'll possibly get asked the question "why do you think you need to
know that?".

Probably though this is considered one of the more reliable ways:

import os

if __name__ == '__main__':
script_path = os.path.dirname(os.path.realpath(__file__))
print("Main module at", script_path)




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Fwd: Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

[boB Stepp]

Forwarding to the Tutor list.  Herr Maier offers a good idea that
would take away much of a remaining issue -- the name "Temporary".  I
need to look into the functools library to see what "partial" does.

-- Forwarded message -
From: Wolfgang Maier 
Date: Mon, Oct 22, 2018 at 5:25 AM
Subject: Re: Can tempfile.NamedTemporaryFile(delete=False) be used to
create *permanent* uniquely named files?
To: boB Stepp 


On 21.10.18 08:13, boB Stepp wrote:
> Use case:  I want to allow a user of my Solitaire Scorekeeper program
> to be able to give any name he wants to each game of solitaire he
> wishes to record.  My thought for permanent storage of each game's
> parameters is to use a dictionary to map the user-chosen game names to
> a unique json filename.  This way I hope no shenanigans can occur with
> weird characters/non-printing characters.
>
> My initial thought was to just have a sequence of game names with
> incrementing numerical suffixes:  game_0, game_1, ... , game_n.  But
> this would require the program to keep track of what the next
> available numerical suffix is.  Additionally, if a user chooses to
> delete a game, then there would be a gap in the numerical sequence of
> the game filenames.  I find such a gap aesthetically displeasing and
> would probably go to additional efforts to reuse such deleted
> filenames, so there would be no such "gaps".
>
> So I am now wondering if using
> tempfile.NamedTemporaryFile(delete=False) would solve this problem
> nicely?  As I am not very familiar with this library, are there any
> unforeseen issues I should be made aware of?  Would this work equally
> well on all operating systems?
>
> TIA!
>

This sounds like a good though surprising use case. The only odd thing
about this is the misleading name then of the function, plus there is
the (vague) possibility that the stdlib module might evolve and no
longer support this undocumented (given that the first sentence in the
module description reads: "This module creates temporary files and
directories.") use case.
I would probably address these issues and the handling of the dir
parameter via a partial function like this:

from functools import partial

SavedGameFile = partial(
 tempfile.NamedTemporaryFile, dir=my_saved_games_dir
 ) # if you like also set the suffix here
SavedGameFile.__doc__ = """\
Create and return a saved game file, ...
"""

This way you have a good name for the function, and you can redefine the
function, if you ever want/need to move away from NamedTemporaryFile,
without having to rewrite every function call.

Wolfgang


-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Can tempfile.NamedTemporaryFile(delete=False) be used to create *permanent* uniquely named files?

[Wolfgang Maier]

On 21.10.18 08:13, boB Stepp wrote:

Use case:  I want to allow a user of my Solitaire Scorekeeper program
to be able to give any name he wants to each game of solitaire he
wishes to record.  My thought for permanent storage of each game's
parameters is to use a dictionary to map the user-chosen game names to
a unique json filename.  This way I hope no shenanigans can occur with
weird characters/non-printing characters.

My initial thought was to just have a sequence of game names with
incrementing numerical suffixes:  game_0, game_1, ... , game_n.  But
this would require the program to keep track of what the next
available numerical suffix is.  Additionally, if a user chooses to
delete a game, then there would be a gap in the numerical sequence of
the game filenames.  I find such a gap aesthetically displeasing and
would probably go to additional efforts to reuse such deleted
filenames, so there would be no such "gaps".

So I am now wondering if using
tempfile.NamedTemporaryFile(delete=False) would solve this problem
nicely?  As I am not very familiar with this library, are there any
unforeseen issues I should be made aware of?  Would this work equally
well on all operating systems?

TIA!



This sounds like a good though surprising use case. The only odd thing 
about this is the misleading name then of the function, plus there is 
the (vague) possibility that the stdlib module might evolve and no 
longer support this undocumented (given that the first sentence in the 
module description reads: "This module creates temporary files and 
directories.") use case.
I would probably address these issues and the handling of the dir 
parameter via a partial function like this:


from functools import partial

SavedGameFile = partial(
tempfile.NamedTemporaryFile, dir=my_saved_games_dir
) # if you like also set the suffix here
SavedGameFile.__doc__ = """\
Create and return a saved game file, ...
"""

This way you have a good name for the function, and you can redefine the 
function, if you ever want/need to move away from NamedTemporaryFile, 
without having to rewrite every function call.


Wolfgang

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor