[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Inada Naoki]

Inada Naoki  added the comment:

+1 to Eryk.

> Hence my newbie questions are:

1) What problem are you trying to solve by this "unlinking trick"?

Same to TempoaryFile in Unix.

2) Do we need to have a separate issue raised for this problem?

I don't think so. We didn't unlink because wi didn't have separate 
implementation for TemporaryFile and NamedTemporaryFile.
When we have two implementations for them, it is straightforward and natural to 
use unlink trick.

3) Is this "unlinking trick" using some documented features of Windows? (will 
it not stop working after some Windows patch)?

I am not sure. But it is "POSIX function" in windows. I believe MS won't break 
compatibility.

4) Will we need to create separate unit tests for this issue?

We already have tests for TemporaryFile.

5) Do we also need to reach a consensus on implementing of this "unlinking 
trick"?

If anyone against it. But I think Eryk's proposal is the most reasonable.

--
nosy: +methane

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Evgeny  added the comment:

Eryk, I agree, that implementing TemporaryFile() in Windows  goes hand in hand 
with the decision to stop using O_TEMPORARY in NamedTemporaryFile()

The only thing I want to point out is that your suggestion also includes this 
"unlinking trick" (sorry, may be there is a better description for this), which 
seems to be separate/extra to the usage of O_TEMPORARY.  

Hence my newbie questions are:

1) What problem are you trying to solve by this "unlinking trick"?

2) Do we need to have a separate issue raised for this problem?

3) Is this "unlinking trick" using some documented features of Windows? (will 
it not stop working after some Windows patch)?

4) Will we need to create separate unit tests for this issue?

5) Do we also need to reach a consensus on implementing of this "unlinking 
trick"?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> you are proposing some trick with first unlinking the file and 
> then employing it as a temporary file using previously known fd.

It doesn't need to unlink the file to make it anonymous. I included that to 
highlight that it's possible. The details can be discussed and hashed out in 
the PR.

I don't think implementing TemporaryFile() in Windows is separate from this 
issue. It goes hand in hand with the decision to stop using O_TEMPORARY in 
NamedTemporaryFile().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Evgeny  added the comment:

> Eryk Sun  added the comment:
> Just implement a separate function for TemporaryFile() instead of aliasing it 
> to NamedTemporaryFile(). See msg390814.

Eryk, forgive my ignorance, but aren't in your msg390814 you are proposing yet 
another enhancement (separate from the issue14243, discussed here), in this 
case for TemporaryFile in Windows systems?

I may be mistaken, but I see that you are proposing some trick with first 
unlinking the file and then employing it as a temporary file using previously 
known fd. This "trick" is not present in the current code and does not seem to 
address issue14243.

Or am I talking total nonsense?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Paul Moore]

Paul Moore  added the comment:

Eryk, thank you for clarifying. I apologise - I got bogged down somewhere in 
the middle of the discussion on reimplementing bits of the CRT (your posts are 
so information-dense that my usual habit of skimming breaks down - that's not a 
complaint, though!)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> Does it mean, that your suggestion to leave the O_TEMPORARY for 
> TemporaryFile means, that NamedTemporaryFile needs to have a 
> mechanism to know whether it was called as a TemporaryFile

Just implement a separate function for TemporaryFile() instead of aliasing it 
to NamedTemporaryFile(). See msg390814.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Evgeny  added the comment:

>Paul Moore  added the comment:

>Evgeny, would you be willing to update your PR (including adding the docs 
>change, and tests to catch as many edge cases as you can think up) to match 
>this behaviour?

Paul, thank you for moving this forward. I will give it a try.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[John Florian]

Change by John Florian :


--
nosy:  -John Florian

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Evgeny  added the comment:

>Eryk Sun  added the comment:

>I replied twice that I thought using the CM exit instead of O_TEMPORARY is 
>okay for NamedTemporaryFile() -- but only if a separate implementation of 
>TemporaryFile() that uses O_TEMPORARY is added at the same time. I want 
>guaranteed cleanup for TemporaryFile() since it's not intended to be reopened.

At the moment, the TemporaryFile directly reuses NamedTemporaryFile for 
none-posix or cygwin systems.

https://github.com/python/cpython/blob/a92d7387632de1fc64de51f22f6191acd0c6f5c0/Lib/tempfile.py#L552

Does it mean, that your suggestion to leave the O_TEMPORARY for TemporaryFile 
means, that NamedTemporaryFile needs to have a mechanism to know whether it was 
called as a TemporaryFile and then to have a different functionality in this 
case relative to the situation it would be called directly?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> deleting on CM exit when used as a CM, and no change in behaviour 
> otherwise (me, Zachary, Ethan, Jason and Steve). Steve also wants
> O_TEMPORARY to be removed, which doesn't seem controversial among
> this group of people.

Removing O_TEMPORARY is not an afterthought here. It is the core of this issue. 
The O_TEMPORARY flag MUST NOT be used if the goal is to make 
NamedTemporaryFile() "particularly useful on Windows". A file that's opened 
with DELETE access cannot be reopened in most cases, because most opens do not 
share delete access, but it also can't be closed to allow it to be reopened 
because the OS will delete it.

I replied twice that I thought using the CM exit instead of O_TEMPORARY is okay 
for NamedTemporaryFile() -- but only if a separate implementation of 
TemporaryFile() that uses O_TEMPORARY is added at the same time. I want 
guaranteed cleanup for TemporaryFile() since it's not intended to be reopened.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Paul Moore]

Paul Moore  added the comment:

Looking at the various comments, I think we have 5 votes for deleting on CM 
exit when used as a CM, and no change in behaviour otherwise (me, Zachary, 
Ethan, Jason and Steve). Steve also wants O_TEMPORARY to be removed, which 
doesn't seem controversial among this group of people.

Eryk has argued for a delete_on_close flag that would need to be explicitly set 
to False, retaining the use of O_TEMPORARY in the default case, but there 
doesn't seem to be a lot of support for that.

If I've misrepresented anyone's view, please speak up!

I didn't look back at the stuff from 2013 and earlier, I'll admit.

I do think this needs care to implement (and document!) correctly. For example, 
consider the following case:

ntf = NamedTemporaryFile()
# Do some stuff (1)
with ntf:
# Do some stuff (2)
# Do some followup stuff

I assume we'd want a close in (1) to delete the file, but a close in (2) to 
leave it open until the CM exit.

Evgeny, would you be willing to update your PR (including adding the docs 
change, and tests to catch as many edge cases as you can think up) to match 
this behaviour?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Zachary Ware]

Zachary Ware  added the comment:

> if NamedTemporaryFile is used as a context manager, the file is closed *on 
> context manager exit* and *not* when the file is closed.

+1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Paul Moore]

Paul Moore  added the comment:

To be explicit, I'm +1 on breaking backward compatibility in the minor form 
described by Ethan: if NamedTemporaryFile is used as a context manager, the 
file is closed *on context manager exit* and *not* when the file is closed.

Breaking compatibility is allowed in minor versions (3.11 at this point, as 
this won't make it in before 3.10 feature freeze). So this is an acceptable 
change.

I don't personally think this needs a transition period or deprecation, and in 
particular I don't think that Jason's "gradual transition" proposal is 
necessary.

I'd be sad if we ended up with an over-complicated solution for no better 
reason than an excess of caution over a backward compatibility issue that we're 
not sure will impact anyone. Do we have any actual examples of code that needs 
the current CM behaviour (as opposed to a general concern that someone might be 
using the functionality)?

(My original over-complicated proposal was based on a mistaken belief that it 
had already been established that backward incompatibility was absolutely not 
allowed. But I never subscribed to that view myself.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Evgeny  added the comment:

On Mon, Apr 12, 2021 at 12:51 AM Jason R. Coombs  wrote:

> Jason R. Coombs  added the comment:

> At least I and Ethan and Martin have expressed a desire for the default, 
> preferred usage work well in a portable environment. Requiring 
> `delete_on_close=False` violates that expectation.

Separately to my previous message. If the only issue with my PR is that default 
solution is not portable, then this can be simply changed within current PR by 
setting default of delete_on_close to fe equal to False.

As I mentioned, I was assuming I cannot break backwards compatibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Evgeny  added the comment:

Dear all, thank you very much for the discussion, I will just try to summarize 
the results of it.


In my PR I used solution, proposed by Eryk. My solution involves introduction 
of the extra flag delete_on_close and making sure, that new situation is fully 
backwards compatible, because I did not feel I have an authority to propose 
anything which would break backwards compatibility (as Python 4.0 I think is 
not on the horizon yet)


As I see from the discussion, the following decisions need to be taken:

WHICH FLAG TO USE

Eryk was proposing that it shall be delete_on_close  (which I have implemented)

Jason thinks, that we shall introduce flag delete_when, however when I read his 
proposal, the functionality which he proposes is not that different from what I 
implemented with  delete_on_close.

Ethan however thinks, that no extra flags are necessary at all



USAGE OF O_TEMPORARY ON WINDOWS

Ethan, Steve thinks, it is not needed

Eryk prefers to provide a way to omit O_TEMPORARY, but still use it by default, 
when it's omitted



CHANGING OF THE CURRENT BEHAVIOUR / BREAKING BACKWARDS COMPATIBILITY
Ethan thinks, that we shall slightly change backwards compatibility in a way 
that if CM is used, than file is deleted on CM exit and not on close as now

Jason thinks that backwards compatibility shall be changed gradually over 
several releases with the advanced depreciation warning 

Question: does everybody definitely agree then, that backwards compatibility 
shall definitely be altered, whether immediately or gradually?



Any other decision to be taken which I missed?


CONCLUSION: 
There seems to be still disagreements and I don't really know how to move this 
forward as  I am not sure I understand the decision making mechanism in the 
Python community (especially when it comes to breaking backwards 
compatibility). Any thoughts on this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> So we stop passing the O_TEMPORARY flag. If __enter__() is called, 
> close() closes the file but doesn't delete anything, and 
> __exit__() closes the file (if open) and deletes it (even if it 
> wasn't open). If there is no __enter__(), close() also deletes the 
> file.

This behavior change is fine if O_TEMPORARY isn't used. I wasn't disagreeing 
with Ethan. I was providing a summary of a common use case that conflicts with 
using O_TEMPORARY to make it clear that this flag has to be omitted if we're 
not implementing something like a delete_on_close boolean option.

Most of my last comment, however, was dedicated to implementing TemporaryFile() 
if this change is applied, instead of leaving it as an alias for 
NamedTemporaryFile(). I can't imagine not wanting the guaranteed cleanup 
semantics of O_TEMPORARY in the case of an anonymous temporary file that 
doesn't need to be reopened. I also want O_SHORT_LIVED. This opens the file 
with the attribute FILE_ATTRIBUTE_TEMPORARY [1], which tells the cache manager 
to try to keep the file contents in memory instead of flushing data to disk.

---

[1] 
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew#caching_behavior

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Ethan Furman]

Ethan Furman  added the comment:

Hey, you agree with me now, so it's not noise.  ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Paul Moore]

Paul Moore  added the comment:

Sorry - I'm maybe making an unwarranted assumption. If simply removing "delete 
on close" behaviour in the CM case is acceptable, then I'm 100% in favour of 
that.

I'd assumed that it was somehow unacceptable, but you're right, and it's not 
clear if Eryk is agreeing or disagreeing with you (I assumed he was 
disagreeing, based mainly on the length and complexity of his response :-)) It 
didn't help that somehow Steve's reply wasn't visible when I posted mine.

Apologies for the noise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Ethan Furman]

Ethan Furman  added the comment:

Paul,

If "rescuing"  (i.e. "fixing" ;) NamedTemporaryFile was arduous, complicated, 
or had serious backwards-compatibility issues then I would completely agree 
with you.  However, the fix is simple, the only backwards-compatible issue is 
the file would still be there /while a context manager was in use/ after it had 
been closed (which conforms to most, if not all, users of context managers) and 
a file would be left on disk in the event of a hard crash (hopefully a rare 
occurrence).

Your proposal, on the other, is a lot of work.  Is the minor backwards 
compatibility worth all that work?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Paul Moore]

Paul Moore  added the comment:

There's a lot of technical discussion of implementation details here, but not 
much about use cases. IMO, what's more important is whether NamedTemporaryFile 
is *useful* to people, and what they want to use it *for*. Working out how to 
implement it can come after we know what people want it to do.

My particular use case is actually pretty simple, and I suspect constitutes a 
fairly major proportion of what people want of this API:

1. Create a temporary file.
2. Write some data into it.
3. At some point, we're done writing to the file.
4. Only after that point, pass the name of the file to "somewhere else" for 
processing. That's often, in my use cases, as an argument to a subprocess.
5. Once we're all done, clean up securely.

The key additional requirement is that this is done "safely" (by which I mean I 
don't have to think about race conditions, etc, as someone else has that 
covered). What I think that means is that we need to maintain an open 
filehandle of *some* sort continually, but there's a point where the user can 
say "I'm done writing, I want to share this now".

Is there an actual known use case for the behaviour of deleting the file on 
close *rather* than at the end of the CM's scope? That's unlike any other CM I 
know of, where the scope ending is what triggers tidy-up.

Maybe NamedTemporaryFile should be retained for backward compatibility, but as 
a normal function, with the CM behaviour deprecated. It would still be 
essentially useless on Windows, but we maybe don't care about that.

In addition, we create a *new* context manager, that simply creates the file at 
the start and deletes it at close of scope. It returns a writeable file object, 
but that can be closed (and the documentation notes that for portability it 
*must* be closed before passing the name to another process for use).

I don't know enough about how we protect this against race condition attacks, 
but I'm pretty sure that this API gives us the best chance we're likely to have 
of doing so in a cross-platform manner. (Maybe we have a "reopen" method rather 
than "close". Or can we open *two* file handles at the start, one for writing 
and one for reading, return the one for writing, and keep the one for reading 
internal, purely to keep the file locked? I feel like that wouldn't work 
because if *we* can open a write handle, so could an attacker - but as I say, 
I'm not an expert).

Basically, I may be wrong, but I feel that we should stop trying to "rescue" 
NamedTemporaryFile, and instead try providing an *alternative* that handles the 
cross-platform use case that started all of this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Steve Dower]

Steve Dower  added the comment:

O_TEMPORARY is clearly not the right option here, and we should just move the 
unlink call into __exit__ and attempt it even if close() has been called.

Windows's "delete on close" functionality is great, but if you haven't designed 
for its semantics, it's unusable. And this library is not designed for it, so 
users can't rely on the native functionality.

So we stop passing the O_TEMPORARY flag. If __enter__() is called, close() 
closes the file but doesn't delete anything, and __exit__() closes the file (if 
open) and deletes it (even if it wasn't open). If there is no __enter__(), 
close() also deletes the file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Ethan Furman]

Ethan Furman  added the comment:

Eryk, I'm not sure if you are agreeing or disagreeing with me. :)

On Windows it sounds like O_TEMPORARY buys us guaranteed file deletion, but 
costs us easy sharing of file resources and a difference in semantics between 
Windows and non-Windows.

Is the automatic deletion truly that valuable?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> My opinion is that no extra flags are necessary.  The default of 
> deleting on close is fine, unless a context manager is active -- in 
> which case delete on CM exit.

There is a use case of needing to let another thread or process open the 
temporary file while in a given context, but ensure that the file is deleted 
when the context exits. The O_TEMPORARY flag is not generally compatible with 
this use case, since very few programs in Windows share delete access. Python's 
open() doesn't, not without an opener. So this case needs to omit the 
O_TEMPORARY flag and rely on the context manager to delete the file.

If there's no need to reopen the file in another thread or process, then using 
O_TEMPORARY is preferred. In this case, the file will deleted even if the 
current process crashes or gets terminated (e.g. by a job object).

NamedTemporaryFile() in Windows could switch to relying on the context manager 
to delete the file. But also add an implementation of TemporaryFile() in 
Windows that uses O_TEMPORARY. Surely if a script has no need to reopen a 
temporary file, then it shouldn't care what the file's name is. 

For example:

def TemporaryFile(mode='w+b', buffering=-1, encoding=None,
  newline=None, suffix=None, prefix=None,
  dir=None, *, errors=None):
if "b" not in mode:
encoding = _io.text_encoding(encoding)

prefix, suffix, dir, output_type = _sanitize_params(prefix, suffix, dir)
flags = _bin_openflags | _os.O_TEMPORARY

(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
try:
_os.unlink(name)
return _io.open(fd, mode, buffering=buffering,
newline=newline, encoding=encoding, errors=errors)
except:
_os.close(fd)
raise

Prior to Windows 10 (tested back to Python 3.2 in Windows 2000), the 
unlink(name) call will leave the file linked in `dir`, but trying to access it 
with a new open will fail with an access-denied error. A file that's in a 
deleted state is only accessible by existing opens. The downside is that the 
temporary file can't be moved to another directory except by an existing open 
(e.g. via SetFileInformationByHandle: FileRenameInfo). Another process that 
wants to delete `dir` won't be able to move the file out of the way. It 
shouldn't be an issue, however, if the file is created in the user's temp 
directory.

In Windows 10, NTFS implements a POSIX delete that also moves the file into a 
reserved system directory, so it doesn't remain linked in `dir` and thus 
doesn't prevent deleting `dir`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Ethan Furman]

Ethan Furman  added the comment:

On 4/11/2021 3:51 PM, Jason R. Coombs wrote:
 > Jason R. Coombs  added the comment:
 >
 > At least I and Ethan and Martin have expressed a desire for the 
default, preferred usage work well in a portable environment. Requiring 
`delete_on_close=False` violates that expectation.

My opinion is that no extra flags are necessary.  The default of 
deleting on close is fine, unless a context manager is active -- in 
which case delete on CM exit.  Note that an internal flag will be needed 
to track the status of being in a context manager, but nothing besides 
behavior in that edge case would change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Jason R. Coombs]

Jason R. Coombs  added the comment:

At least I and Ethan and Martin have expressed a desire for the default, 
preferred usage work well in a portable environment. Requiring 
`delete_on_close=False` violates that expectation.

How about something like this instead:

- Add an `delete_when=None`, also accepting "close" and "exit".
- "close" means delete on close.
- "exit" means delete when the context manager exits.
- When `delete_when` is None, the default behavior is selected (currently 
`close`).
- At some point (now or in the future), raise a deprecation warning if 
`delete_when=None` is passed (require it to be explicit) and explain that the 
default in the future will be `delete_when="exit"`.
- Document that passing an explicit `None` for `delete_when` is not supported 
(don't do it).
- In a release after the deprecation has been released, change the default to 
`delete_when="exit"` and drop support for `None`.


Note, this deprecation approach could be enacted with "delete_on_close" and 
boolean values, but I prefer more explicit values for shorter-named variables.


This approach would allow a user to opt in to the future behavior which has the 
desired effect of preferring the default behavior (in as little as two 
releases).

I might be tempted to create a `backports` package for users of earlier Python 
versions to get the future behavior sooner.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Evgeny  added the comment:

Dear all, how can we realistically move this forward?

This issue is 9 years old by now.
Everybody from the discussion agrees, this is an issue.


There were several proposals made, all of them slightly different.

7 months ago I have implemented solution, pretty much similar to the one, 
proposed by eryksun is the https://bugs.python.org/msg376656

https://github.com/python/cpython/pull/22431

So far I received no comments. 
I am not quite familiar with the decision making in Python development. Who can 
take a decision which way to go?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Ethan Furman]

Ethan Furman  added the comment:

I think the best solution, albeit slightly backwards incompatible, is to change 
NamedTemporaryFile such that if (and only if) it is being used as a context 
manager, the underlying file is not closed until the context manager ends.  
This should be the default behavior with no new flags.

Since context managers are used for resource acquisition (`__enter__`) and 
cleanup (`__exit__`), having the resource disappear in the middle of the `with` 
block is counter-intuitive.

Given that closing a file in the middle of its CM seems like a very odd thing 
to do (one could just leave the CM and the file would close), do we need a 
deprecation period or can we just make the change?

--
nosy: +ethan.furman
title: tempfile.NamedTemporaryFile not particularly useful on   Windows -> 
tempfile.NamedTemporaryFile not particularly useful on Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Change by Evgeny :


--
keywords: +patch
pull_requests: +22618
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/22431

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Evgeny]

Evgeny  added the comment:

Hello, this is to let you know, that I have created a pull request for this 
issue

https://github.com/python/cpython/pull/22431

I am not really an experienced programmer, but I will give it a try

--
nosy: +ev2geny

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> We'd CreateFile the file and then immediately pass it to 
> _open_osfhandle

Unlike _wopen, _open_osfhandle doesn't truncate Ctrl+Z (0x1A) from the last 
byte when the flags value contains _O_TEXT | _O_RDWR. _wopen implements this to 
allow appending data in text mode. The implementation is based on GetFileType 
(skip pipes and character devices), _lseeki64, _read, and _chsize[_s].

O_TEXT (ANSI text mode) has to be supported for now, but it doesn't properly 
fit in Python 3. The io module opens files using the CRT's binary mode. It 
doesn't implement newline translation for bytes I/O. And io.TextIOWrapper 
doesn't support Ctrl+Z as a logical EOF marker. 

As long as it's supported, O_TEXT should be made the default in os.open (but 
not in msvcrt.open_osfhandle), independent of the CRT default fmode (i.e. 
_get_fmode and _set_fmode). Many callers already assume that's the case. For 
example, tempfile.mkstemp with text=True uses tempfile._text_openflags, which 
doesn't include os.O_TEXT. That assumption is currently wrong if 
_set_fmode(_O_BINARY) is called.

Thankfully, Python has never documented support for the _O_WTEXT, _O_U16TEXT, 
and _O_U8TEXT Unicode text modes in os.open. To my knownledge, there is no 
reasonable way to reimplement these modes. The C runtime doesn't expose a 
public interface to modify a file's internal text and Unicode modes, and 
_open_osfhandle only supports ANSI text mode. If _Py_wopen is implemented, it 
will have to fail the Unicode (UTF-16 or UTF-8) modes with EINVAL. Even without 
_Py_wopen, I'd prefer to modify os.open to fail them because wrapping a 
Unicode-mode fd with io.FileIO doesn't function reliably. FileIO doesn't 
guarantee wchar_t aligned reads and writes, which the CRT requires in Unicode 
mode.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Steve Dower]

Steve Dower  added the comment:

We'd CreateFile the file and then immediately pass it to _open_osfhandle, which 
would keep the semantics the same apart from the share flags.

I'm not entirely against getting rid of O_TEXT support, but haven't taken the 
time to think through the implications.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> we'd have to reimplement the UCRT function using the system API. 

Could the implementation drop support for os.O_TEXT? I think Python 3 should 
have removed both os.O_TEXT and os.O_BINARY. 3.x has no need for the C 
runtime's ANSI text mode, with its Ctrl+Z behavior inherited from MS-DOS. I'd 
prefer that os.open always used _O_BINARY and raised a ValueError if passed any 
of the C runtime's text modes, including _O_TEXT, _O_WTEXT, _O_U16TEXT, and 
_O_U8TEXT.

If _O_TEXT is supported, then we have to copy the C runtime's behavior, which 
truncates a Ctrl+Z from the end of the file if it's opened with read-write 
access. If Unicode text modes are supported, then we have to read the BOM, 
which can involve opening the file twice if the caller doesn't request read 
access.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Steve Dower]

Steve Dower  added the comment:

The comment you quoted was referring to the NamedTemporaryFile(do_not_delete) 
flag.

Yes, we'd have to reimplement the UCRT function using the system API. 
Ultimately, it's not a great compatibility layer if you want to match POSIX 
semantics and not just the C specification, which is why we do it so often :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> Nothing preventing someone from contributing the flag on open as well. 

To be clear, supporting delete-access sharing would require re-implementing C 
_wopen in terms of CreateFileW, _open_osfhandle, etc. It could be implemented 
as _Py_wopen in Python/fileutils.c.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Steve Dower]

Steve Dower  added the comment:

Issue41490 can also be fixed by using FILE_SHARE_DELETE on all opened files 
(and that's a release blocker, so we need to fix it somehow), and if DeleteFile 
has been updated as you suggest then it might even help with the "pip replacing 
its own script executable" issue.

Nothing preventing someone from contributing the flag on open as well. There's 
definitely value there, but I think it's a workaround when we can make things 
Just Work more transparently.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> Why do we need to use this O_TEMPORARY flag at all?

Using the O_TEMPORARY flag isn't necessary, but it's usually preferred because 
it ensures the file gets deleted even if the process is terminated abruptly. 
However, it opens the file with delete access, and most Windows programs don't 
share delete access for normal file opens. This can be worked around in Python 
code by using an opener that calls CreateFileW with delete-access sharing. But 
it can't be worked around in general. 

I prefer to provide a way to omit O_TEMPORARY, but still use it by default. 
When it's omitted, I'd also like to be able to close the file within the 
context block without deleting it, for which one use case is to reopen the file 
in a program that doesn't share read or write access. A new delete_on_close 
option would support this case, in addition to providing a way to omit the 
O_TEMPORARY flag. For example:

with tempfile.NamedTemporaryFile(delete_on_close=False) as f:
f.close()
subprocess.run([cmd, f.name])

The file will still be deleted by the context manager, but the tradeoff is that 
it's not as reliable as the default delete-on-close behavior that uses the 
O_TEMPORARY flag.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Chary Chary]

Chary Chary  added the comment:

I am not sure, this is the correct place to ask this "educational" question, 
but I will do this any way:

where is this O_TEMPORARY flag defined?

if I looks at 
[tempfile.py](https://github.com/python/cpython/blob/3ff6975e2c0af0399467f234b2e307cc76efcfa9/Lib/tempfile.py#L539)
 then appears, that it is defined in the os.py module.

But in 
[os.py](https://github.com/python/cpython/blob/3ff6975e2c0af0399467f234b2e307cc76efcfa9/Lib/os.py)
 I can't find it

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Chary Chary]

Chary Chary  added the comment:

Why do we need to use this O_TEMPORARY flag at all?

I understand that we are using OS functionality, available on Windows, rather 
than implementing it in Python. But why doing this, if we already do this for 
none-nt systems in Python any way?

Doesn't it just complicate the code?

https://github.com/python/cpython/blob/fa8c9e70104b0aef966a518eb3a80a4881906ae0/Lib/tempfile.py#L423

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> For this case, I think the best thing we can probably do is change the 
> default share mode for _all_ opens to include FILE_SHARE_DELETE. 

The C runtime doesn't provide a way to share delete access, except for the 
O_TEMPORARY flag, so Python would have to re-implement open(). Also, this 
doesn't help with re-opening a temporary file in another process since most 
programs do not share file delete access. 

There could be an option to enable a context-manager delete that's independent 
of closing the file. For example, if delete=True and delete_on_close=False, 
then _TemporaryFileCloser.close doesn't delete the file. Instead the file would 
be deleted via os.unlink in _TemporaryFileWrapper.__exit__. The default would 
be delete=True and delete_on_close=True, which would use the O_TEMPORARY flag 
in Windows. Combining delete=False with delete_on_close=True would raise a 
ValueError.

> bringing the default Windows behaviour slightly more in line with 
> how POSIX likes to do things.

In Windows 10, using FILE_SHARE_DELETE is even closer to POSIX behavior when 
the filesystem is NTFS, which supports POSIX delete semantics by renaming the 
file to a hidden system directory ("\$Extend\$Deleted") and setting its delete 
disposition. WinAPI DeleteFileW has been updated to use POSIX semantics if the 
filesystem supports it:

>>> f = tempfile.NamedTemporaryFile()
>>> h = msvcrt.get_osfhandle(f.fileno())
>>> os.unlink(f.name)
>>> info = GetFileInformationByHandleEx(h, FileStandardInfo)
>>> info['DeletePending']
True
>>> GetFinalPathNameByHandle(h, 0)
'?\\C:\\$Extend\\$Deleted\\00180002C4F4301F419F'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Chary Chary]

Chary Chary  added the comment:

Steve Dower, thanks for looking at this.

After reading the thread from my amature point of view I kind of liked 
suggestion of Daniel Lenski to replace the binary delete argument of the 
current NamedTemporaryFile implementation with finer-grained options 
https://bugs.python.org/issue14243#msg164369

This would also take care of the comment from Dave Abrahams, that 

<> https://bugs.python.org/issue14243#msg155457

As for your comment to include FILE_SHARE_DELETE. If the decision is taken to 
go this path, shall we also not include FILE_SHARE_READ and FILE_SHARE_WRITE?
https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Steve Dower]

Steve Dower  added the comment:

In general, if a bug here appears to be inactive, it's probably waiting on 
someone to volunteer to move it forward. Often merely posting to a thread is 
enough.

For this case, I think the best thing we can probably do is change the default 
share mode for _all_ opens to include FILE_SHARE_DELETE. This would also help a 
number of other situations, as well as bringing the default Windows behaviour 
slightly more in line with how POSIX likes to do things.

As far as I'm aware this would only be harmful in cases where people are trying 
to implicitly lock files on Windows by keeping an open handle, and are using a 
different code path on other platforms where that won't work.

--
versions: +Python 3.10 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Chary Chary]

Chary Chary  added the comment:

Dear all, are there any plans to move this quite old issue forward?

I stumbled across this issue, because I found that at the moment there is no 
out of the box solution to use  tempfile.NamedTemporaryFile in Windows in such 
scenario (which is often used in unit testing):

* in test module:
1) create and open temporary file
2) write data to it
3) pass name of the temporary file to the operational code

* In operational code, being tested
1) open file, using name of the temporary file 
2) read data from this temporary file 

--
nosy: +chary314

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Ethan Smith]

Change by Ethan Smith :


--
nosy: +Ethan Smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Nathaniel Smith]

Changes by Nathaniel Smith :


--
nosy: +njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun added the comment:

Richard wrote:
> while a handle is open with share mode X, you can only reopen 
> the file if you also use share mode X

To clarify, the share mode is not a property of a handle. It's a property of a 
File object. A handle is a generic reference to any kind of kernel object, not 
just a File.

The following is a brief discussion about access sharing and the way file 
deletion works in Windows. Both of these tend to frustrate Unix programmers who 
end up supporting Windows. In this discussion, a "File", with an uppercase 'F', 
is the Windows kernel object type that references an open device or file-system 
directory, file, or stream. A "file", with a lowercase 'f', is a data file.

Shared access is implemented for File objects [1] and tracked in a SHARE_ACCESS 
record. When opening a file or directory, its shared access state is updated by 
the kernel function IoCheckShareAccess [2]. Discretionary shared access is 
primarily a concern for file systems, but volume devices and disk devices (e.g. 
\\.\C: and \\.\PhysicalDrive0) also use it. Devices that are flagged for 
mandatory exclusive access [3] (e.g. \\.\COM1) generally ignore the share mode. 
Some non-exclusive devices also ignore the share mode (e.g. \\.NUL and \\.CON). 

If it's not ignored, the share mode affects delete, write, read, and execute 
access. The following File access rights are affected: DELETE, FILE_WRITE_DATA, 
FILE_APPEND_DATA, FILE_READ_DATA, and FILE_EXECUTE. The share mode thus affects 
any combination of generic access -- GENERIC_ALL, GENERIC_WRITE, GENERIC_READ, 
GENERIC_EXECUTE. 

A File object's requested sharing is stored in its SharedDelete, SharedWrite, 
and SharedRead members. The granted access that's relevant to the share mode is 
stored in the DeleteAccess, WriteAccess (write/append), and ReadAccess 
(read/execute) members. Given these values, checking for a sharing violation 
and updating the shared access counts uses the following logic:

RequireSharedDelete = DeleteAccessCount > 0;
RequireSharedWrite = WriteAccessCount > 0;
RequireSharedRead = ReadAccessCount > 0;
DenyDeleteAccess = SharedDeleteCount < OpenCount;
DenyWriteAccess = SharedWriteCount < OpenCount;
DenyReadAccess = SharedReadCount < OpenCount;

if (RequireSharedDelete && !SharedDelete ||
RequireSharedWrite && !SharedWrite ||
RequireSharedRead && !SharedRead ||
DenyDeleteAccess && DeleteAccess ||
DenyWriteAccess && WriteAccess ||
DenyReadAccess && ReadAccess)
{
return STATUS_SHARING_VIOLATION;
}

OpenCount++;
DeleteAccessCount += DeleteAccess;
WriteAccessCount += WriteAccess;
ReadAccessCount += ReadAccess;
SharedDeleteCount += SharedDelete;
SharedWriteCount += SharedWrite;
SharedReadCount += SharedRead;

For example, to be granted delete access, all existing File object references 
must share delete access. However, if a file is opened with delete sharing but 
delete access hasn't been granted, then it can be opened again without delete 
sharing.

The SHARE_ACCESS structure is usually stored in a file (or stream) control 
block (FCB/SCB), which is a structure that coordinates access to a file or 
directory across multiple File objects. The FsContext member of a File object 
points at the FCB. 

A file system stores its private state for a File in a context control block 
(CCB), to which the File's FsContext2 member points. The CCB is where a file 
system tracks, for example, whether the file should be deleted when the object 
is closed. 

Deleting a file sets a delete disposition in the FCB/SCB (or LCB if hard links 
are supported). The file can't be unlinked until all referencing File objects 
have been closed and the underlying FCB/SCB/LCB is closing. Until then a 
'deleted' file is still linked in the parent directory and prevents the 
directory from being deleted.

A deleted but still referenced file is in a semi-zombie state. Windows file 
systems don't allow opening such a file for any access, but it can still be 
accessed via existing objects. If one of these File objects has delete access, 
it can be used to unset the delete disposition (e.g. via 
SetFileInformationByHandle) to make the file accessible again.

[1]: https://msdn.microsoft.com/en-us/library/ff545834
[2]: https://msdn.microsoft.com/en-us/library/ff548341
[3]: https://msdn.microsoft.com/en-us/library/ff563827

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Eryk Sun]

Eryk Sun added the comment:

Nick wrote:
> 1. Open the file in the current process
> 2. Write content to the file
> 3*. Close the file in the current process

In step 1, do you mean calling NamedTemporaryFile with delete=False? In that 
case there's no immediate problem with opening the file again in Windows. If 
you mean calling NamedTemporaryFile with delete=True, then step 3 deletes the 
file. 

Adding support for Windows share modes would be useful in general and would 
help within the current process. However, users may also need to open the 
temporary file in another process. Most programs don't open their files with 
shared delete access. 

There's a workaround to allow the file to be opened normally, but it involves 
setting the delete disposition and then clearing it in a pointless dance. It 
would be better to implement an option such as delete=AFTER_CM_EXIT, to try to 
remove the file without relying on O_TEMPORARY. The downside is that the file 
won't be deleted if the interpreter crashes, gets terminated or if another 
process has the file open without delete sharing.

--
nosy: +eryksun
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Nick Coghlan]

Nick Coghlan added the comment:

John: I don't think it would be clutter to have an explicit reminder about that 
point in the NamedTemporaryFile documentation, so feel free to file a separate 
enhancement issue for it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[John Florian]

John Florian added the comment:

Okay Nick. Thanks for the detailed info. I suspected buffering was a factor, 
but wasn't certain. Would it be worthwhile pursuing a note in the docs or would 
that constitute clutter over what should be a standard assumption? I was thrown 
off course for all the prior uses without issues, but in hindsight I don't know 
offhand how many involved a subprocess.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Nick Coghlan]

Nick Coghlan added the comment:

John, your problem sounds different - if you're opening the files in binary 
mode, then you'll be getting a default buffer that's probably 4k or 8k in size, 
so if you're writing less content than that, the subprocess won't see anything 
until you explicitly flush() the buffer to disk (and even if you're writing 
more than that, the subprocess may see a truncated version without an explicit 
flush()). 

By contrast, the issue here relates to the fact that on Windows it's currently 
necessary to do the following in order to get multiple handles to a 
NamedTemporaryFile:

1. Open the file in the current process
2. Write content to the file
3*. Close the file in the current process
4. Open the file by name in another process (or the current process)
5. Read content from the file
6. Close the second file handle
7*. Delete the file

*On POSIX systems, you can just skip step 3 and leave closing the original file 
handle until step 7, and that's the design that the current NamedTemporaryFile 
is built around. Most of the discussion above is about figuring out how to make 
that same approach "just work" on Windows (perhaps with some additional flags 
used in step 4), rather than  providing a third option beyond the current 
delete-on-close and delete-manually.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Jakub Wilk]

Changes by Jakub Wilk :


--
nosy: +jwilk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[John Florian]

John Florian added the comment:

I just stumbled onto this though I'm not writing for Windows.  Instead, I'm on 
Fedora 25 with Python 3.5.2 and I went nearly crazy tracing down what seemed to 
be inconsistent behavior.  My use case has Python using 
NamedTemporaryFile(delete=True) in a CM to produce content fed into a 
subprocess.  The code had been reliably working and then it just didn't.  The 
only thing changing was the content being written, an rendered Jinja2 template. 
 I believe the fate is determined by the content length.  In debugging another 
problem, I'd been trivializing the template and once it got down to about 3k 
(rendered) the subprocess began seeing a file whose length was 0 bytes.  Make 
the template bigger and all works again.  Calling close() resolves the issue, 
but of course requires delete=False which removed much of the value of this 
object.  Preliminary testing looks like flush() may also resolve the issue.

Have I just been naive and getting lucky all along because this is expected or 
is there something else fishy here worth investigation?

--
nosy: +John Florian

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Paul Moore]

Changes by Paul Moore p.f.mo...@gmail.com:


--
nosy: +paul.moore

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Martin Panter]

Changes by Martin Panter vadmium...@gmail.com:


--
nosy: +vadmium

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Mark Lawrence]

Mark Lawrence added the comment:

This is one of several Windows file related issues, see also #12939, #21859, 
#15244 and possibly others.  How can we take this forward? If it's of any use I 
can help with testing as I've a new Windows 8.1 HP laptop with 8G of memory.

--
nosy: +BreamoreBoy, steve.dower, zach.ware -brian.curtin
versions: +Python 3.5 -Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Gabi Davar]

Changes by Gabi Davar grizzly@gmail.com:


--
nosy: +Gabi.Davar

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Piotr Dobrogost]

Piotr Dobrogost added the comment:

@sbt

Thanks for info. Also you mentioned looking at c:/Program Files (x86)/Microsoft 
Visual Studio 10.0/VC/crt/src/open.c What version of Visual Studio/SDK this 
file is available in?

Also I'd like to point out that this problem came up at Stack Overflow in 
question How to create a temporary file that can be read by a subprocess? 
(http://stackoverflow.com/q/15169101/95735)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Piotr Dobrogost]

Piotr Dobrogost added the comment:

@sbt

 (...) and it seems that on Windows open() is more or less implemented
 as a wrapper of sopen(..., ..., SH_DENYNO, ...).

 So the only reason that trying to reopen a NamedTemporaryFile fails on
 Windows is because when we reopen we need to use O_TEMPORARY.

Could you elaborate on this? What's the relation between SH_DENYNO argument to 
sopen() and O_TEMPORARY flag?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Richard Oudkerk]

Richard Oudkerk added the comment:

Sorry, I was not very clear.

If you use the O_TEMPORARY flag with open() to get a file handle, then the 
share mode used with the underlying CreateFile() function is 

X = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE

whereas, if you don't use O_TEMPORARY then the share mode is

Y = FILE_SHARE_READ | FILE_SHARE_WRITE

While a handle is open with share mode X, you can only reopen the file if you 
also use share mode X.  Therefore (using the msvcrt) you can only reopen it 
using O_TEMPORARY.*


* sopen() does give some extra control over the share mode, but you still can't 
use it to get share mode X without also using O_TEMPORARY.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Piotr Dobrogost]

Changes by Piotr Dobrogost p...@bugs.python.dobrogost.net:


--
nosy: +piotr.dobrogost

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Richard Oudkerk]

Richard Oudkerk shibt...@gmail.com added the comment:

I have opened Issue #15244 with a patch to add a share module to the stdlib.

After monkey patching builtins.open(), io.open() and os.open() to be 
equivalents using FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, the 
regression test suite still runs successfully.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Tim Golden]

Tim Golden m...@timgolden.me.uk added the comment:

On 30/06/2012 06:45, Daniel Lenski wrote:
 My preferred solution would be to replace the binary delete argument of the 
 current NamedTemporaryFile implementation with finer-grained options:
delete=False # don't delete
delete=True  # delete after file closed, current behavior
delete=AFTER_CLOSE   # delete after file closed
delete=AFTER_CM_EXIT # delete after context manager exits
delete=AFTER_CM_EXIT_NO_EXCEPTION # delete after CM exit, unless this is 
 due to an exception

I'm aware that Richard  others are fleshing out alternatives. But my
having asked you to propose something I wanted to come back on this
particular suggestion. I think it's just too complex an API. Not least
because, on Windows, we're making use of a filesystem feature which will
delete on closure regardless (so the implementation on Windows skips the
context-based delete).

I'm not sure what we'll end up with but I'm more inclined towards the
sort of method-based closer/reopener which is more explicit.

--
title: tempfile.NamedTemporaryFile not particularly useful on Windows - 
tempfile.NamedTemporaryFile not particularly useful on Windows

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Richard Oudkerk]

Richard Oudkerk shibt...@gmail.com added the comment:

I wrote in an earlier message that a file opened with O_TEMPORARY must be 
reopened with O_TEMPORARY.  This is not quite accurate.

Using O_TEMPORARY causes the FILE_SHARE_DELETE sharing mode to be used, and a 
file currently opened with FILE_SHARE_DELETE can only be reopened with 
FILE_SHARE_DELETE.

Unfortunately using O_TEMPORARY is the only way allowed by msvcrt to get 
FILE_SHARE_DELETE, even though it also has the orthogonal effect of unlinking 
the file when all handles are closed.

The nice thing about FILE_SHARE_DELETE is that it gives Unix-like behaviour: 
the file can be renamed or deleted while you have an open handle, and you can 
still continue to use the handle.

Attached is a mostly untested attempt at writing replacements for open() and 
os.open() which use the FILE_SHARE_DELETE sharing mode.  Among other things, 
these can be used for reopening temporary files.

Even if tempfile does not use make use of this, I think something similar would 
be useful in the stdlib.

--
Added file: http://bugs.python.org/file26229/share.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Daniel Lenski]

Daniel Lenski dlen...@gmail.com added the comment:

Richard, I think the problem with this is that it spreads the non-portable
or OS-dependent parts of the code over several places rather than
concentrating them all in one place.

After close_without_unlink(), what would happen when the context manager
exits or when the object is garbage collected?  Would it then get unlinked?

My preference would be to specify the behavior of close/__exit__/GC
operations at the time of the NamedTemporaryFile creation, so that the rest
of the code can be left unchanged.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Richard Oudkerk]

Richard Oudkerk shibt...@gmail.com added the comment:

The webpage

http://msdn.microsoft.com/en-us/library/aa273350(v=vs.60).aspx

describes the sopen() function which is like open() but has an extra shflag 
parameter for specifying the sharing allowed.

If sopen() and the associated constants SH_DENYRD, SH_DENYWR, SH_DENYRW and 
SH_DENYNO were exposed in the os module, then maybe tempfile could use 
os.sopen() on Windows instead of os.open() to allow the file to be reopened 
without closing.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 If sopen() and the associated constants SH_DENYRD, SH_DENYWR, SH_DENYRW 
 and SH_DENYNO were exposed in the os module, then maybe tempfile could 
 use os.sopen() on Windows instead of os.open() to allow the file to be 
 reopened without closing.

Sounds like a good way forward.

--
components: +Windows
stage:  - needs patch
versions: +Python 3.4 -Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Tim Golden]

Tim Golden m...@timgolden.me.uk added the comment:

On 01/07/2012 21:37, Antoine Pitrou wrote:

 Antoine Pitrou pit...@free.fr added the comment:

 If sopen() and the associated constants SH_DENYRD, SH_DENYWR, SH_DENYRW
 and SH_DENYNO were exposed in the os module, then maybe tempfile could
 use os.sopen() on Windows instead of os.open() to allow the file to be
 reopened without closing.

 Sounds like a good way forward.

Agreed. Richard: do you have time to put something together?
I'm happy to try if you don't.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Richard Oudkerk]

Richard Oudkerk shibt...@gmail.com added the comment:

 Agreed. Richard: do you have time to put something together?
 I'm happy to try if you don't.

I'm looking into it.

Unfortunately, it seems that you need to use non-default flags when reopening a 
shared file.  Eg, if the file is currently opened with SH_DENYNO and 
O_TEMPORARY, then you must reopen it using SH_DENYNO and O_TEMPORARY.

However, I have an initial implementation of os.sopen() which makes the 
following work:

  import os, tempfile

  FNAME = foo.txt
  DATA = hello bob

  def opener(name, flag, mode=0o777):
  return os.sopen(name, flag | os.O_TEMPORARY, os.SH_DENYNO, mode)

  with open(FNAME, w, opener=opener) as f:
  f.write(DATA)
  f.flush()
  with open(FNAME, r, opener=opener) as f:
  assert f.read() == DATA

  assert not os.path.exists(FNAME)

BTW, Maybe it would be better to add a keyword-only shareflag argument to 
os.open() rather than add os.sopen().

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Richard Oudkerk]

Richard Oudkerk shibt...@gmail.com added the comment:

I checked the source in

   c:/Program Files (x86)/Microsoft Visual Studio 10.0/VC/crt/src/open.c

and it seems that on Windows open() is more or less implemented as a wrapper of 
sopen(..., ..., SH_DENYNO, ...).

So the only reason that trying to reopen a NamedTemporaryFile fails on Windows 
is because when we reopen we need to use O_TEMPORARY.

The following works for unmodified python:

  import os, tempfile

  DATA = bhello bob

  def temp_opener(name, flag, mode=0o777):
  return os.open(name, flag | os.O_TEMPORARY, mode)

  with tempfile.NamedTemporaryFile() as f:
  f.write(DATA)
  f.flush()
  with open(f.name, rb, opener=temp_opener) as f:
  assert f.read() == DATA

  assert not os.path.exists(f.name)

So maybe we should just define tempfile.opener().

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Nick Coghlan]

Nick Coghlan ncogh...@gmail.com added the comment:

Alternatively, perhaps it would make sense to have a reopen() method on file 
objects that covers the necessary dance to reopen with the correct flags?

That would solve more problems than just this one (possibly including making it 
possible to reopen StringIO and BytesIO objects).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Davide Rizzo]

Davide Rizzo sor...@gmail.com added the comment:

Daniel, Nick, shouldn't the context manager yield f within a with block?

--
nosy: +davide.rizzo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Richard Oudkerk]

Richard Oudkerk shibt...@gmail.com added the comment:

Rather than add a NamedTemporaryFile.delete_after() classmethod, would it not 
be simpler to just add a close_without_unlink() method to NamedTemporaryFile?

with NamedTemporaryFile() as f:
write to f
f.close_without_unlink()
with open(f.name, 'rb') as f:
read from f

--
nosy: +sbt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Daniel Lenski]

Daniel Lenski dlen...@gmail.com added the comment:

Davide, the @contextlib.contextmanager decorator effectively wraps the
yield statement in the necessary glue so that everything prior to the yield
statement occurs in the __enter__() method of the contextmanager, while
everything subsequent occurs in the __exit__() method.

On Sat, Jun 30, 2012 at 1:46 AM, Davide Rizzo rep...@bugs.python.orgwrote:


 Davide Rizzo sor...@gmail.com added the comment:

 Daniel, Nick, shouldn't the context manager yield f within a with block?

 --
 nosy: +davide.rizzo

 ___
 Python tracker rep...@bugs.python.org
 http://bugs.python.org/issue14243
 ___


--
title: tempfile.NamedTemporaryFile not particularly useful on   Windows - 
tempfile.NamedTemporaryFile not particularly useful on Windows

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Daniel Lenski]

Changes by Daniel Lenski dlen...@gmail.com:


--
nosy: +dlenski

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Tim Golden]

Tim Golden m...@timgolden.me.uk added the comment:

Daniel. If you have any interest in this issue, would you mind 
summarising the state of affairs, please? I have no direct interest in 
the result but I'm happy to commit a patch or even to work one up if 
somone can come up with a single, concrete suggestion.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Daniel Lenski]

Daniel Lenski dlen...@gmail.com added the comment:

Tim Golden,
My preferred solution would be to replace the binary delete argument of the 
current NamedTemporaryFile implementation with finer-grained options:
   delete=False # don't delete
   delete=True  # delete after file closed, current behavior
   delete=AFTER_CLOSE   # delete after file closed
   delete=AFTER_CM_EXIT # delete after context manager exits
   delete=AFTER_CM_EXIT_NO_EXCEPTION # delete after CM exit, unless this is due 
to an exception

I have implemented a Windows-friendly solution to the latter case using Nick 
Coghlan's code.  My version does not delete the file until the context manager 
exits, and *if* the context manager exits due to an exception it leaves the 
file in place and reports its location, to aid me in debugging.

--
Added file: http://bugs.python.org/file26215/ntempfile.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

delete_after what? I know it is somewhat implicit in the fact that it is a 
context manager call, but that is not the only context the method name will be 
seen in. (eg: 'dir' list of methods, doc index, etc).  Even as a context 
manager my first thought in reading it was delete after what?, and then I 
went, oh, right.

How about delete_on_exit?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

By the way, I still think it would be nicer just to have the context manager 
work as expected with delete=True (ie: doesn't delete until the end of the 
context manager, whether the file is closed or not).  I'm OK with being voted 
down on that, though.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

 By the way, I still think it would be nicer just to have the context
 manager work as expected with delete=True (ie: doesn't delete until
 the end of the context manager, whether the file is closed or not).
 I'm OK with being voted down on that, though.

Indeed, the current behaviour under Windows seems to be kind of a
nuisance, and having to call a separate method doesn't sound very
user-friendly.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Jason R. Coombs]

Jason R. Coombs jar...@jaraco.com added the comment:

I agree. If the primary usage of the class does not work well on Windows, 
developers will continue to write code using the primary usage because it works 
on their unix system, and it will continue to cause failures when run on 
windows. Because Python should run cross-platform, I consider this a bug in the 
implementation and would prefer it be adapted such that the primary use case 
works well on all major platforms.

If there is a separate class method for different behavior, it should be for 
the specialized behavior, not for the preferred, portable behavior.

I recognize there are backward-compatibility issues here, so maybe it's 
necessary to deprecate NamedTemporaryFile in favor of a replacement.

--
title: tempfile.NamedTemporaryFile not particularly useful on Windows - 
tempfile.NamedTemporaryFile not particularly useful on Windows

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

Well, fixing NamedTemporaryFile in either of the ways we've discussed isn't 
going to fix people writing non-portable code.  A unix coder isn't necessarily 
going to close the file before reading it.  However, it would at least 
significantly increase the odds that the code would be portable, while the 
current situation *ensures* that the code is not portable.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Nick Coghlan]

Nick Coghlan ncogh...@gmail.com added the comment:

I agree we need to add something here to better support the idiom where the 
close and delete operations on a NamedTemporaryFile are decoupled without 
the delete becoming a completely independent call to os.unlink().

I agree with RDM's proposal in issue 14514 that the replacement should be 
delete on __exit__ but not on close. As with generator context managers, I'd 
also add in the last ditch cleanup behaviour in __del__.

Converting the issue to a feature request for 3.3 - there's no bug here, just 
an interaction with Windows that makes the existing behavioural options 
inconvenient.

After all, you can currently get deterministic cleanup (with a __del__ 
fallback) via:

  @contextmanager
  def named_temp(name):
f = NamedTemporaryFile(name, delete=False)
try:
yield f
finally:
try:
os.unlink(name)
except OSError:
pass

You need to be careful to make sure you keep the CM alive (or it will delete 
the file behind your back), but the idiom RDM described in the other issues 
handles that for you:

  with named_temp(fname) as f:
 data = Data\n
 f.write(data)
 f.close() # Windows compatibility
 with open(fname) as f:
 self.assertEqual(f.read(), data)

As far as the API goes, I'm inclined to make a CM with the above behavour 
available as a new class method on NamedTemporaryFile:

  with NamedTemporaryFile.delete_after(fname) as f:
  # As per the workaround

--
title: NamedTemporaryFile unusable under Windows - tempfile.NamedTemporaryFile 
not particularly useful on Windows
type: behavior - enhancement
versions:  -Python 2.7, Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14243] tempfile.NamedTemporaryFile not particularly useful on Windows

[Nick Coghlan]

Nick Coghlan ncogh...@gmail.com added the comment:

Although, for the stdlib version, I wouldn't suppress the OS Error (I'd follow 
what we currently do for TemporaryDirectory)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14243
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com