The file that is downloaded is deterministically named, except for the
pid in the directory name (download/<pid>) which I didn't change as part
of this. The downloaded (compressed) file is named exactly as it is on
the server.
The temporary file is only used when the file is decompressed and then
renamed into the right path. I don't see how this has anything to do
with mirroring and content caching. Wouldn't you be caching the
compressed files?
Does anyone have any suggestion on how to solve this problem without
using mkstemp()?
Here are the constraints:
1) the temporary uncompressed file has to be in the same directory as
the eventual target, because it replaced the target file by doing a
hardlink (we don't want to have to copy it across file systems there).
2) the temporary uncompressed pathname has to have a reasonable chance
of not being longer than the pathname of any file in the image. We don't
want the packaging system to be the limit on the length of pathnames
that can be delivered on Windows. With mkstemp, the file name ends up
being about 8 characters. In the degenerate case where the target file
is name "a" and happens to be the longest path, the packaging system
will still be the limit on pathname length. But this is better than
what we had before where the temporary name was at least 40 characters
longer than the target file name.
We could do something like just adding ".tmp" to the target file name,
but then we would have a problem if both "foo" and "foo.tmp" were
delivered into the same directory. "foo.tmp" would get overwritten when
"foo" was installed. This same problem exists for any extension. The
only reason the current method works is that it is highly unlikely that
someone wants to deliver both "foo" and ".mypkg-foo-<hashval>" in the
same directory.
I really don't think using mkstemp is a problem in the case. I'm
waiting for Danek to return to understand why he thinks this is a problem.
Tom
[EMAIL PROTECTED] wrote:
I've been making some changes to filelist while considering some of the
work we'd like to do with mirroring and content caching.
That's brought me around to Danek's opinion of this change.
We really need a deterministic way of naming files that we've downloaded
and mkstemp() doesn't do that for us.
On the server, we create subdirectories that are named by partitioning
the hash into chunks. For example, the file named
0d94f45cb179d2f3de2aad4f8f0f0d8bb52df115 is stored in file/0d/94f45c.
Perhaps you could employ a similar technique?
-j
On Fri, May 16, 2008 at 05:17:37PM -0700, Danek Duvall wrote:
I'm not okay with the fix to 941; one of the good things about using the
hash is that it's deterministic, while mkstemp() isn't. I do like the
removal of file_cleanup(), but I don't think this change is appropriate.
One of the things Bart's working on at the moment is changing how the files
are downloaded, and I'm sure he has some ideas on what the download area
needs to look like. I imagine we could get away without using the
pathname, and using the hash alone, which would (I believe) fix your
problem. Please talk to him about this.
The fix to 1454 is fine, though you should get rid of the spaces at the
end of the line while you're there.
As best I can tell, the changes to setup.py are fine, though I wonder
whether the print statement in build_py_func() is just for debugging?
1068 looks fine. I don't like having to explicitly list everything, and
would rather have some way of decorating the methods so that lint would
shut up, but this is fine for now.
I might experiment with going back to using classes to get your
inheritance, but once you've loaded the module with the class, extract all
its methods into the module namespace.
Since I'm out next week, you'll need to lobby someone else to shepherd your
changes in, though I can do it once I get back if no one else has done so.
Danek
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
begin:vcard
fn:Tom Mueller
n:Mueller;Tom
org:Sun Microsystems, Inc.;Update Center/OpenInstaller Software
adr:;;21915 Hillandale Dr;Elkhorn;NE;68022;USA
email;internet:[EMAIL PROTECTED]
title:Senior Staff Engineer
tel;work:877-250-4011
tel;fax:877-250-4011
tel;home:402-916-9943
x-mozilla-html:TRUE
version:2.1
end:vcard
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
