Re: _wstat on Windows (actually stat stuff in general)
On Mon, 26 Sep 2011 08:59:29 +0200, Kean Johnston wrote: > So, for stat, we have to do a double conversion: first from the input UTF-8 > to UTF-16, and then from UTF-16 to MBCS so that we can call stat instead of > wstat so that users don't get any nasty surprises when using g_stat() on a > symlink in Vista. *le sigh* Won't that fail on filenames that have characters outside of the active codepage instead? -- < Jernej Simončič ><><><><>< http://eternallybored.org/ > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
Won't that fail on filenames that have characters outside of the active codepage instead? Sure but that will actually fail rather than silently succeeding with an incorrect value. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
2011/9/26 Jernej Simončič : > Won't that fail on filenames that have characters outside of the active > codepage instead? You can use g_win32_locale_filename_from_utf8() to obtain a short filename (8.3) that will refer to the filename with unrepresentable characters. You can then use it with "narrow" functions of the Windows API. This will not work if short filename generation is turned off in the registry. I don't know whether it's still turned on by default. Regards, Krzysztof ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
2011/9/26 Kean Johnston : > Despire the > coolness of Windows supporting time fields 64-bits wide for the least amount > of pain it would probably be best if the time fields were left at 32-bits > (although it would be really cool and forward thinking if we used 64-bits), FWIW I'm in favor of 64 bit time fields, as the event horizon for 32-bit fields is only 27 years away. Even if you think that it's very unlikely that any present day programs will still be used in 27 years, you can think of this as enabling Unix timestamp calculations with dates much further into the future than possible now. Regards, Krzysztof ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 26 Sep 2011 14:59:43 +0200, Kean Johnston wrote: >> Won't that fail on filenames that have characters outside of the active >> codepage instead? > Sure but that will actually fail rather than silently succeeding with an > incorrect value. But which is more likely to happen - user having a filename with foreign characters, or user having a symlink (which on Windows can only be created by administrators)... -- < Jernej Simončič ><><><><>< http://eternallybored.org/ > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
2011/9/26 Jernej Simončič : > But which is more likely to happen - user having a filename with foreign > characters, or user having a symlink (which on Windows can only be created > by administrators)... Based on the bugs reported against Inkscape that were caused by this kind of filename, I would say the former (foreign characters) is much more common. A lot of users have English Windows but have a local keyboard layout and name their documents in their local language (e.g. Russian, Polish) which is not representable in latin1. Regards, Krzysztof ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
> But which is more likely to happen - user having a filename with foreign > characters, or user having a symlink (which on Windows can only be created > by administrators)... You asked the question slightly incorrectly: which is more likely to happen: a user have a file on their filesystem that their system codepage cannot support or having a symlink. I'd say the latter is more frequent because every single home directory of every user has a bunch of them, whereas if a filesystem has funny characters in its file names there's a much greater chance the system generated them and can therefore read them. Plus I am fairly sure (but not 100%) that MBCS can represent all of the characters UTF-16 can (which is what the _wstat function uses). Kean ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 26 Sep 2011 15:25:06 +0200, Kean Johnston wrote: > You asked the question slightly incorrectly: which is more likely to > happen: a user have a file on their filesystem that their system codepage > cannot support or having a symlink. I'd say the latter is more frequent > because every single home directory of every user has a bunch of them, Actually, I only seem to have them in 32bit installs of Windows 7 - 64bit installs don't have them in the user profile (they are there in the root of the drive). > whereas if a filesystem has funny characters in its file names there's a > much greater chance the system generated them and can therefore read them. > Plus I am fairly sure (but not 100%) that MBCS can represent all of the > characters UTF-16 can (which is what the _wstat function uses). MBCS isn't able to represent all the characters (though depending on the codepage, it does cover a fair amount), but that's not really relevant - how do you represent тест in CP-1250? -- < Jernej Simončič ><><><><>< http://eternallybored.org/ > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
MBCS isn't able to represent all the characters (though depending on the codepage, it does cover a fair amount), but that's not really relevant - how do you represent тест in CP-1250? Give that to me as a sequence of wchar_t's and I'll figure it out :) ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 26 Sep 2011 15:45:20 +0200, Kean Johnston wrote: > Give that to me as a sequence of wchar_t's and I'll figure it out :) It's \u0442\u0435\u0441\u0442. -- < Jernej Simončič ><><><><>< http://eternallybored.org/ > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
But which is more likely to happen - user having a filename with foreign characters, or user having a symlink (which on Windows can only be created by administrators)... Are you sure about that (I havent had the time to create a quick application that tests this) but http://msdn.microsoft.com/en-us/library/windows/desktop/aa363866%28v=vs.85%29.aspx makes no mention of elevated priveliges. Are you sure you are not confusing junction points with symbolic links? ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
2011/9/26 Kean Johnston : > Plus I am > fairly sure (but not 100%) that MBCS can represent all of the characters > UTF-16 can (which is what the _wstat function uses). MBCS is misleadingly named in Microsoft documentation. It is not always a multibyte encoding. It is a locale-specific encoding that depends on your localization of Windows and AFAIK is guaranteed to be ASCII compatible, and in many cases it is actually a single-byte encoding. On English Windows, this is cp1252 which is almost identical to iso8859-1. On Polish Windows, this is cp1250. On Chinese Windows it's probably gb18030. And so on. In many versions, MBCS can only represent a little more than plain ASCII. "Funny characters" (e.g. Unicode characters not representable in the MBCS / locale encoding) are in fact rather common and cause tangible bugs when using "narrow" Windows APIs (e.g. Inkscape opens all drawings normally but this one won't open!), while it's rather hard to pin down a case where the symlink problem you mention would affect the user. The ordinary user is unlikely to create any symlinks, and I'm not even sure that _wstat would be called when selecting the linked file for opening in the file chooser. Regards, Krzysztof ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 26 Sep 2011 15:55:20 +0200, Kean Johnston wrote: > Are you sure about that (I havent had the time to create a quick > application that tests this) but > http://msdn.microsoft.com/en-us/library/windows/desktop/aa363866%28v=vs.85%29.aspx > > makes no mention of elevated priveliges. Are you sure you are not confusing > junction points with symbolic links? I'm sure. Open an unelevated command prompt and use the mklink command. You'll be able to create junctions (/j) and hardlinks (/h), but not symlinks. -- < Jernej Simončič ><><><><>< http://eternallybored.org/ > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
user. The ordinary user is unlikely to create any symlinks, and I'm not even sure that _wstat would be called when selecting the linked file for opening in the file chooser. Forget about a file chooser. I want to write a utility. I want to be able to stat files and get their sizes to display how many bytes the files in a directory uses. How do I do that with a broken _wstat? Or, I want to *write* a file chooser. I want to be able to display how many bytes a file is. A file happens to be a symlink. How do I do that with a broken _wstat? ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
I'm sure. Open an unelevated command prompt and use the mklink command. That could be weirdness with the mklink command. I'll write a test using the actual API's. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 26 Sep 2011 16:11:11 +0200, Kean Johnston wrote: > Forget about a file chooser. I want to write a utility. I want to be able > to stat files and get their sizes to display how many bytes the files in a > directory uses. How do I do that with a broken _wstat? Or, I want to > *write* a file chooser. I want to be able to display how many bytes a file > is. A file happens to be a symlink. How do I do that with a broken _wstat? Switching _wstat to _stat will only make the problem worse. -- < Jernej Simončič ><><><><>< http://eternallybored.org/ > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
Switching _wstat to _stat will only make the problem worse. How can returning an error (which the application can trap, and possibly take remedial action of its own - for example trying _wstat itself), *EVER* be worse than a function silently succeeding with a bogus return value? ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 2011-09-26 at 13:55 +0200, Jernej Simončič wrote: > On Mon, 26 Sep 2011 08:59:29 +0200, Kean Johnston wrote: > > > So, for stat, we have to do a double conversion: first from the input UTF-8 > > to UTF-16, and then from UTF-16 to MBCS so that we can call stat instead of > > wstat so that users don't get any nasty surprises when using g_stat() on a > > symlink in Vista. *le sigh* > > Won't that fail on filenames that have characters outside of the active > codepage instead? Couldn't GLib directly use GetFileAttributesExW instead, or does this have similar issues? It would require a few extra lines of code to fill in the stat structure, but if it can properly handle arbitrary filenames and symbolic links, it's probably worth it. Jürg ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
Couldn't GLib directly use GetFileAttributesExW instead, or does this Sadly the file attributes constants dont have anything like FILE_ATTRIBUTE_SYMLINK etc, so we wouldn't be able to determine if the file is a symbolic link or not. Which I guess would be OK because this is g_stat() not g_lstat(). I guess what we could do (if GetFileAttributesExW works correctly on symbolic links) is fill in the basic stat structure with _wstat() and then just change the size field with whatever GetFileAttributesExW reports). ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 26 Sep 2011 16:12:25 +0200, Kean Johnston wrote: > That could be weirdness with the mklink command. I'll write a test using > the actual API's. It's not - symlinks are intentionally restricted to administrators to prevent the security issues they would introduce in applications predating symlink support. -- < Jernej Simončič ><><><><>< http://eternallybored.org/ > ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
It's not - symlinks are intentionally restricted to administrators to prevent the security issues they would introduce in applications predating symlink support. Ah ok cool, useful information, thank you! ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 2011-09-26 at 19:11 +0200, Kean Johnston wrote: > > Couldn't GLib directly use GetFileAttributesExW instead, or does this > Sadly the file attributes constants dont have anything like > FILE_ATTRIBUTE_SYMLINK etc, so we wouldn't be able to determine if the file > is a symbolic link or not. Which I guess would be OK because this is > g_stat() not g_lstat(). I guess what we could do (if GetFileAttributesExW > works correctly on symbolic links) is fill in the basic stat structure with > _wstat() and then just change the size field with whatever > GetFileAttributesExW reports). It should be possible to get that information from FindFirstFileEx, as far as I can tell. However, this should only be done for g_lstat and not g_stat, right? Jürg ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
FWIW I'm in favor of 64 bit time fields, as the event horizon for 32-bit fields is only 27 years away. Even if you think that it's very unlikely that any present day programs will still be used in 27 years, you can think of this as enabling Unix timestamp calculations with dates much further into the future than possible now. Any of the other heavy-hitters in glib want to chime in on the OTHER stat stuff other than the Windows portion? The bit on generalizing it and making it no longer be sensitive to _FILE_OFFSET_BITS? Matthias? Emmanuele? Ryan? Others? ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On 09/26/2011 08:59 AM, Kean Johnston wrote: Here's how I would define the GStatBuf data type: Won't you break ABI if you're changing the layout of the struct on linux/unixes? As I understand this is not an issue on Windows since everyone ships the libraries with the binary, but it is on Linux. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On 9/27/2011 9:08 AM, Steve Frécinaux wrote: On 09/26/2011 08:59 AM, Kean Johnston wrote: Here's how I would define the GStatBuf data type: Won't you break ABI if you're changing the layout of the struct on linux/unixes? As I understand this is not an issue on Windows since everyone ships the libraries with the binary, but it is on Linux. Nope. On linux there IS no ABI to break - everything is #define'd to its libc name - so there is no real g_stat(). Any existing code out there will just be calling stat(). If we make this change however we will be introducing an ABI that will need to be maintained which is why its important everyone agrees the data types are wide enough. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Tue, 2011-09-27 at 11:36 +0200, Kean Johnston wrote:
> On 9/27/2011 9:08 AM, Steve Frécinaux wrote:
> > Won't you break ABI if you're changing the layout of the struct on
> > linux/unixes? As I understand this is not an issue on Windows since
> > everyone ships the libraries with the binary, but it is on Linux.
I was just about to write a reply mentioning this -- glad someone else
brought it up. :)
I disagree that we don't have an ABI to maintain on Windows. I think
people on Windows are somewhat likely to download precompiled binary
versions of our DLLs for use with developing their software (since the
build process is so complicated). We can easily introduce extremely
difficult-to-debug situations for people who assumed that the DLL was
binary-compatible with the old one.
> Nope. On linux there IS no ABI to break - everything is #define'd to its
> libc name - so there is no real g_stat(). Any existing code out there will
> just be calling stat(). If we make this change however we will be
> introducing an ABI that will need to be maintained which is why its
> important everyone agrees the data types are wide enough.
While I mostly agree with this, it's only true in the case that both the
code calling g_stat() and the code inspecting its result are always in
the same codebase.
If there is any API that takes a 'GStatBuf *' then that ABI will change
as a result of recompiling against the new GLib (ie: recompiling a
library with no code changes will change the ABI of the library). I'm
not sure there are any cases of this, but it's something to be aware of.
A more practical concern, however, is the fact that GStatBuf was only
introduced somewhat recently:
http://git.gnome.org/browse/glib/commit/?id=1229281d95802c4c190284c7d331f67194a2553e
This means that there is an awful lot of valid existing code (including
within GLib itself) that looks like this:
{
struct stat buf;
g_stat (filename, &buf);
}
which (if I understand your proposal correctly) would be badly broken.
Cheers
___
gtk-devel-list mailing list
gtk-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
I disagree that we don't have an ABI to maintain on Windows. I think
people on Windows are somewhat likely to download precompiled binary
versions of our DLLs for use with developing their software (since the
build process is so complicated). We can easily introduce extremely
difficult-to-debug situations for people who assumed that the DLL was
binary-compatible with the old one.
This is true. On Windows the code is actually fairly well insulated from
random size changes based on macros. The only problem is Tor chose the
"wrong" stat structure IMHO. These days, files > 4GB are common. Since GLib
is a platform tool and you would rightfully expect to be able to write,
say, an archival tool that could compress big files, and this currently
isn't possible. Or you may want to write a backup tool and preserve
timestamps. Also not currently possible. Of course said applications could
just use other API's or specify their own stat structures and not use
g_stat() at all but in that case what is the purpose of having g_stat() at
all? Or any of the other gstdio.h wrappers. The purpose (I believe) is to
ensure that if you use those wrappers they will behave the same way on all
platforms GLib has been ported to. This is largely the case but it does
break down on the fringes.
While I mostly agree with this, it's only true in the case that both the
code calling g_stat() and the code inspecting its result are always in
the same codebase.
And therein lies the EXACT reason why having a well defined stat structure
with data types wide enough to cover all cases is such a requirement.
library with no code changes will change the ABI of the library). I'm
not sure there are any cases of this, but it's something to be aware of.
Certainly, and we could do some rather trivial things to insulate against
that. Call the structure something else (although as you mentioned GStatBuf
is sufficiently new that I don't think we'd have a problem). Or announce
the breakage prominently. We can work around the ABI change.
This means that there is an awful lot of valid existing code (including
within GLib itself) that looks like this:
{
struct stat buf;
g_stat (filename,&buf);
}
which (if I understand your proposal correctly) would be badly broken.
That code would of course change to
{
GStatBuf buf;
g_stat (filename, &buf);
}
The code wouldn't be broken at all. In fact it would be less broken. If,
for example, GLib wasn't compiled with _FILE_OFFSET_BITS=64 then
internally, all of its usage of g_stat() can only deal with 31-bit file
sizes. User code using GLib compiled WITH that set will support file sizes
with 63 bits.
Almost all of the functions currently "wrapped" in gstdio.h are problematic
with LFS. On Linux they are currently just macros. Changing those to
functions won't break any existing code on Linux because those symbols
aren't even in libglib-2.0.so. But in order to provide a consistent,
doesn't-change-with-macros interface that can *become* the GLib ABI is
useful. That code can be constructed inside GLib such that it is always
compiled with LFS in mind. For example, we can ensure that g_open() always
calls open64 or whatever it's called on the system in question. By rigidly
defining GStatBuf to use identically sized on all platforms fields, we make
g_stat() more useful. Heck, it even becomes possible to share binary dumps
of the thing on like-endian machines should you want to do that.
In a nutshell, if gstdio.[ch] were slightly tweaked to be actual functions
and not veneer macro wrappers, and they all took suitably sized arguments
then the code becomes that much more portable and easier to debug and less
surprising. I'd also add the various seek functions as they too are
problematic because they take a file offset but that also doesn't break any
ABI it just adds a new one. I think the *only* platform affected by the
changes I am proposing is Windows (or any UNIX system that defines
G_STDIO_NO_WRAP_ON_UNIX) and that only for g_stat() and that can be easily
worked around. But at the end of it we will have a completely consistent
API across all platforms. The *only* thorny question really, is what width
do we make the st_?time fields? 32-bit or 64-bit (or, as on MacOS, both).
And if we make them 64-bit what exactly does that represent? Nanoseconds
since the Epoch? I think the easiest way by far is to have those fields
defined thus in the structure:
gint32 st_atime, st_mtime, st_ctime;
gint64 st_atime64, st_mtime64, st_ctime64;
That supports the vast majority of the code out there that is UNIX-centric
and supports the notion of a 31-bit time field measured as seconds since
the epoch of Jan 1 1970. But code that wants higher precision can use the
64-bit variants on systems that provide it and simply multiply out the
32-bit ones to give usable values on those that don't.
Note that on Windows the 64-bit time fields are just more seconds since the
Epo
Re: _wstat on Windows (actually stat stuff in general)
On Mon, 2011-09-26 at 08:59 +0200, Kean Johnston wrote:
> There is also a broader issue with stat. The size of struct stat can vary
> according to what macros are defined when you include . If you
> have _FILE_OFFSET_BITS=64 defined your size fields are 64-bit, otherwise
> they *can* be 64-bit but are most often 32-bit. Thus, if glib was compiled
> without _FILE_OFFSET_BITS defined, and an application uses g_stat() but
> DOES have it defined, you're in for trouble (and the inverse is true - if
> glib was compiled with _FILE_OFFSET_BITS=64 and the app wasn't, the same
> hilarity ensues).
I don't see how this would be any problem. Both the app and the library
will get the struct stat that matches what they were compiled with. The
only problem would be if you passed the struct stat between glib and the
app, but that doesn't happen as g_stat is a macro.
> On almost all UNIX systems time_t is a signed 32-bit int thus capable of
> representing time from the epoch through some time in 2038. Windows has
> support for 64-bit time fields which is surprisingly forward thinking of them.
time_t is typically a signed long, which is 64bit on all 64bit linux
versions.
> Since GLib is a "platform library" it would be extremely useful if
> gstsdio.h defined a GLib version of the stat structures instead of just
> doing "typedef struct stat GStatBuf". That structure can be consistently
> defined, with sufficiently large data types that it will work on all
> systems. Despire the coolness of Windows supporting time fields 64-bits
> wide for the least amount of pain it would probably be best if the time
> fields were left at 32-bits (although it would be really cool and forward
> thinking if we used 64-bits), but to use 64-bit fields for all size fields,
> and 32-bit fields for all others. That way gstdio.[ch] can be compiled in a
> very specific way to get the information needed and then smush it into that
> portable structure. Here's how I would define the GStatBuf data type:
>
> typedef struct GStatBuf {
>guint32 st_dev;
>guint64 st_ino;
>guint16 st_mode;
>guint16 st_nlink;
>guint32 st_uid;
>guint32 st_gid;
>guint32 st_rdev;
>gint64 st_size;
>gint32 st_atime;
>gint32 st_mtime;
>gint32 st_ctime;
> } GStatBuf;
I don't see what advantages this gives you. It will break all existing
code that uses g_stat and struct stat, you can't share GStatBuf with
other struct stat using code, you can't access extensions like
nanosecond mtimes, it doesn't have things like blocksize and st_blocks
which means you can't calculate actual space used.
Also, we *do* have an abstraction that lets you write portable code. Its
called GFileInfo. g_stat is by its nature a lowlevel unix-like call, it
was added mainly as a quick way to port unix code to windows. So,
introducing our own copy of struct stat on windows might make sense
(with its c runtime library issues), but we don't want to change it on
unix.
___
gtk-devel-list mailing list
gtk-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Wed, 2011-09-28 at 10:00 +0200, Kean Johnston wrote:
>
> > This means that there is an awful lot of valid existing code (including
> > within GLib itself) that looks like this:
> >
> > {
> >struct stat buf;
> >
> >g_stat (filename,&buf);
> > }
> >
> > which (if I understand your proposal correctly) would be badly broken.
> That code would of course change to
> {
>GStatBuf buf;
>
>g_stat (filename, &buf);
> }
It would be a break in source-compat, which we can't just do. Glib is an
API stable library used by thousands of other libraries and apps.
> The code wouldn't be broken at all. In fact it would be less broken. If,
> for example, GLib wasn't compiled with _FILE_OFFSET_BITS=64 then
> internally, all of its usage of g_stat() can only deal with 31-bit file
> sizes. User code using GLib compiled WITH that set will support file sizes
> with 63 bits.
I don't understand why you keep bringing this up. glib has
AC_SYS_LARGEFILE in configure.ac, so it will never be built with
anything but 64bits internally.
> Almost all of the functions currently "wrapped" in gstdio.h are problematic
> with LFS. On Linux they are currently just macros.
They are problematic due to LFS being problematic, but that is just how
unix works. If you want to work with other libraries etc on the lower
level you have to be aware of this. If you don't want that, use the
proper glib i/o abstractions in GIO that are not using anything like
struct stat.
> That supports the vast majority of the code out there that is UNIX-centric
> and supports the notion of a 31-bit time field measured as seconds since
> the epoch of Jan 1 1970. But code that wants higher precision can use the
> 64-bit variants on systems that provide it and simply multiply out the
> 32-bit ones to give usable values on those that don't.
All 64bit linux arches uses 64bit signed time_t, and its in seconds.
Higher precision timestamps are done in linux by introducing separate
nanosecond fields in struct stat. This is all transparently handled by
GIO if you just use GFileInfo.
> Note that on Windows the 64-bit time fields are just more seconds since the
> Epoch, with an upper limit of Dec 31 23:59:59 3000. If we decide that the
> 64-bit time field is really nanoseconds since the Epoch (a much more usable
> value IMHO) then it can represent dates up to some time in the year 2262. I
> don't think it matters what it represents as long as we define it. If GLib
> only breaks in the year 2262 I'm quite Ok with that :-)
Of course we can't just make time_t be nanoseconds, that would break a
lot of other functions that expect it to be in seconds.
___
gtk-devel-list mailing list
gtk-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
time_t is typically a signed long, which is 64bit on all 64bit linux versions. Even more good reason to standardize then. I don't see what advantages this gives you. It will break all existing code that uses g_stat and struct stat, you can't share GStatBuf with other struct stat using code, you can't access extensions like nanosecond mtimes, it doesn't have things like blocksize and st_blocks which means you can't calculate actual space used. So easily solved. Call the damned thing g_statfile() and have the structure be GFileStat. Something else. The Windows stat structure doesn't have block size and st_blocks anyway so again your code needs to become poluted with ifdefs. We can add the block size and number of blocks, and on Windows we can either find another API that gives us the same information of set the block size == file size and the number of blocks to 1 so that the code presents a uniform interface. It is *EXACTLY* because of all of these differences that having a wrapper is useful. Also, we *do* have an abstraction that lets you write portable code. Its called GFileInfo. g_stat is by its nature a lowlevel unix-like call, it was added mainly as a quick way to port unix code to windows. So, introducing our own copy of struct stat on windows might make sense (with its c runtime library issues), but we don't want to change it on unix. Only if you define "UNIX" as "Linux". OSX has other fields Linux doesn't. Some UNIX variants have 16-bit uid_t's others have 32. Some have as low as 16-bit ino_t's others have 64. There are all KINDS of ways in which it differs. Offering a portable, no-ifdefs-required, suitably-large-sized-to-accomodate-everyone structure ... yes *STRUCTURE* that all code can use completely portably without having to worry about anything ... SURELY you can see the value in that? GFileInfo from GIO? You have GOT to be kidding me? As a replacement for stat()? When I want to look up a file attribute I have to go through hash table lookups for attributes and a completely open-ended size (GArray *attributes) and all that parent class and instance overhead - versus having a single structure I can sizeof() and write to a file? In what universe is that a better approach? GLib does plenty of abstractions of things that you could just use the native API for. Like g_malloc(). You could just use malloc(). But there are variances on how malloc() behaves so you wrap it to get consistent behaviour so that everywhere you use g_malloc(0) you know exactly what you are getting. You wrap printf because not all printf's are alike. stat() is just the same. It needs wrapping because not all stats are the same. It can be trivially done. It can be done in such a way that it doesn't break any ABIs or even API's. What's the problem? ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
Le mercredi 28 septembre 2011 à 11:03 +0200, Kean Johnston a écrit : > Only if you define "UNIX" as "Linux". OSX has other fields Linux doesn't. > Some UNIX variants have 16-bit uid_t's others have 32. Some have as low as > 16-bit ino_t's others have 64. There are all KINDS of ways in which it > differs. Offering a portable, no-ifdefs-required, > suitably-large-sized-to-accomodate-everyone structure ... yes *STRUCTURE* > that all code can use completely portably without having to worry about > anything ... SURELY you can see the value in that? GFileInfo from GIO? You > have GOT to be kidding me? As a replacement for stat()? When I want to look > up a file attribute I have to go through hash table lookups for attributes > and a completely open-ended size (GArray *attributes) and all that parent > class and instance overhead - versus having a single structure I can > sizeof() and write to a file? In what universe is that a better approach? Do you have a use case where hash-table lookups would be a bottleneck? With dual-core CPUs we have nowadays, disk access is likely to be much slower than the hash-table work that GIO produces. And few programs would need to stat that many files anyway. Cheers ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
Do you have a use case where hash-table lookups would be a bottleneck? Small mobile devices. Devices with only 64MB of RAM to play with. Embedded devices. Just because GLib is used mainly in GNOME don't make the assumption its the ONLY place. Not every deployment of GLib applications is on a multi-core CPU system with multi-gigs of memory, or even with traditional hard disks. Or even hard disks at all. Places where you are using GLib becuase its relatively small, not using the massive overhead that is GIO and which serves a different purpose entirely. Just becuase GLib and GIO come in the same tar does NOT mean they are always deployed side by side. With dual-core CPUs we have nowadays, disk access is likely to be much slower than the hash-table work that GIO produces. And few programs would need to stat that many files anyway. Really? On what do you base that information? There are three applications I want to write that even brought me to the GLib / Gtk+ world and ALL of them use stat because they are dealing with files. The bottom line is, although stat may be a low level system call, applications deal with files. A LOT. A HELL of a lot. stat is a basic file operation. It can be made to be more portable. That's all I am trying to do. It can abstract 95% of all of the problem areas. If you have an application that needs very specific, tied-to-one-OS stat structures then by all means use stat. But when a *Platform* library isn't abstracting a basic *platform* call, that's a problem and I don't understand all the resistance to it. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Wed, Sep 28, 2011 at 10:27 AM, Kean Johnston wrote: >> Do you have a use case where hash-table lookups would be a bottleneck? > > Small mobile devices. Devices with only 64MB of RAM to play with. Embedded > devices. Just because GLib is used mainly in GNOME don't make the assumption > its the ONLY place. Not every deployment of GLib applications is on a > multi-core CPU system with multi-gigs of memory, or even with traditional > hard disks. Or even hard disks at all. Places where you are using GLib > becuase its relatively small, not using the massive overhead that is GIO and > which serves a different purpose entirely. Just becuase GLib and GIO come in > the same tar does NOT mean they are always deployed side by side. Writing embedded code and writing code that is portable between three major desktop OSes are essentially orthogonal concerns. Scripting languages such as Python use hash tables for all their namespace lookups and as a common data structure - it's not a huge performance concern. >> With dual-core CPUs we have nowadays, disk access is likely to be much >> slower than the hash-table work that GIO produces. And few programs >> would need to stat that many files anyway. > > Really? On what do you base that information? There are three applications I > want to write that even brought me to the GLib / Gtk+ world and ALL of them > use stat because they are dealing with files. The bottom line is, although > stat may be a low level system call, applications deal with files. A LOT. A > HELL of a lot. stat is a basic file operation. It can be made to be more > portable. That's all I am trying to do. It can abstract 95% of all of the > problem areas. If you have an application that needs very specific, > tied-to-one-OS stat structures then by all means use stat. But when a > *Platform* library isn't abstracting a basic *platform* call, that's a > problem and I don't understand all the resistance to it. The problem is that stat is API from the 1970's. None of us really want to prolong the life of code with such readable names as S_IFMT, S_IFREG, st_blksize and EBADF. One of GIO's design goals was to come up with a fully portable, modern abstraction on top of all this. I for one would much rather be working with G_FILE_ATTRIBUTE_STANDARD_IS_BACKUP, G_FILE_ATTRIBUTE_TIME_ACCESS etc. GLib is a 21st century platform library, not a pile of portability hacks. I admit that for convenience it's managed to acquire a bunch of portability hacks, but this isn't a design goal. If you would like to use GIO but find measurable performance problems or missing functionality, bug reports are welcome. Sam ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
The problem is that stat is API from the 1970's. None of us really So is malloc. So is chmod. They are wrapped. want to prolong the life of code with such readable names as S_IFMT, S_IFREG, st_blksize and EBADF. One of GIO's design goals was to come up with a fully portable, modern abstraction on top of all this. I for And if your application can benefit from that, please, by all means go ahead and USE it! But why should that be used as a precluder for someone who wants to USE the smaller, more efficient, working since the 1970's API? A USEFUL tool provides choices. That's what I want to do - provide a choice. I don't want to use GIO thank you very much. Pull in a library with 173762 lines of code (aside from GObject which it depends on)) so I can use G_FILE_ATTRIBUTE_TIME_ACCESS instead of buf.st_atime? No thanks. GLib is a 21st century platform library, not a pile of portability hacks. I admit that for convenience it's managed to acquire a bunch of portability hacks, but this isn't a design goal. If you would like to I want to add one more portability hack. It serves a need and doesn't impact anyone who isn't interested in using it. I am trying to improve Glib. Not GIO, but Glib. I want to use GLib. Not GObject and libFFI, not Gthread, not GIO. GLib. It has its own headers and its own library. There is one function I want to make more portable, and 2 I want to add because they are missing (seek and tell). If the way you develop software allows you to bring in hundreds of thousands of lines of code rather than using a single structure, then please, be my guest and do so. But please for the love of all that's holy stop trying to tell me that's how *I* should be doing things. use GIO but find measurable performance problems or missing 1,155,072 jx10-gio2.dll - 'nuff said. I hear people complaining about lack of maintainers all the time. I am trying very hard to participate in the community and add value. As all newcomers to a project do I pick something small that can benefit others as well as scratch a personal itch. That's how a development community grows. Whether you realize it or not you guys are being down-right contributor hostile. I have absolutely *NO* problem making my own changes to glib and maintaining them for my own purposes. I can use my time writing software instead of debating things with you. But if you want help, and you want fresh blood and you want a developer who is trying desperately to maintain the software on the worlds most common platform and offer precious cycles to your team, then please consider that not every single change has to appeal to one audience, that there are multiple uses of GLib far divorced from the GNOME community or UNIX and that not everyone thinks as you do or uses the software the way you do. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Wed, 28 Sep 2011 at 11:03:00 +0200, Kean Johnston wrote: > Offering a portable, no-ifdefs-required, > suitably-large-sized-to-accomodate-everyone structure ... yes > *STRUCTURE* that all code can use completely portably without having > to worry about anything ... ... until GLib is ported to platform-of-the-week (Hurd? Plan 9? Windows 8?) which has interesting fields that weren't in the traditional "struct stat"; at which point it either becomes non-comprehensive on the new platforms (missing information that the new platform provides), or variable-size (#ifdefs), or an ABI break on existing platforms (GLib 4). Comprehensive, same layout on all platforms, fixed layout over time: pick any two. GFileInfo is opaque/extensible/indeterminate-size precisely so that the fact that its layout changes over time is not an ABI break. S ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Wed, 2011-09-28 at 11:03 +0200, Kean Johnston wrote: > So easily solved. Call the damned thing g_statfile() and have the structure > be GFileStat. So then we would have 'struct stat' and 'GStatBuf' which would work with g_stat(), g_lstat() and fstat(). Additionally we would add a 'GFileStat' that only works with new calls 'g_statfile()', 'g_lstatfile()' and (presumably) 'g_fstatfile()' (for fstat() functionality applied to the new structure type). My head is spinning... > Only if you define "UNIX" as "Linux". OSX has other fields Linux doesn't. > Some UNIX variants have 16-bit uid_t's others have 32. Some have as low as > 16-bit ino_t's others have 64. There are all KINDS of ways in which it > differs. Offering a portable, no-ifdefs-required, > suitably-large-sized-to-accomodate-everyone structure ... yes *STRUCTURE* > that all code can use completely portably without having to worry about > anything ... SURELY you can see the value in that? GFileInfo from GIO? You > have GOT to be kidding me? As a replacement for stat()? When I want to look > up a file attribute I have to go through hash table lookups for attributes > and a completely open-ended size (GArray *attributes) and all that parent > class and instance overhead - versus having a single structure I can > sizeof() and write to a file? In what universe is that a better approach? I don't find the ability to write 'struct stat' to a file to be a particularly compelling usecase... You propose to avoid indirection by moving people away from GFileInfo to GFileStat. At the same time, you'd be adding more indirection to the g_stat() users though -- a copy of all of the various fields of the system native structure to our shadow structure. Unless you actually propose that we keep both APIs around and undeprecated -- and my head is spinning again I don't really appreciate the problems that are caused by different sized inode, uid, etc. fields... It might be unsightly or aesthetically displeasing on some level, but the something like: if (buf.st_uid == getuid()) ... will obviously always work properly. I think, if anything, we cause ourselves more trouble by breaking with the system definition. The only benefit to your proposal (from a UNIX standpoint, at least) is that it helps avoid accidental portability problems where a non-portable field is used. I'm not sure that it's worth all the drawbacks. Cheers ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
> "_wstat does not work with Windows Vista symbolic links. In these cases,
> _wstat will always report a file size of 0. _stat does work correctly with
> symbolic links."
Ugh, how weird and unexpected. (My expectation would have been that
neither supports symbolic links.) For which C runtime is this, btw,
the system msvcrt.dll, or some of the MSVC-version-specific ones?
Anyway, I think the only sensible thing to do is then to not use
either _wstat() or _stat(). We don't want to miss the full Unicode
support, and I guess this symbolic link stuff is useful, too.
Instead, we should just use the Win32 API to fill in the meaningful
fields of a struct stat. Yeah, following symbolic links manually might
be painful; I have no idea how to do that.
(Of course, when I say "we", I actually mean "you", as I don't really
take much part in GLib/GTK+ development currently... sorry. Just
noticed this thread when browsing the gtk-devel-list archove, and
*had* to follow-up...)
Anyway, isn't g_stat() kinda deprecated? Should it be? Partly exactly
because of this horrible potential for confusion with the several
"struct stat" variants on Windows (and also on some Unixes, I guess,
although there at least there is typically only one C
library). Doesn't GIO's APIs to get file
properties/attributes/whatever they are called supersede g_stat()?
> Here's how I would define the GStatBuf data type:
>
> typedef struct GStatBuf {
>guint32 st_dev;
>guint64 st_ino;
>guint16 st_mode;
>guint16 st_nlink;
>guint32 st_uid;
>guint32 st_gid;
>guint32 st_rdev;
>gint64 st_size;
>gint32 st_atime;
>gint32 st_mtime;
>gint32 st_ctime;
> } GStatBuf;
Er, no. NO.
This definitely is pointless, GIO supersedes g_stat() etc in much
cleaner fashion.
Why on earth would we want to use 32-bit timestamps in a new API? Or
just 16-bit st_nlink?
Why would we want the extremely Unix-specific st_mode to show up in a
portability layer at all? st_mode is on Windows always just a
best-effort (or not even that good an effort) fabrication. Many
POSIXes support ACLs in file systens and such might be common for some
file systems on some POSIX platforms. Any software supposed to be
generic and good should take that into consideration anyway when
figuring out the protection of some file. A single st_mode field is
basically useless in modern portable APIs.
st_uid and st_gid are also totally Unix-specifc, of course.
> While in gstdio I'd also add the glaring-by-their-omission g_lseek,
> g_fseek, and g_ftell all of which could take a gint32 as their argument or
> return value.
That is a glaring omission only if you misunderstand the purpose of
gstdio.h. It is not supposed to be a wrapper for any file I/O related
functions. Its sole purpose (at least when it was created) is to take
care of the *file name* problems. It even says this in a comment:
/* Wrappers for C library functions that take pathname arguments. On
* Unix, the pathname is a file name as it literally is in the file
* system. On well-maintained systems with consistent users who know
* what they are doing and no exchange of files with others this would
* be a well-defined encoding, preferably UTF-8. On Windows, the
* pathname is always in UTF-8, even if that is not the on-disk
* encoding, and not the encoding accepted by the C library or Win32
* API.
*/
Beafing up gstdio.h now is too late. All newer file system portability
related stuff in GLib is in GIO, surely.
--tml
___
gtk-devel-list mailing list
gtk-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
Ugh, how weird and unexpected. (My expectation would have been that neither supports symbolic links.) For which C runtime is this, btw, the system msvcrt.dll, or some of the MSVC-version-specific ones? The MSDN documentation doesn't specify. I suspect its all of them TBH. Instead, we should just use the Win32 API to fill in the meaningful fields of a struct stat. Yeah, following symbolic links manually might be painful; I have no idea how to do that. Well I speak under correction but its my belief that The other API's follow them for you. I'm fairly certain GetFileAttributesEx() does. (Of course, when I say "we", I actually mean "you", as I don't really take much part in GLib/GTK+ development currently... sorry. Just noticed this thread when browsing the gtk-devel-list archove, and *had* to follow-up...) Glad you did :) That is a glaring omission only if you misunderstand the purpose of gstdio.h. It is not supposed to be a wrapper for any file I/O related functions. Its sole purpose (at least when it was created) is to take care of the *file name* problems. It even says this in a comment: Ah I had missed that bit. Beafing up gstdio.h now is too late. All newer file system portability related stuff in GLib is in GIO, surely. Sure but if all I want to know how big a file is, do I *really* want to link in all of that extra crud? GIO is huge. There are times when the low level functions are just plain appropriate. I've given up on the idea of a universal stat structure, but the g_stat() wrapper is still useful on Windows exactly because of the file name issue. People really do want to use stat rather than GIO. I do, and I've had private emails from others who do too. Right now though the biggest deficiency with it (aside from the symlinks and _wstat thing which needs to be addressed differently) is the actual stat structure you chose. It only supports 32-bit (well 31) file sizes and that's a real problem on Windows these days where ISO images are frequent and much larger than 2GB. I would dearly *love* to change that but that will mean breaking the ABI. Of course since everyone is off using GIO that shouldn't break too many people should it :) But that structure can be changed in a non-ABI breaking way and I hope to at least get that much past the powers that be. Kean ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
>> Ugh, how weird and unexpected. (My expectation would have been that >> neither supports symbolic links.) For which C runtime is this, btw, >> the system msvcrt.dll, or some of the MSVC-version-specific ones? > > The MSDN documentation doesn't specify. I suspect its all of them TBH. Actually, looking in the C library sources that come with MSVC2008, I see no sign of such a difference in _stat() vs. _wstat(). Both in stat.c and stat64.c (these are the files that contain the actual implementation of both the "ANSI" and wide-char versions of the stat family of functions) I see code that checks (findbuf.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) && (findbuf.dwReserved0 == IO_REPARSE_TAG_SYMLINK) . But on the other hand, as far as I can see, this code is used to make stat() (etc) work as lstat(), not to actually follow the symlink! (But just if the symlink is the last component of the path; I assume symlinks in the middle of the pathname are followed transparently by the FindFirstFile(). So I wonder which is correct, the documentation, or what the code seems to imply. Experimentation would be useful... Of course, it might be that it is actually FindFirstFileA() vs. FindFirstFileW() that exhibits some difference. > Sure but if all I want to know how big a file is, do I *really* want to link > in all of that extra crud? GIO is huge. There are times when the low level > functions are just plain appropriate. I would say, bah. Any actively maintained (or recently written) GLib-using code that doesn't use GIO by now is just being maintained or written in a misguided fashion. > People really do want to use stat rather than GIO. I do, and I've had > private emails from others who do too. Well, then you just will have to live with the various breakages;) I don't think you will be able to convince people it would be a good idea to add more bells and whistles in the form of stat() -style APIs when GLib already has a much cleaner and ABI/API stability/extendability safe (and other nice words) way to get information about files. --tml ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Thu, Sep 29, 2011 at 11:35, Tor Lillqvist wrote: > >> Sure but if all I want to know how big a file is, do I *really* want to link >> in all of that extra crud? GIO is huge. There are times when the low level >> functions are just plain appropriate. > > I would say, bah. Any actively maintained (or recently written) > GLib-using code that doesn't use GIO by now is just being maintained > or written in a misguided fashion. Tor, did I understand your position right that any program written in the speed in minds and uses Glib is written in a misguided fashion? -- Andrew W. Nosenko ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
hi; On 29 September 2011 09:57, Andrew W. Nosenko wrote: > On Thu, Sep 29, 2011 at 11:35, Tor Lillqvist wrote: >> I would say, bah. Any actively maintained (or recently written) >> GLib-using code that doesn't use GIO by now is just being maintained >> or written in a misguided fashion. > > Tor, did I understand your position right that any program written in > the speed in minds and uses Glib is written in a misguided fashion? that's a fairly creative way of misinterpreting that sentence. if GIO is measurably slower at doing I/O than a stat(), please: file bugs along with profiling data. ciao, Emmanuele. -- W: http://www.emmanuelebassi.name B: http://blogs.gnome.org/ebassi/ ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Thu, Sep 29, 2011 at 10:05:57AM +0100, Emmanuele Bassi wrote: > On 29 September 2011 09:57, Andrew W. Nosenko > wrote: > > On Thu, Sep 29, 2011 at 11:35, Tor Lillqvist wrote: > >> I would say, bah. Any actively maintained (or recently written) > >> GLib-using code that doesn't use GIO by now is just being maintained > >> or written in a misguided fashion. > > > > Tor, did I understand your position right that any program written in > > the speed in minds and uses Glib is written in a misguided fashion? > > that's a fairly creative way of misinterpreting that sentence. > > if GIO is measurably slower at doing I/O than a stat(), please: file > bugs along with profiling data. I wish GLib was the nice utility library it used to be. A utility library provides functionality people find useful; there is no agenda and consequently no need to tell people they are misguided if they just want to use the functionality they find useful and do not want use specific complex frameworks because they find them cumbersome. Yeti ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Thu, Sep 29, 2011 at 12:05, Emmanuele Bassi wrote:
> hi;
>
> On 29 September 2011 09:57, Andrew W. Nosenko
> wrote:
>> On Thu, Sep 29, 2011 at 11:35, Tor Lillqvist wrote:
>>> I would say, bah. Any actively maintained (or recently written)
>>> GLib-using code that doesn't use GIO by now is just being maintained
>>> or written in a misguided fashion.
>>
>> Tor, did I understand your position right that any program written in
>> the speed in minds and uses Glib is written in a misguided fashion?
>
> that's a fairly creative way of misinterpreting that sentence.
No, that is creative way to point author to avoid improper
generalizations and be more precise in phrases.
> if GIO is measurably slower at doing I/O than a stat(), please: file
> bugs along with profiling data.
GIO as layer is so comparable to stat() as apples comparable to
grapes, and we both know it :-)
But, unfortunatelly, all GMainLoop based stuffs (including GIO) are
indeed slow when we come to high workload if compared to kqueue based
(and I think epoll-based also) main loop implementations.
Therefore, either you use GIO, and limited to the current GMainLoop
speeds, or use something kqueue-based and have no way to GIO.
Fill Bugzilla? No needs, it's already done both years ago and quite recently:
https://bugzilla.gnome.org/show_bug.cgi?id=156048 ("epoll support
in gmain.c", 2004 year)
https://bugzilla.gnome.org/show_bug.cgi?id=657729 ("modernise
GMainLoop", 2011 year)
--
Andrew W. Nosenko
___
gtk-devel-list mailing list
gtk-devel-list@gnome.org
http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On Thu, 2011-09-29 at 12:44 +0300, Andrew W. Nosenko wrote: > On Thu, Sep 29, 2011 at 12:05, Emmanuele Bassi wrote: > > if GIO is measurably slower at doing I/O than a stat(), please: file > > bugs along with profiling data. > > GIO as layer is so comparable to stat() as apples comparable to > grapes, and we both know it :-) > > But, unfortunatelly, all GMainLoop based stuffs (including GIO) are > indeed slow when we come to high workload if compared to kqueue based > (and I think epoll-based also) main loop implementations. > > Therefore, either you use GIO, and limited to the current GMainLoop > speeds, or use something kqueue-based and have no way to GIO. The closest GIO function is g_file_query_info, and it doesn't use GMainLoop at all, as far as I can tell. Async functions use GMainLoop, but given that stat() is not async, there is no point in comparing the performance of stat() to async GIO functions. Jürg ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
if GIO is measurably slower at doing I/O than a stat(), please: file bugs along with profiling data. This is WAY off topic but still ... how can you possibly believe it is NOT slower? One system call versus: 1. Allocate arrays 2. Allocate hash tables 3. Do strdups 4. Calculate hashes 5. Involve at least 14 functions that I counted 6. Heap fragmentation How can you *POSSIBLY* expect that to be faster? The fact that you may not visually notice the speed difference because we all have the luxury of modern systems that are fast does NOT mean it is a superior solution. GIO adds 176000-odd lines of source code to an app. How is it even POSSIBLE you can question it is slower than 1 system call? If you really, honestly believe that, my faith in you (until now quite high) and in GLib and its future (until now also quite high) just plummeted through the floor. I really don't need accurate GPS measuring to know that a Bugati Veyron is faster than a Fiat Uno, any more than I need to provide you with profiling data to prove that GIO is slower than g_stat(). This over-engineering and relying on silicon to cover bad designs is what kills software in the long run. System calls in UNIX have survived 40 years for a reason. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On 29 September 2011 10:53, Kean Johnston wrote: > I really don't need accurate GPS measuring to know that a Bugati Veyron is > faster than a Fiat Uno, any more than I need to provide you with profiling > data to prove that GIO is slower than g_stat(). I can also tell you that for driving from my house to the supermarket, the Veyron won't be any faster because of other far larger sources of delays, such as the speed limit (or HDD latency). Of course GIO is slower when you look at the LoC count, the question is for the typical case is performance acceptable. If your application is opening a million files then maybe it's not typical. Ross ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
Of course GIO is slower when you look at the LoC count, the question is for the typical case is performance acceptable. If your application is opening a million files then maybe it's not typical. Maybe my application runs on a router and not on an 64-core Core i5000 with 16 petabytes of RAM. Just as Im not sure I'd drive a Bugati Veyron to the supermarket for a pint of milk, I'm not sure I'd use a Core i7 for a router than costs $150. If the "general utility" part of GLib extends to requiring massively more complex resources than alternatives, that pretty much seals GLib's fate as a "general purpose" anything and becomes instead a "very specific to the GNOME desktop" purpose library. No way you can convince me otherwise I'm afraid, and that's not because I'm being stubborn, it's because I (and I think you) know I'm right. GIO is appropriate for some applications, of that I have no doubt, but trying to convince me that it's a viable alternative to stat() when all I want it the damned file size? Never gonna happen. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
On 29 September 2011 11:13, Kean Johnston wrote: > No way you can convince me > otherwise I'm afraid, and that's not because I'm being stubborn, it's > because I (and I think you) know I'm right. GIO is appropriate for some > applications, of that I have no doubt, but trying to convince me that it's a > viable alternative to stat() when all I want it the damned file size? Never > gonna happen. If you are happy limiting yourself to a single platform and don't need some the magic that comes with GIO, feel free to use stat(). It's not like GIO is going to break that for you. Ross ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
If you are happy limiting yourself to a single platform and don't need some the magic that comes with GIO, feel free to use stat(). It's not UNIXWARE, SCO OpenServer, AIX, Solaris, MacOS X, Windows, Linux, FreeBSD, OpenBSD, NetBSD, Minix, HP-UX ... did I miss any platforms? I'll consider myself limited. ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: _wstat on Windows (actually stat stuff in general)
Am 29.09.2011 12:24, schrieb Kean Johnston: If you are happy limiting yourself to a single platform and don't need some the magic that comes with GIO, feel free to use stat(). It's not UNIXWARE, SCO OpenServer, AIX, Solaris, MacOS X, Windows, Linux, FreeBSD, OpenBSD, NetBSD, Minix, HP-UX ... did I miss any platforms? I'll consider myself limited. How about a test case that first times stat recursively over a large folder and then the same with g_file_query_info. If you could run that on some typical hardware you're working with, I think that should be quite interesting. Personally I regularly talk to people who use my code on "obscure" systems I don't have, maybe never heard of. And I really need practical examples of what they are doing and how it's performing. It doesn't help to emphasize your point if there is no way for the other person to understand it. ciao, Christian ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
Re: EXTERNAL: Re: _wstat on Windows (actually stat stuff in general)
On 9/29/2011 6:24 AM, Kean Johnston wrote: If you are happy limiting yourself to a single platform and don't need some the magic that comes with GIO, feel free to use stat(). It's not UNIXWARE, SCO OpenServer, AIX, Solaris, MacOS X, Windows, Linux, FreeBSD, OpenBSD, NetBSD, Minix, HP-UX ... did I miss any platforms? I'll consider myself limited. What about VAX/VMS :-) Damon Register ___ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list
