RE: ANN: H98 FFI Addendum 1.0, Release Candidate 13
I have put RC 14 at http://www.cse.unsw.edu.au/~chak/haskell/ffi/ including all the feedback on RC13. Please especially have a look at Section 6.3 (Section CString), where some of the wording changed. The spec is silent on how exactly a Haskell Char is translated to a CWchar, and there aren't any conversion functions ala castCharToCCHar / castCCharToChar. So presumably the expected behaviour is that the implementation does its best to translate between Unicode Char and whatever encoding the prevailing C library is using for wchar. Any sensible implementation will be using Unicode for wchar too, so the translation will be a simple no-op, but the C standard doesn't specify this. Older systems will probably have a locale-dependent encoding for wchar. The GNU C library has a slight bug in this regard, too (see previous discussion). I expect that when we implement the CWString operations for GHC we won't bother with any locale-dependent translations, so the implementation will only work on sensible systems. There is a fair bit that is non-obvious here, so I feel the spec ought to say something. Cheers, Simon ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
On Wed, Nov 05, 2003 at 07:24:03PM +1100, Manuel M T Chakravarty wrote: Do you intend to wait until an implementation of this is committed to CVS before finalizing the spec? (I hope so.) We already have a reference implementation (by John Meacham), which is available online. I am not sure whether having it in CVS is going to add much. I think it will improve the chances of catching problems in the details, which are a special risk when adding sizeable things late in the process. Looking through that implementation, I note (in relation to Simon's question) that the CWString functions are only provided if the C compiler defines __STDC_ISO_10646__ (so the conversion is fromIntegral). Should the spec say that? If not, and the conversion is possibly locate-dependent, you'd need a counterpart of charIsRepresentable for the wide representation. You have the bit about unrepresentable Chars becoming '?' in the legacy byte string part. I think it belongs in locale-dependent CString part. (I believe the single byte conversion discards all but the lower 8 bits, though there's probably no need to say that.) Other minor things (about old stuff): * I think the description of newCString, etc needs to say that the memory so allocated may be released using MarshalAlloc.free. * CString is said to be NUL-terminated, but presumably CStringLen is not assumed to have a NUL. Perhaps this would be clarified by defining it as (Ptr CChar, Int) instead of (CString, Int) and similarly for CWString. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
Simon Marlow [EMAIL PROTECTED] wrote, I have put RC 14 at http://www.cse.unsw.edu.au/~chak/haskell/ffi/ including all the feedback on RC13. Please especially have a look at Section 6.3 (Section CString), where some of the wording changed. The spec is silent on how exactly a Haskell Char is translated to a CWchar, and there aren't any conversion functions ala castCharToCCHar / castCCharToChar. Hmm, should we maybe have a `castCharToCwchar' and `castCwcharToChar'? So presumably the expected behaviour is that the implementation does its best to translate between Unicode Char and whatever encoding the prevailing C library is using for wchar. Any sensible implementation will be using Unicode for wchar too, so the translation will be a simple no-op, but the C standard doesn't specify this. Older systems will probably have a locale-dependent encoding for wchar. The GNU C library has a slight bug in this regard, too (see previous discussion). I expect that when we implement the CWString operations for GHC we won't bother with any locale-dependent translations, so the implementation will only work on sensible systems. There is a fair bit that is non-obvious here, so I feel the spec ought to say something. Yes, I agree. The question is, what do we actually want for the standard? Do we want to restrict the standard to only work for sensible systems? If so, what is the proper phrase to identify sensible systems? Cheers, Manuel ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
Alastair Reid [EMAIL PROTECTED] wrote, I wonder how this discrepancy between the libraries and the report arose. For what it's worth, the library has been that way since June 28, 2001: http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/libraries/base/Foreign/ Storable.hs?rev=1.1content-type=text/x-cvsweb-markup http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/libraries/base/GHC/ Storable.lhs?rev=1.1content-type=text/x-cvsweb-markup This seems to be copied from Marcin's QForeign which had the same type from Jan 2, 2001 - March 18, 2001 (i.e., the entire lifetime of the library) http://cvs.sourceforge.net/viewcvs.py/qforeign/qforeign/lib/ QStorable.hsc?rev=1.7view=markup Did I simply make a mistake when writting the report (then, this would qualify as an error in the report anyway)? On this evidence, it looks like it's an error in the report. I agree. I am changing the report. Thanks for the forensics. And thanks to Sven for spotting the mismatch. Cheers, Manuel ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
Ross Paterson [EMAIL PROTECTED] wrote, In the sentence The marshalling takes the current Unicode encoding on the Haskell side into account. (which seems to have been there before), current seems wrong, since the Haskell side is constant. How about something like The marshalling converts each Haskell character, representing a Unicode code point, to one or more bytes in a manner determined by the current locale. and dropping the later sentence about the locale. Done. It might be worth emphasizing that the Len is the number of bytes rather than Chars. Done. In the part about the single-byte versions, it might be worth tightening the warning to say that these preserve only the first 256 values of Char. Ok. (That is the Latin-1 subset, so calling them ASCII seems a misnomer.) Yes, but we can't call it Latin-1 either, because that implies a locale (there is also Latin-2 etc). Manuel ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
John Meacham [EMAIL PROTECTED] wrote, On Fri, Oct 31, 2003 at 12:32:55PM +, Ross Paterson wrote: Making the Right Thing the default, though it may cost more, seems appropriate. In the sentence The marshalling takes the current Unicode encoding on the Haskell side into account. (which seems to have been there before), current seems wrong, since the Haskell side is constant. How about something like The marshalling converts each Haskell character, representing a Unicode code point, to one or more bytes in a manner determined by the current locale. and dropping the later sentence about the locale. This sounds good to me. I also might reword the paragraph introducing the 8bit versions, as the efficiency reason for using them is less important than the API one. meaning that some C APIs specify that a localized string should be passed, while others explicitly don't use localization and only expect ASCII (or another specific encoding such as utf8) strings and this is most likely what will determine the choice of string marshalers. True. I changed that. What happens if one attempts to convert a Char that has no encoding in the current locale? my implementation converts unrepresentable characters to '?'. But a case could be made for throwing a CharsetConversion exception of some sort or simply eliding invalid characters. I am not sure what is best, I chose the '?' route because it matches what happens when you don't have a font installed and get a replacement character and is less troublesome for the user. I agree. I now documented to conversion to '?'. Cheers, Manuel ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
I have put RC 14 at http://www.cse.unsw.edu.au/~chak/haskell/ffi/ including all the feedback on RC13. Please especially have a look at Section 6.3 (Section CString), where some of the wording changed. Cheers, Manuel ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
W licie z sob, 01-11-2003, godz. 02:16, John Meacham pisze: my implementation converts unrepresentable characters to '?'. But a case could be made for throwing a CharsetConversion exception of some sort or simply eliding invalid characters. I am not sure what is best, It depends on the application: - When displaying email on screen, it's best to transliterate by removing accents etc. - When communicating with a database, it's best to report an error so configuration can be fixed (perhaps charsets are wrongly set). - When converting HTML, it's best to provide a hook to replace unrepresentable characters with #; -- __( Marcin Kowalczyk \__/ [EMAIL PROTECTED] ^^ http://qrnik.knm.org.pl/~qrczak/ ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
On Sun, Nov 02, 2003 at 09:31:30PM +1100, Manuel M T Chakravarty wrote: Ross Paterson [EMAIL PROTECTED] wrote, (That is the Latin-1 subset, so calling them ASCII seems a misnomer.) Yes, but we can't call it Latin-1 either, because that implies a locale (there is also Latin-2 etc). CByteString? Do you intend to wait until an implementation of this is committed to CVS before finalizing the spec? (I hope so.) Incidentally, the spec says CWChar but the libraries say CWchar. It seems they always disagreed. (If the libs change, maybe CPtrdiff and CFpos should be CPtrDiff and CFPos too.) ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
I wonder how this discrepancy between the libraries and the report arose. For what it's worth, the library has been that way since June 28, 2001: http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/libraries/base/Foreign/ Storable.hs?rev=1.1content-type=text/x-cvsweb-markup http://cvs.haskell.org/cgi-bin/cvsweb.cgi/fptools/libraries/base/GHC/ Storable.lhs?rev=1.1content-type=text/x-cvsweb-markup This seems to be copied from Marcin's QForeign which had the same type from Jan 2, 2001 - March 18, 2001 (i.e., the entire lifetime of the library) http://cvs.sourceforge.net/viewcvs.py/qforeign/qforeign/lib/ QStorable.hsc?rev=1.7view=markup Did I simply make a mistake when writting the report (then, this would qualify as an error in the report anyway)? On this evidence, it looks like it's an error in the report. The type is certainly consistent with the type of plusPtr if you think about implementing peekByteOff like this: peekByteOff p o = peek (plusPtr p o) And, since peekByteOff is primarily for use when peekElemOff doesn't work (i.e., for structs), it makes sense to change the type. -- Alastair Reid ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
On Fri, Oct 31, 2003 at 06:45:41PM +1100, Manuel M T Chakravarty wrote: (1) The addition of a variant of foreign finalizers that take an extra environment argument that facilitates the emulation of closures in C. This adds two functions to `ForeignPtr'. Regarding type FinalizerEnvPtr env a = FunPtr (Ptr env - Ptr a - IO ()) newForeignPtrEnv :: FinalizerEnvPtr env a - Ptr env - Ptr a - IO (ForeignPtr a) addForeignPtrFinalizerEnv :: FinalizerEnvPtr env a - Ptr env - ForeignPtr a - IO () Doesn't this mean that the env must be either static storage (gaining nothing) or dynamic storage that cannot be reclaimed (which brings us back to where we started)? ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
I think it's ok for the finalizer function to delete the environment object if it wants isn't it? This isn't always the right thing to do but I think it is in the common case that there is a unique environment object for every finalized object. In the other common case that there is a single environment object for a set of objects (e.g., for all objects of the same 'type'), the finalizer might decrement a reference count on the environment object. Have I missed some subtlety? -- Alastair Reid Regarding type FinalizerEnvPtr env a = FunPtr (Ptr env - Ptr a - IO ()) newForeignPtrEnv :: FinalizerEnvPtr env a - Ptr env - Ptr a - IO (ForeignPtr a) addForeignPtrFinalizerEnv :: FinalizerEnvPtr env a - Ptr env - ForeignPtr a - IO () Doesn't this mean that the env must be either static storage (gaining nothing) or dynamic storage that cannot be reclaimed (which brings us back to where we started)? ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
Manuel M T Chakravarty wrote: [...] For the second time...I consider Version 1.0 of the FFI Addendum to be completed; no changes except linguistic ones and plain error corrections will be accepted anymore for this version. Never say never... The signatures for peekElemOff and pokeElemOff in the report are less general than the ones in fptools/libraries/base: peekElemOff :: Storable a = Ptr a - Int - IO a vs. peekElemOff :: Storable a = Ptr b - Int - IO a (same for pokeElemOff). The latter is more flexible and is more in line with the signatures of plusPtr/minusPtr. Furthermore, changing such a basic function would probably break quite a lot of code, so I propose to change the report, not the libraries. Cheers, S. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
[As I was about to send this, it occured to me that maybe you had made a typo and meant to write 'peekByteOff' instead of 'peekElemOff'? This seems less dodgy since peekByteOff is more usually used to access elements of structs where different elements have different types.] I think the report is right and the libraries should be fixed even if a lot of code has to be changed to make the two agree. With the report version: 1) Simple programming errors are readily caught. 2) Less type information needs to be written (obviously at the cost of not catching the type errors in (1).) That is, if you specify the type of the value being peeked, you can infer the type of the pointer and vice-versa. 3) If you want to do something funny, you have to explicitly cast the pointer to make it happen which alerts readers to the fact that you're doing something funny. To be honest, I can't think of many cases where I would want the types you propose. peekElemOff is basically for arrays where all elements have the same type - why would you want to treat them as having different types? -- Alastair Reid Never say never... The signatures for peekElemOff and pokeElemOff in the report are less general than the ones in fptools/libraries/base: peekElemOff :: Storable a = Ptr a - Int - IO a vs. peekElemOff :: Storable a = Ptr b - Int - IO a (same for pokeElemOff). The latter is more flexible and is more in line with the signatures of plusPtr/minusPtr. Furthermore, changing such a basic function would probably break quite a lot of code, so I propose to change the report, not the libraries. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: ANN: H98 FFI Addendum 1.0, Release Candidate 13
Alastair Reid wrote:
[As I was about to send this, it occured to me that maybe you had made a typo
and meant to write 'peekByteOff' instead of 'peekElemOff'? This seems less
dodgy since peekByteOff is more usually used to access elements of structs
where different elements have different types.]
*aaargl* Of course, sorry for the confusion caused by my typo... I fully
agree with your arguments for {peek,poke}ElemOff, but as you already
mentioned above, the typical scenario for {peek,poke}ByteOff is access to
structs, so the more general type is much more useful. It could be argued
that this is a cast via the backdoor, but that would hold for {plus,minus}Ptr,
too.
Cheers,
S.
___
FFI mailing list
[EMAIL PROTECTED]
http://www.haskell.org/mailman/listinfo/ffi
