Re: Question: Unicode and Filetools

2006-09-15 Thread Andre Poenitz
On Fri, Sep 15, 2006 at 12:02:05PM +0200, Abdelrazak Younes wrote: > Georg Baum wrote: > >Abdelrazak Younes wrote: > > > >>If only Lars and Georg work on this, I am afraid that it will take some > >>time before all the corner cases are worked out step by step. > > > >I don't think that I am going t

Re: Question: Unicode and Filetools

2006-09-15 Thread Abdelrazak Younes
Georg Baum wrote: Abdelrazak Younes wrote: Well, I hope you don't include me in this group because I did some work on the unicode front. No, I do not include you. Good... no... I can resist! I disagree with you about how the uncide things should go, but I don't want to force you to do thi

Re: Question: Unicode and Filetools

2006-09-15 Thread Georg Baum
Abdelrazak Younes wrote: > Well, I hope you don't include me in this group because I did some work > on the unicode front. No, I do not include you. I disagree with you about how the uncide things should go, but I don't want to force you to do things in a way that you personally think is wrong.

Re: Question: Unicode and Filetools

2006-09-15 Thread Abdelrazak Younes
Georg Baum wrote: Abdelrazak Younes wrote: If only Lars and Georg work on this, I am afraid that it will take some time before all the corner cases are worked out step by step. I don't think that I am going to do any further unicode work. I'll just sit and wait what happens. Welcome to the

Re: Question: Unicode and Filetools

2006-09-15 Thread Peter Kümmel
Georg Baum wrote: > Abdelrazak Younes wrote: > >> If only Lars and Georg work on this, I am afraid that it will take some >> time before all the corner cases are worked out step by step. > > I don't think that I am going to do any further unicode work. I'll just sit > and wait what happens. These

Re: Question: Unicode and Filetools

2006-09-15 Thread Georg Baum
Abdelrazak Younes wrote: > If only Lars and Georg work on this, I am afraid that it will take some > time before all the corner cases are worked out step by step. I don't think that I am going to do any further unicode work. I'll just sit and wait what happens. These endless discussions about how

Re: Question: Unicode and Filetools

2006-09-14 Thread Abdelrazak Younes
Andre Poenitz wrote: On Thu, Sep 14, 2006 at 02:09:01PM +0200, Abdelrazak Younes wrote: Abdelrazak Younes wrote: Georg Baum wrote: Then I suggest that you don't get involved anymore in the converting business, that will be easier for all of us. I might do that indeed. Just a final note about

Re: Question: Unicode and Filetools

2006-09-14 Thread Andre Poenitz
On Wed, Sep 13, 2006 at 04:30:08PM +0200, Abdelrazak Younes wrote: > >Performance is not the most important thing in this case IMO. The most > >important thing is code that is easy to understand. > > In this case, I think the format should be an enum instead. > > enum FileFormat { > jpg, > lyx, >

Re: Question: Unicode and Filetools

2006-09-14 Thread Andre Poenitz
On Thu, Sep 14, 2006 at 02:09:01PM +0200, Abdelrazak Younes wrote: > Abdelrazak Younes wrote: > >Georg Baum wrote: > > >>Then I suggest that you don't get involved anymore in the converting > >>business, that will be easier for all of us. > > > >I might do that indeed. > > Just a final note about

Re: Question: Unicode and Filetools

2006-09-14 Thread Abdelrazak Younes
Abdelrazak Younes wrote: Abdelrazak Younes wrote: Georg Baum wrote: Then I suggest that you don't get involved anymore in the converting business, that will be easier for all of us. I might do that indeed. Just a final note about this transition: I've spent many hours already trying to c

Re: Question: Unicode and Filetools

2006-09-14 Thread Abdelrazak Younes
Abdelrazak Younes wrote: Georg Baum wrote: Then I suggest that you don't get involved anymore in the converting business, that will be easier for all of us. I might do that indeed. Just a final note about this transition: I've spent many hours already trying to convert filetools to docstr

Re: Question: Unicode and Filetools

2006-09-14 Thread Abdelrazak Younes
Georg Baum wrote: Abdelrazak Younes wrote: Georg Baum wrote: It looks to me as if you want to convert almost everything to docstring. That would have been better done with some global search/replace. I really think we should have done just that. We are just complicating our life for no real b

Re: Question: Unicode and Filetools

2006-09-14 Thread Georg Baum
Abdelrazak Younes wrote: > Georg Baum wrote: >> It looks to me as if you want to convert almost everything to docstring. >> That would have been better done with some global search/replace. > > I really think we should have done just that. We are just complicating > our life for no real benefit.

Re: Question: Unicode and Filetools

2006-09-14 Thread Georg Baum
Abdelrazak Younes wrote: > For win/MSVC, boost::uint32_t is fine. The real problem as I understand > is mingw and cygwin. But I guess even those will be OK to use > boost::uin32_t when used with STLPort. If I am right (Georg, could you > confirm that?) No, STLPort does not help as Enrico found ou

Re: Question: Unicode and Filetools

2006-09-14 Thread Abdelrazak Younes
Lars Gullik Bjønnes wrote: Abdelrazak Younes <[EMAIL PROTECTED]> writes: | This back and forth conversion tp/from utf8 during this transition | phase is a real hassle. Taking this decision now will simplify and | speed-up greatly the transition. We also need quickly the operator | that Georg is

Re: Question: Unicode and Filetools

2006-09-14 Thread Abdelrazak Younes
Georg Baum wrote: Abdelrazak Younes wrote: Lars Gullik Bjønnes wrote: Abdelrazak Younes <[EMAIL PROTECTED]> writes: | class MenuId: public KeyString {}; | | class DialogId: public KeyString {}; | | | We need to take a decision now! No we don't. That's your opinion. We are not in such a

Re: Question: Unicode and Filetools

2006-09-14 Thread Lars Gullik Bjønnes
Abdelrazak Younes <[EMAIL PROTECTED]> writes: | This back and forth conversion tp/from utf8 during this transition | phase is a real hassle. Taking this decision now will simplify and | speed-up greatly the transition. We also need quickly the operator | that Georg is working on (+=, streams, etc)

Re: Question: Unicode and Filetools

2006-09-14 Thread Georg Baum
Abdelrazak Younes wrote: > Lars Gullik Bjønnes wrote: >> Abdelrazak Younes <[EMAIL PROTECTED]> writes: > >> | class MenuId: public KeyString {}; >> | >> | class DialogId: public KeyString {}; >> | >> | >> | We need to take a decision now! >> >> No we don't. > > That's your opinion. > >> We

Re: Question: Unicode and Filetools

2006-09-14 Thread Enrico Forestieri
On Thu, Sep 14, 2006 at 09:59:01AM +0200, Lars Gullik Bjønnes wrote: > Abdelrazak Younes <[EMAIL PROTECTED]> writes: > > | Andre Poenitz wrote: > | > On Wed, Sep 13, 2006 at 03:40:34PM +0200, Abdelrazak Younes wrote: > | >> I am tempted to convert everything even when a std::string would > | >> su

Re: Question: Unicode and Filetools

2006-09-14 Thread Abdelrazak Younes
Lars Gullik Bjønnes wrote: Abdelrazak Younes <[EMAIL PROTECTED]> writes: | class MenuId: public KeyString {}; | | class DialogId: public KeyString {}; | | | We need to take a decision now! No we don't. That's your opinion. We are not in such a hurry. This back and forth conversion

Re: Question: Unicode and Filetools

2006-09-14 Thread Lars Gullik Bjønnes
Abdelrazak Younes <[EMAIL PROTECTED]> writes: | Andre Poenitz wrote: | > On Wed, Sep 13, 2006 at 03:40:34PM +0200, Abdelrazak Younes wrote: | >> I am tempted to convert everything even when a std::string would | >> suffice. For example getFormatFromContents(), should that be: | >> | >> lyx::docstr

Re: Question: Unicode and Filetools

2006-09-14 Thread Abdelrazak Younes
Andre Poenitz wrote: On Wed, Sep 13, 2006 at 03:40:34PM +0200, Abdelrazak Younes wrote: I am tempted to convert everything even when a std::string would suffice. For example getFormatFromContents(), should that be: lyx::docstring const getFormatFromContents(lyx::docstring const & name); or s

Re: Question: Unicode and Filetools

2006-09-13 Thread Andre Poenitz
On Wed, Sep 13, 2006 at 04:17:15PM +0200, Lars Gullik Bjønnes wrote: > Abdelrazak Younes <[EMAIL PROTECTED]> writes: > > | Lars Gullik Bjønnes wrote: > | > Abdelrazak Younes <[EMAIL PROTECTED]> writes: > | > | Am I right that all of the filetools.h API should be converted to > | > docstring? > | >

Re: Question: Unicode and Filetools

2006-09-13 Thread Andre Poenitz
On Wed, Sep 13, 2006 at 03:40:34PM +0200, Abdelrazak Younes wrote: > I am tempted to convert everything even when a std::string would > suffice. For example getFormatFromContents(), should that be: > > lyx::docstring const getFormatFromContents(lyx::docstring const & name); > > or > > std::stri

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Georg Baum wrote: Am Mittwoch, 13. September 2006 17:41 schrieb Abdelrazak Younes: Angus Leeming wrote: Note that keeping std::string for the format name will enable you to enforce the distinction at compile time. Even clearer if you follow Georg's suggestion of typedef std::string lyx:

Re: Question: Unicode and Filetools

2006-09-13 Thread Georg Baum
Am Mittwoch, 13. September 2006 17:41 schrieb Abdelrazak Younes: > Angus Leeming wrote: > > Note that keeping std::string for the format name will enable you to enforce > > the distinction at compile time. Even clearer if you follow Georg's suggestion > > of > > typedef std::string lyx::asc

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Angus Leeming wrote: Abdelrazak Younes <[EMAIL PROTECTED]> writes: No, all is as it should be. NTFS uses UTF-16 (thanks again, Joost), so that can be stored reasonably well in std::wstring although there will be corner cases where two wchar_ts are required to represent a single unicode charac

Re: Question: Unicode and Filetools

2006-09-13 Thread Angus Leeming
Abdelrazak Younes <[EMAIL PROTECTED]> writes: > > | > I'd prefere is most of this work tried to leverage boost::filesystem > > | > as much as possible. > > | > > | Yes, I think I've read somewhere that boost::filesystem will support > > | the default encoding of the system in 1.34. > > > > Note t

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Lars Gullik Bjønnes wrote: Abdelrazak Younes <[EMAIL PROTECTED]> writes: | I don't like much asciistring because docstring can be ascii also. No... it cannot it has 4-byte chars remember. orrh I am not stupid! I mean ascii is a subset of ucs4 using the first byte...

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Lars Gullik Bjønnes wrote: Abdelrazak Younes <[EMAIL PROTECTED]> writes: | Lars Gullik Bjønnes wrote: | > Abdelrazak Younes <[EMAIL PROTECTED]> writes: | > | Am I right that all of the filetools.h API should be converted to | > docstring? | > | | I have started the conversion already so please t

Re: Question: Unicode and Filetools

2006-09-13 Thread Lars Gullik Bjønnes
Abdelrazak Younes <[EMAIL PROTECTED]> writes: | I don't like much asciistring because docstring can be ascii also. No... it cannot it has 4-byte chars remember. -- Lgb

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Angus Leeming wrote: Abdelrazak Younes writes: Georg Baum wrote: Abdelrazak Younes wrote: Then we need docstring as we don't know which kind of extension a user could create. Please: format name != file extension!!! I fixed several bugs resulting from this misunderstanding. format names are

Re: Question: Unicode and Filetools

2006-09-13 Thread Angus Leeming
Abdelrazak Younes writes: > Georg Baum wrote: > > Abdelrazak Younes wrote: >>> Then we need docstring as we don't know which kind of extension a user >>> could create. >> Please: format name != file extension!!! I fixed several bugs >> resulting from this misunderstanding. >> format names are uniq

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Georg Baum wrote: Abdelrazak Younes wrote: Then we need docstring as we don't know which kind of extension a user could create. Please: format name != file extension!!! I fixed several bugs resulting from this misunderstanding. format names are unique, file extensions not. Sure... I meant f

Re: Question: Unicode and Filetools

2006-09-13 Thread Georg Baum
Abdelrazak Younes wrote: > Then we need docstring as we don't know which kind of extension a user > could create. Please: format name != file extension!!! I fixed several bugs resulting from this misunderstanding. format names are unique, file extensions not. Georg

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Georg Baum wrote: Abdelrazak Younes wrote: So you would be OK if I change also all code that deals with format names? The question is: Do non-ASCII format names make sense? I really don't know. Me neither but probably not. Unless a Chinese user for example use a format is not defined in AS

Re: Question: Unicode and Filetools

2006-09-13 Thread Angus Leeming
Joost Verburg wrote: > Originally, all Unicode Windows features and the NTFS file system > used UCS-2 encoding. After Unicode 3.0 became available this has > been upgraded. Windows 2000 and later support UTF-16. Thanks, Joost. Angus

Re: Question: Unicode and Filetools

2006-09-13 Thread Georg Baum
Abdelrazak Younes wrote: > So you would be OK if I change also all code that deals with format names? The question is: Do non-ASCII format names make sense? I really don't know. If the answer to that question is yes and it does not create problems elsewhere, then it would be OK with me. > In thi

Re: Question: Unicode and Filetools

2006-09-13 Thread Joost Verburg
Angus Leeming wrote: You reckon? I think that it was pretty farsighted. NTFS was designed in about 1992. For the pedantically-minded, I believe that NTFS is encoded in UCS-2, a fixed- length 16 bit subset of UTF-16. See http://en.wikipedia.org/wiki/UTF-16. Originally, all Unicode Windows feat

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Georg Baum wrote: Abdelrazak Younes wrote: I plan to first encapsulate the conversion to/from ascii inside each of the filetools functions. Then I will look at how boost::filesystem can handle unicode, if at all possible... I am tempted to convert everything even when a std::string would suffi

Re: Question: Unicode and Filetools

2006-09-13 Thread Angus Leeming
Lars Gullik Bjønnes <[EMAIL PROTECTED]> writes: > | I think NTFS is UFT16... > Hmm... strange choice imho. You reckon? I think that it was pretty farsighted. NTFS was designed in about 1992. For the pedantically-minded, I believe that NTFS is encoded in UCS-2, a fixed- length 16 bit subset of UT

Re: Question: Unicode and Filetools

2006-09-13 Thread Georg Baum
Abdelrazak Younes wrote: > I plan to first encapsulate the conversion to/from ascii inside each of > the filetools functions. Then I will look at how boost::filesystem can > handle unicode, if at all possible... > > I am tempted to convert everything even when a std::string would > suffice. For e

Re: Question: Unicode and Filetools

2006-09-13 Thread Lars Gullik Bjønnes
Abdelrazak Younes <[EMAIL PROTECTED]> writes: | Lars Gullik Bjønnes wrote: | > Abdelrazak Younes <[EMAIL PROTECTED]> writes: | > | Am I right that all of the filetools.h API should be converted to | > docstring? | > | | I have started the conversion already so please tell me quick if | > | that's

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Lars Gullik Bjønnes wrote: Abdelrazak Younes <[EMAIL PROTECTED]> writes: | Am I right that all of the filetools.h API should be converted to docstring? | | I have started the conversion already so please tell me quick if | that's not the case. Filesystems and unicode is tricky. And I am not s

Re: Question: Unicode and Filetools

2006-09-13 Thread Georg Baum
Lars Gullik Bjønnes wrote: > Filesystems and unicode is tricky. > And I am not sure if a "blind" conversion is the way to go. > > But, yes, I think we should allow for utf-8 filesystem (I do not think > anyone uses utf-16 in the filesystem?) > > I'd prefere is most of this work tried to leverage

Re: Question: Unicode and Filetools

2006-09-13 Thread Helge Hafting
Lars Gullik Bjønnes wrote: Abdelrazak Younes <[EMAIL PROTECTED]> writes: | Am I right that all of the filetools.h API should be converted to docstring? | | I have started the conversion already so please tell me quick if | that's not the case. Filesystems and unicode is tricky. And I am not s

Re: Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Georg Baum wrote: Abdelrazak Younes wrote: Am I right that all of the filetools.h API should be converted to docstring? If filenames with non-ASCII characters should be supported, then yes. I think we should support that indeed. I have started the conversion already so please tell me qu

Re: Question: Unicode and Filetools

2006-09-13 Thread Lars Gullik Bjønnes
Abdelrazak Younes <[EMAIL PROTECTED]> writes: | Am I right that all of the filetools.h API should be converted to docstring? | | I have started the conversion already so please tell me quick if | that's not the case. Filesystems and unicode is tricky. And I am not sure if a "blind" conversion is

Re: Question: Unicode and Filetools

2006-09-13 Thread Georg Baum
Abdelrazak Younes wrote: > Am I right that all of the filetools.h API should be converted to > docstring? If filenames with non-ASCII characters should be supported, then yes. > I have started the conversion already so please tell me quick if that's > not the case. You will probably need to add

Question: Unicode and Filetools

2006-09-13 Thread Abdelrazak Younes
Am I right that all of the filetools.h API should be converted to docstring? I have started the conversion already so please tell me quick if that's not the case. Abdel.