Re: CVS commit: src/external/bsd/mdocml/include
On Sun, Mar 18, 2018 at 01:26:12PM +, Christos Zoulas wrote: > I would just leave it broken until the perseant branch is merged :-) > You can always fix it, buy changing the locale in the environment to one > that works, instead of gutting it for all locales. IIUC we have to weigh: - the number of documents in base or pkgsrc that benefit from unicode support (my guess: close to zero) - the number of users of non-unicode based locales where the current version is just broken (a regression from previous relesases), which is pretty huge The version after the change should mostly work exactly the same for all common documents even when the user uses a UTF-8 locale setting. Or am I missing something? Martin
Re: CVS commit: src/external/bsd/mdocml/include
In article <20180318.153625.846954900.nakay...@netbsd.org>, Takeshi Nakayamawrote: chris...@astron.com (Christos Zoulas) wrote > >> In article <20180317235857.ga5...@homeworld.netbsd.org>, >> wrote: >> >Hi, >> > >> >I'm not sure what the changes affect. Does this mean UTF-8 won't work >> >either? that's sounds like a big price to pay. >> >Also, the problem doesn't sound specific to mdocml. >> >> Well, yes it is a bit drastic. Perhaps it can be handled with an >> environment variable instead (to disable for locales that don't work). > >I thought about the following change, but it is too ugly. I would just leave it broken until the perseant branch is merged :-) You can always fix it, buy changing the locale in the environment to one that works, instead of gutting it for all locales. christos
Re: CVS commit: src/external/bsd/mdocml/include
>>> chris...@astron.com (Christos Zoulas) wrote > In article <20180317235857.ga5...@homeworld.netbsd.org>, >wrote: > >Hi, > > > >I'm not sure what the changes affect. Does this mean UTF-8 won't work > >either? that's sounds like a big price to pay. > >Also, the problem doesn't sound specific to mdocml. > > Well, yes it is a bit drastic. Perhaps it can be handled with an > environment variable instead (to disable for locales that don't work). I thought about the following change, but it is too ugly. -- Takeshi Nakayama --- external/bsd/mdocml/dist/term_ascii.c.orig 2017-03-21 19:13:13.0 +0900 +++ external/bsd/mdocml/dist/term_ascii.c 2018-03-18 15:30:41.0 +0900 @@ -22,6 +22,7 @@ #include #if HAVE_WCHAR #include +#include #endif #include #include @@ -99,7 +100,7 @@ v = TERMENC_LOCALE == enc ? setlocale(LC_CTYPE, "") : setlocale(LC_CTYPE, UTF8_LOCALE); - if (NULL != v && MB_CUR_MAX > 1) { + if (NULL != v && MB_CUR_MAX > 1 && NULL != strstr(v, ".UTF-8")) { p->enc = enc; p->advance = locale_advance; p->endline = locale_endline;
Re: CVS commit: src/external/bsd/mdocml/include
>>> m...@netbsd.org wrote > Hi, > > I'm not sure what the changes affect. Does this mean UTF-8 won't work > either? that's sounds like a big price to pay. > Also, the problem doesn't sound specific to mdocml. No, this is specific to mdocml. mdocml assums wchar_t's internal representation UCS-4 (__STDC_ISO_10646__). See line 31 of: http://mandoc.bsd.lv/cgi-bin/cvsweb/configure.local.example?annotate=1.33 But our locale's wchar_t is not always UCS-4, we use a locale-dependent encoding for now (eg. ja_JP.eucJP). If branch perseant-stdc-iso10646 is merged, I think that the situation will change. > Is the problem: netbsd mishandles some locales it claims to support? No. -- Takeshi Nakayama
Re: CVS commit: src/external/bsd/mdocml/include
In article <20180317235857.ga5...@homeworld.netbsd.org>,wrote: >Hi, > >I'm not sure what the changes affect. Does this mean UTF-8 won't work >either? that's sounds like a big price to pay. >Also, the problem doesn't sound specific to mdocml. Well, yes it is a bit drastic. Perhaps it can be handled with an environment variable instead (to disable for locales that don't work). christos
Re: CVS commit: src/external/bsd/mdocml/include
Hi, I'm not sure what the changes affect. Does this mean UTF-8 won't work either? that's sounds like a big price to pay. Also, the problem doesn't sound specific to mdocml. Is the problem: netbsd mishandles some locales it claims to support? On Sat, Mar 17, 2018 at 11:06:48AM +, Takeshi Nakayama wrote: > Module Name: src > Committed By: nakayama > Date: Sat Mar 17 11:06:48 UTC 2018 > > Modified Files: > src/external/bsd/mdocml/include: config.h > > Log Message: > Disable wchar_t support since our C library does not treat wchar_t > as UCS-4 in the case of non-UTF-8 locales. > > This feature was controlled by USE_WCHAR and disabled on NetBSD 7. > > > To generate a diff of this commit: > cvs rdiff -u -r1.3 -r1.4 src/external/bsd/mdocml/include/config.h > > Please note that diffs are not public domain; they are subject to the > copyright notices on the relevant files. > > Modified files: > > Index: src/external/bsd/mdocml/include/config.h > diff -u src/external/bsd/mdocml/include/config.h:1.3 > src/external/bsd/mdocml/include/config.h:1.4 > --- src/external/bsd/mdocml/include/config.h:1.3 Sat Mar 18 15:24:17 2017 > +++ src/external/bsd/mdocml/include/config.h Sat Mar 17 11:06:48 2018 > @@ -41,7 +41,7 @@ > #define HAVE_STRTONUM 0 > #define HAVE_SYS_ENDIAN 1 > #define HAVE_VASPRINTF 1 > -#define HAVE_WCHAR 1 > +#define HAVE_WCHAR 0 > #define HAVE_OHASH 0 > > #define BINM_APROPOS "apropos" >
Re: CVS commit: src/external/bsd/mdocml/dist
In article 20130322152757.gu25...@nef.pbox.org, Alistair Crooks a...@pkgsrc.org wrote: I usually find that a #ifndef __arraycount #define__arraycount(__x) (sizeof(__x) / sizeof(__x[0])) #endif before the function, and then just use __arraycount(r-nr), is a bit more readable. I will let the author decide once I get this completely working :-) thanks, christos
Re: CVS commit: src/external/bsd/mdocml/dist
On Sun, Dec 18, 2011 at 01:40:08PM +, Nicolas Joly wrote: Module Name: src Committed By: njoly Date: Sun Dec 18 13:40:08 UTC 2011 Modified Files: src/external/bsd/mdocml/dist: arch.in Log Message: Lowercase arch strings, and add missing ones. The former is wrong and should have at least been discussed on the mdocml mailing lists... Joerg
Re: CVS commit: src/external/bsd/mdocml/dist
On Sun, Dec 18, 2011 at 04:58:02PM +0100, Joerg Sonnenberger wrote: On Sun, Dec 18, 2011 at 01:40:08PM +, Nicolas Joly wrote: Module Name:src Committed By: njoly Date: Sun Dec 18 13:40:08 UTC 2011 Modified Files: src/external/bsd/mdocml/dist: arch.in Log Message: Lowercase arch strings, and add missing ones. The former is wrong and should have at least been discussed on the mdocml mailing lists... Really ? Doesn't arch.in, like other .in files, intended for operating system customization ? -- Nicolas Joly Projects and Developments in Bioinformatics Institut Pasteur, Paris.
Re: CVS commit: src/external/bsd/mdocml/dist
On Sun, Dec 18, 2011 at 05:26:00PM +0100, Nicolas Joly wrote: On Sun, Dec 18, 2011 at 04:58:02PM +0100, Joerg Sonnenberger wrote: On Sun, Dec 18, 2011 at 01:40:08PM +, Nicolas Joly wrote: Module Name: src Committed By: njoly Date: Sun Dec 18 13:40:08 UTC 2011 Modified Files: src/external/bsd/mdocml/dist: arch.in Log Message: Lowercase arch strings, and add missing ones. The former is wrong and should have at least been discussed on the mdocml mailing lists... Really ? Doesn't arch.in, like other .in files, intended for operating system customization ? There is no point in forcing maintainance overhead just for the sake of it. Joerg
Re: CVS commit: src/external/bsd/mdocml/dist
On Fri, Oct 28, 2011 at 10:00:20AM -0400, Christos Zoulas wrote: Module Name: src Committed By: christos Date: Fri Oct 28 14:00:20 UTC 2011 Modified Files: src/external/bsd/mdocml/dist: tbl_data.c Log Message: rename data() - parse_data() to appease the powerpc64 toolchain: {standard input}: Assembler messages: {standard input}:105: Error: symbol `.data' is already defined That sounds like a bug in the PPC64 toolchain? Joerg
Re: CVS commit: src/external/bsd/mdocml/dist
In article 20111029080748.ga16...@britannica.bec.de, Joerg Sonnenberger jo...@britannica.bec.de wrote: On Fri, Oct 28, 2011 at 10:00:20AM -0400, Christos Zoulas wrote: Module Name: src Committed By:christos Date:Fri Oct 28 14:00:20 UTC 2011 Modified Files: src/external/bsd/mdocml/dist: tbl_data.c Log Message: rename data() - parse_data() to appease the powerpc64 toolchain: {standard input}: Assembler messages: {standard input}:105: Error: symbol `.data' is already defined That sounds like a bug in the PPC64 toolchain? Yes, I don't understand why such limitation should exist. I have noted it in the HACKS file. christos
Re: CVS commit: src/external/bsd/mdocml
In article 20110819113153.ga5...@britannica.bec.de, Joerg Sonnenberger jo...@britannica.bec.de wrote: On Fri, Aug 19, 2011 at 10:55:20AM +, Christos Zoulas wrote: Every project I know off makes changes locally first and then pushes them upstream. It is not practical to wait for upstream to be fixed first, specially in cases of security fixes. In some cases we maintain many thousands of lines of diff just because upstream will not take them, and the version control systems do a pretty decent job merging new vendor branches. This is just ignorant of the fact that a number of NetBSD developer maintain projects where they also have upstream commit access. As such you are doing nothing but increasing the maintainance cost. Stop making it harder. Silly GCC warnings fall into this category. It is 6 of one half a dozen of the other. I maintain file in the tree and out of the tree and I find changes coming in from all places. If we want to treat packages with upstream access differently, then perhaps we should do so, but I don't think it is worth it. christos
Re: CVS commit: src/external/bsd/mdocml
In article 20110818142524.ga26...@britannica.bec.de, Joerg Sonnenberger jo...@britannica.bec.de wrote: On Thu, Aug 18, 2011 at 01:47:19AM +, Christos Zoulas wrote: In article 20110817212805.gb16...@britannica.bec.de, Joerg Sonnenberger jo...@britannica.bec.de wrote: Could you please stop randomly changing 3rd party code without contacting the maintainer? Unless the rules have changed, for simple compilation fixes we don't do that. Otherwise the overhead of doing so would be too large. In this particular case, do you have a different/better way of fixing this? If yes, I'd like to know now. How can it be a simple compilation fix if you also change the Makefile? You are craeting additional work for every future import without providing much value. This is not specific to this case, but many other of your recent changes to stuff that can just as well be changed upstream first and reimported later. There were two separate things I was doing in the past two days: - Cleaning up all the XXX gcc4.5 changes that needed to edit both the Makefiles that contained things like -Wno-error, and fixing the sources to eliminate the errors. - Auditing the code for printf format errors and fixing them. This requires adding -Wno-format-nonliteral to the cases that we have to use non literal format strings or fixing the code to make strings constant where possible. Every project I know off makes changes locally first and then pushes them upstream. It is not practical to wait for upstream to be fixed first, specially in cases of security fixes. In some cases we maintain many thousands of lines of diff just because upstream will not take them, and the version control systems do a pretty decent job merging new vendor branches. christos
Re: CVS commit: src/external/bsd/mdocml
Joerg Sonnenberger jo...@britannica.bec.de writes: On Fri, Aug 19, 2011 at 10:55:20AM +, Christos Zoulas wrote: Every project I know off makes changes locally first and then pushes them upstream. It is not practical to wait for upstream to be fixed first, specially in cases of security fixes. In some cases we maintain many thousands of lines of diff just because upstream will not take them, and the version control systems do a pretty decent job merging new vendor branches. This is just ignorant of the fact that a number of NetBSD developer maintain projects where they also have upstream commit access. As such you are doing nothing but increasing the maintainance cost. Stop making it harder. Silly GCC warnings fall into this category. Joerg, Christos has been the upstream maintainer of more than one piece of code we've brought in tree, and has been sending patches back upstream long enough that it's fair to say he's one of the originators of TNF's policies in this regard. As long as I remember, we've had a strict policy of submitting changes upstream where possible, but of _not_ gating fixes on this process -- particularly fixes which are security or correctness related (the latter includes fixes without which we'd have to break our policy that all code should compile with -Wall -Werror). With this in mind, you can assert what you think TNF's policy should change _to_, but it's simply incorrect to suggest that what you ask for _is_ the current policy (much less to accuse one of the originators of the current policy of being `ignorant' of what that policy is). Just sayin'... -- Jim Wise jw...@draga.com pgpE7qnFDPZoR.pgp Description: PGP signature
Re: CVS commit: src/external/bsd/mdocml
On Fri, Aug 19, 2011 at 09:19:46AM -0400, Jim Wise wrote: As long as I remember, we've had a strict policy of submitting changes upstream where possible, but of _not_ gating fixes on this process -- particularly fixes which are security or correctness related (the latter includes fixes without which we'd have to break our policy that all code should compile with -Wall -Werror). Some of the code changed here has its authoritive source in pkgsrc. The gating is just an excuse for not doing it properly and increasing maintainance burden without no reason at all. If Christos maintains something and likes to pay the price, I don't care. This is not true here though. It is especially true for something where the compiler is just plainly wrong. Alas GCC has a long history of not choosing a good balance between noise and actual warnings. Joerg
Re: CVS commit: src/external/bsd/mdocml
Joerg Sonnenberger jo...@britannica.bec.de writes: On Fri, Aug 19, 2011 at 09:19:46AM -0400, Jim Wise wrote: As long as I remember, we've had a strict policy of submitting changes upstream where possible, but of _not_ gating fixes on this process -- particularly fixes which are security or correctness related (the latter includes fixes without which we'd have to break our policy that all code should compile with -Wall -Werror). Some of the code changed here has its authoritive source in pkgsrc. The gating is just an excuse for not doing it properly and increasing maintainance burden without no reason at all. If Christos maintains something and likes to pay the price, I don't care. This is not true here though. It is especially true for something where the compiler is just plainly wrong. Alas GCC has a long history of not choosing a good balance between noise and actual warnings. And the project has a long history of making sure all (these days: most) code in the base distribution builds with -Wall -Werror. This isn't because every GCC warning is right, of course -- it's because when you turn off warnings for a whole file, you risk hiding another warning which _is_ a sign of an actual code problem. -- Jim Wise jw...@draga.com pgpqnKjQbqlt4.pgp Description: PGP signature
Re: CVS commit: src/external/bsd/mdocml
On Fri, Aug 19, 2011 at 10:57:53AM -0400, Jim Wise wrote: And the project has a long history of making sure all (these days: most) code in the base distribution builds with -Wall -Werror. This isn't because every GCC warning is right, of course -- it's because when you turn off warnings for a whole file, you risk hiding another warning which _is_ a sign of an actual code problem. When have you last checked the number of -Wno-* instances in base? Sorry, but pessimizing code to work around clearly bogus GCC warnings is not helpful. It doesn't make code easier to read, it doesn't improve code quality. It's the same kind of behavior that results in isalpha((int)ch) code. Joerg
Re: CVS commit: src/external/bsd/mdocml
Joerg Sonnenberger jo...@britannica.bec.de writes: On Fri, Aug 19, 2011 at 10:57:53AM -0400, Jim Wise wrote: And the project has a long history of making sure all (these days: most) code in the base distribution builds with -Wall -Werror. This isn't because every GCC warning is right, of course -- it's because when you turn off warnings for a whole file, you risk hiding another warning which _is_ a sign of an actual code problem. When have you last checked the number of -Wno-* instances in base? Sorry, but pessimizing code to work around clearly bogus GCC warnings is not helpful. It doesn't make code easier to read, it doesn't improve code quality. It's the same kind of behavior that results in isalpha((int)ch) code. A very long time ago -- but two notes: a.) looking at the rdiff in the original commit message, this wasn't disabling a specific warning with -Wno-*, it was passing -Wno-error, to allow the build to proceed regardless of warnings in this file b.) When you go fishing, you ever catch _all_ the fish? If there are other unfixed warnings in the tree, it's because no one's done the work to fix them, not -- unless policy has changed since I've been paying as much attention -- because the goal is no longer to have the tree build with -Wall -Werror At this late date, NetBSD isn't going to distinguish itself with features, mass adoption, or desktop whiz-banginess. It _does_ still stand out as the FOSS OS project that cares the most about code quality. Let's not break that. -- Jim Wise jw...@draga.com pgpJcTp3T7T0D.pgp Description: PGP signature
Re: CVS commit: src/external/bsd/mdocml
On Fri, Aug 19, 2011 at 07:54:28PM +0200, Joerg Sonnenberger wrote: When have you last checked the number of -Wno-* instances in base? Sorry, but pessimizing code to work around clearly bogus GCC warnings is not helpful. It doesn't make code easier to read, it doesn't improve code quality. It's the same kind of behavior that results in isalpha((int)ch) code. Since we're still in the thread started by the mdocml change, I'll point to the URL that Kristaps found which convinced him to accept the diff. http://c-faq.com/ansi/constmismatch.html Thomas
Re: CVS commit: src/external/bsd/mdocml
On Thu, Aug 18, 2011 at 01:47:19AM +, Christos Zoulas wrote: In article 20110817212805.gb16...@britannica.bec.de, Joerg Sonnenberger jo...@britannica.bec.de wrote: Could you please stop randomly changing 3rd party code without contacting the maintainer? Unless the rules have changed, for simple compilation fixes we don't do that. Otherwise the overhead of doing so would be too large. In this particular case, do you have a different/better way of fixing this? If yes, I'd like to know now. How can it be a simple compilation fix if you also change the Makefile? You are craeting additional work for every future import without providing much value. This is not specific to this case, but many other of your recent changes to stuff that can just as well be changed upstream first and reimported later. Joerg
Re: CVS commit: src/external/bsd/mdocml
Could you please stop randomly changing 3rd party code without contacting the maintainer? Joerg On Wed, Aug 17, 2011 at 05:30:14AM -0400, Christos Zoulas wrote: Module Name: src Committed By: christos Date: Wed Aug 17 09:30:14 UTC 2011 Modified Files: src/external/bsd/mdocml/dist: mdoc_validate.c src/external/bsd/mdocml/lib/libmandoc: Makefile Log Message: fix cast warnings To generate a diff of this commit: cvs rdiff -u -r1.1.1.13 -r1.2 src/external/bsd/mdocml/dist/mdoc_validate.c cvs rdiff -u -r1.2 -r1.3 src/external/bsd/mdocml/lib/libmandoc/Makefile Please note that diffs are not public domain; they are subject to the copyright notices on the relevant files.
Re: CVS commit: src/external/bsd/mdocml
In article 20110817212805.gb16...@britannica.bec.de, Joerg Sonnenberger jo...@britannica.bec.de wrote: Could you please stop randomly changing 3rd party code without contacting the maintainer? Unless the rules have changed, for simple compilation fixes we don't do that. Otherwise the overhead of doing so would be too large. In this particular case, do you have a different/better way of fixing this? If yes, I'd like to know now. christos
Re: CVS commit: src/external/bsd/mdocml/dist
On Sat, Nov 07, 2009 at 02:52:02AM +, Christos Zoulas wrote: In article 20091106183619.ga2...@britannica.bec.de, Joerg Sonnenberger jo...@britannica.bec.de wrote: On Fri, Nov 06, 2009 at 04:53:58PM +, Christos Zoulas wrote: - portability: warnx - fprintf/perror This change is backwards; we should not be using perror/fprintf but warn/warnx. I disagree. They are not portable. Many platforms don't have them. Some other platforms have functions of the same name, but with different behavior (OS X). It should autoconf them. PR 38488. Joerg
Re: CVS commit: src/external/bsd/mdocml/dist
On Sat, Nov 07, 2009 at 12:48:16AM -0700, M. Warner Losh wrote: which ones have the same name, but different behavior on OS X? A quick read of the man pages suggests they are identical... Check about whether they need a \n at the end of the format. David -- David Laight: da...@l8s.co.uk
Re: CVS commit: src/external/bsd/mdocml/dist
In article 20091107.004816.-432821225@bsdimp.com, M. Warner Losh i...@bsdimp.com wrote: In message: hd2nci$40...@ger.gmane.org chris...@astron.com (Christos Zoulas) writes: : In article 20091106183619.ga2...@britannica.bec.de, : Joerg Sonnenberger jo...@britannica.bec.de wrote: : On Fri, Nov 06, 2009 at 04:53:58PM +, Christos Zoulas wrote: : - portability: warnx - fprintf/perror : : This change is backwards; : we should not be using perror/fprintf but warn/warnx. : : I disagree. They are not portable. Many platforms don't have them. : Some other platforms have functions of the same name, but with different : behavior (OS X). : : It should autoconf them. which ones have the same name, but different behavior on OS X? A quick read of the man pages suggests they are identical... These functions are all implemented in one file that's been trivial to port to Linux and Windows for me in the past... I agree with Christos here: their use is pervasive and it will be confusing to know when to use err and when to use perror + exit... My point here is that we should be pushing ahead with useful features that make the code cleaner and easier to read and maintain, not encourage the laggard OS's not to upgrade/fix their API's by using ancient and awkward API's such as perror(). christos
Re: CVS commit: src/external/bsd/mdocml/dist
In message: 20091107084411.gt15...@snowdrop.l8s.co.uk David Laight da...@l8s.co.uk writes: : On Sat, Nov 07, 2009 at 12:48:16AM -0700, M. Warner Losh wrote: : : which ones have the same name, but different behavior on OS X? A quick read : of the man pages suggests they are identical... : : Check about whether they need a \n at the end of the format. They aren't needed on MacOS, nor FreeBSD. A quick test on 10.5.8 shows that warn doesn't need it. Is there one that does need them? Warner
Re: CVS commit: src/external/bsd/mdocml/dist
On Oct 21, 2009, at 10:45 AM, Joerg Sonnenberger wrote: Module Name:src Committed By: joerg Date: Wed Oct 21 17:45:09 UTC 2009 Update of /cvsroot/src/external/bsd/mdocml/dist In directory ivanova.netbsd.org:/tmp/cvs-serv24959 Log Message: Initial import mdocml-1.9.9. mdocml provides a fast parser for man(7) and mdoc(7) formated documents. Awesome! On a 32MB shark: $ time mandoc gcc.1 /dev/null 1.96 real 1.57 user 0.38 sys $ time nroff -msafer -man gcc.1 /dev/null 88.31 real81.35 user 1.81 sys