Re: [BUG core?] Regression with parsing Windows’ command-line
On Dec 3 16:24, Corinna Vinschen via Cygwin wrote: > On Dec 3 19:28, Takashi Yano via Cygwin wrote: > > On Fri, 2 Dec 2022 19:40:30 -0800 > > Ilya Zakharevich wrote: > > > On Wed, Nov 16, 2022 at 04:48:25AM -0800, I wrote: > > > > De-quoting (converting the Windows’ command-line into argc/argv) does > > > > not remove double quotes if characters not fit for 8-bit (?) are > > > > present. > > > > > > > > To reproduce, do in CMD’s command line: > > > > > > > > D:\> D:\Programs\cygwin2022\bin\perl -wle "print for @ARGV" . "/i/" > > > > "/и/" . > > > > . > > > > /i/ > > > > "/и/" > > > > . > > > [...] > I pushed a patch and the test release is rebuilding while I type. This will be fixed in test release cygwin-3.4.0-0.875.g07a9a6c21ab3, which should be up in an hour or so. Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [BUG core?] Regression with parsing Windows’ command-line
On Dec 3 22:42, Takashi Yano via Cygwin wrote: > [...] > The patch above also affects __C_locale. > The patch below should be more appropriate. Looks like we had the same idea at the same time :) Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [BUG core?] Regression with parsing Windows’ command-line
On Dec 3 19:28, Takashi Yano via Cygwin wrote:
> On Fri, 2 Dec 2022 19:40:30 -0800
> Ilya Zakharevich wrote:
> > On Wed, Nov 16, 2022 at 04:48:25AM -0800, I wrote:
> > > De-quoting (converting the Windows’ command-line into argc/argv) does
> > > not remove double quotes if characters not fit for 8-bit (?) are present.
> > >
> > > To reproduce, do in CMD’s command line:
> > >
> > > D:\> D:\Programs\cygwin2022\bin\perl -wle "print for @ARGV" . "/i/"
> > > "/и/" .
> > > .
> > > /i/
> > > "/и/"
> > > .
> > [...]
> This certainly seems to be a problem of cygwin1.dll.
>
> Though I am not sure this is the right thing, I have confirmed
> that the following patch solves the issue.
>
> diff --git a/newlib/libc/locale/lctype.c b/newlib/libc/locale/lctype.c
> index 644669765..732d132e1 100644
> --- a/newlib/libc/locale/lctype.c
> +++ b/newlib/libc/locale/lctype.c
> @@ -25,11 +25,20 @@
>
> #define LCCTYPE_SIZE (sizeof(struct lc_ctype_T) / sizeof(char *))
>
> +#ifdef __CYGWIN__
> +static char numsix[] = { '\6', '\0'};
> +#else
> static char numone[] = { '\1', '\0'};
> +#endif
>
> const struct lc_ctype_T _C_ctype_locale = {
> +#ifdef __CYGWIN__
> + "UTF-8",/* codeset */
> + numsix /* mb_cur_max */
> +#else
> "ASCII",/* codeset */
> numone /* mb_cur_max */
> +#endif
Good idea, but this transforms the "C" locale into the "C.UTF-8" locale
once and for all.
What we're actually missing is a matching _C_utf8_ctype_locale which can
be used by Cygwin as default locale setting, AFAICS.
I pushed a patch and the test release is rebuilding while I type.
Thanks,
Corinna
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [BUG core?] Regression with parsing Windows’ command-line
On Sat, 3 Dec 2022 19:28:10 +0900
Takashi Yano wrote:
> On Fri, 2 Dec 2022 19:40:30 -0800
> Ilya Zakharevich wrote:
> > On Wed, Nov 16, 2022 at 04:48:25AM -0800, I wrote:
> > > De-quoting (converting the Windows’ command-line into argc/argv) does
> > > not remove double quotes if characters not fit for 8-bit (?) are present.
> > >
> > > Broken in: CYGWIN_NT-6.1 Bu 3.3.4(0.341/5/3) 2022-01-31 19:35
> > > x86_64 Cygwin
> > > Works in: CYGWIN_NT-6.1-WOW Bu 2.2.1(0.289/5/3) 2015-08-20 11:40 i686
> > > Cygwin
> > >
> > > To reproduce, do in CMD’s command line:
> > >
> > > D:\> D:\Programs\cygwin2022\bin\perl -wle "print for @ARGV" . "/i/"
> > > "/и/" .
> > > .
> > > /i/
> > > "/и/"
> > > .
> >
> > I triple-checked
> > • with a Win10 machine (and a version of cygwin given above),
> > • with a fresh latest(=test)-cygwin-dll installation on a Win7 (as above)
> > machine.
> >
> > Same bug everywhere.
>
> This certainly seems to be a problem of cygwin1.dll.
>
> Though I am not sure this is the right thing, I have confirmed
> that the following patch solves the issue.
>
> diff --git a/newlib/libc/locale/lctype.c b/newlib/libc/locale/lctype.c
> index 644669765..732d132e1 100644
> --- a/newlib/libc/locale/lctype.c
> +++ b/newlib/libc/locale/lctype.c
> @@ -25,11 +25,20 @@
>
> #define LCCTYPE_SIZE (sizeof(struct lc_ctype_T) / sizeof(char *))
>
> +#ifdef __CYGWIN__
> +static char numsix[] = { '\6', '\0'};
> +#else
> static char numone[] = { '\1', '\0'};
> +#endif
>
> const struct lc_ctype_T _C_ctype_locale = {
> +#ifdef __CYGWIN__
> + "UTF-8",/* codeset */
> + numsix /* mb_cur_max */
> +#else
> "ASCII",/* codeset */
> numone /* mb_cur_max */
> +#endif
> #ifdef __HAVE_LOCALE_INFO_EXTENDED__
> ,
> { "0", "1", "2", "3", "4", /* outdigits */
The patch above also affects __C_locale.
The patch below should be more appropriate.
diff --git a/newlib/libc/locale/locale.c b/newlib/libc/locale/locale.c
index e523d2366..7485ac292 100644
--- a/newlib/libc/locale/locale.c
+++ b/newlib/libc/locale/locale.c
@@ -244,6 +244,21 @@ const struct __locale_t __C_locale =
};
#endif /* _MB_CAPABLE */
+#ifdef __CYGWIN__
+static charnumsix[] = { '\6', '\0'};
+static const struct lc_ctype_T _C_UTF8_ctype_locale = {
+ "UTF-8",/* codeset */
+ numsix /* mb_cur_max */
+#ifdef __HAVE_LOCALE_INFO_EXTENDED__
+ ,
+ { "0", "1", "2", "3", "4", /* outdigits */
+ "5", "6", "7", "8", "9" },
+ { L"0", L"1", L"2", L"3", L"4", /* woutdigits */
+ L"5", L"6", L"7", L"8", L"9" }
+#endif
+};
+#endif
+
struct __locale_t __global_locale =
{
{ "C", "C", DEFAULT_LOCALE, "C", "C", "C", "C", },
@@ -272,10 +287,11 @@ struct __locale_t __global_locale =
{ NULL, NULL },/* LC_ALL */
#ifdef __CYGWIN__
{ &_C_collate_locale, NULL }, /* LC_COLLATE */
+{ &_C_UTF8_ctype_locale, NULL }, /* LC_CTYPE */
#else
{ NULL, NULL },/* LC_COLLATE */
-#endif
{ &_C_ctype_locale, NULL },/* LC_CTYPE */
+#endif
{ &_C_monetary_locale, NULL }, /* LC_MONETARY */
{ &_C_numeric_locale, NULL }, /* LC_NUMERIC */
{ &_C_time_locale, NULL }, /* LC_TIME */
--
Takashi Yano
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [BUG core?] Regression with parsing Windows’ command-line
On Fri, 2 Dec 2022 19:40:30 -0800
Ilya Zakharevich wrote:
> On Wed, Nov 16, 2022 at 04:48:25AM -0800, I wrote:
> > De-quoting (converting the Windows’ command-line into argc/argv) does
> > not remove double quotes if characters not fit for 8-bit (?) are present.
> >
> > Broken in: CYGWIN_NT-6.1 Bu 3.3.4(0.341/5/3) 2022-01-31 19:35
> > x86_64 Cygwin
> > Works in: CYGWIN_NT-6.1-WOW Bu 2.2.1(0.289/5/3) 2015-08-20 11:40 i686
> > Cygwin
> >
> > To reproduce, do in CMD’s command line:
> >
> > D:\> D:\Programs\cygwin2022\bin\perl -wle "print for @ARGV" . "/i/" "/и/"
> > .
> > .
> > /i/
> > "/и/"
> > .
>
> I triple-checked
> • with a Win10 machine (and a version of cygwin given above),
> • with a fresh latest(=test)-cygwin-dll installation on a Win7 (as above)
> machine.
>
> Same bug everywhere.
This certainly seems to be a problem of cygwin1.dll.
Though I am not sure this is the right thing, I have confirmed
that the following patch solves the issue.
diff --git a/newlib/libc/locale/lctype.c b/newlib/libc/locale/lctype.c
index 644669765..732d132e1 100644
--- a/newlib/libc/locale/lctype.c
+++ b/newlib/libc/locale/lctype.c
@@ -25,11 +25,20 @@
#define LCCTYPE_SIZE (sizeof(struct lc_ctype_T) / sizeof(char *))
+#ifdef __CYGWIN__
+static charnumsix[] = { '\6', '\0'};
+#else
static charnumone[] = { '\1', '\0'};
+#endif
const struct lc_ctype_T _C_ctype_locale = {
+#ifdef __CYGWIN__
+ "UTF-8",/* codeset */
+ numsix /* mb_cur_max */
+#else
"ASCII",/* codeset */
numone /* mb_cur_max */
+#endif
#ifdef __HAVE_LOCALE_INFO_EXTENDED__
,
{ "0", "1", "2", "3", "4", /* outdigits */
--
Takashi Yano
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [BUG core?] Regression with parsing Windows’ command-line
On Wed, Nov 16, 2022 at 04:48:25AM -0800, I wrote: > De-quoting (converting the Windows’ command-line into argc/argv) does > not remove double quotes if characters not fit for 8-bit (?) are present. > > Broken in: CYGWIN_NT-6.1 Bu 3.3.4(0.341/5/3) 2022-01-31 19:35 > x86_64 Cygwin > Works in: CYGWIN_NT-6.1-WOW Bu 2.2.1(0.289/5/3) 2015-08-20 11:40 i686 > Cygwin > > To reproduce, do in CMD’s command line: > > D:\> D:\Programs\cygwin2022\bin\perl -wle "print for @ARGV" . "/i/" "/и/" . > . > /i/ > "/и/" > . I triple-checked • with a Win10 machine (and a version of cygwin given above), • with a fresh latest(=test)-cygwin-dll installation on a Win7 (as above) machine. Same bug everywhere. > Likewise for other characters which do not fit into 8-bits. (I have a > TrueType font enabled in a console.) > > (If you do not know how to type this, just copy-and paste it into a >console with CMD.) Ilya -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
[BUG core?] Regression with parsing Windows’ command-line
De-quoting (convrting the Windows’ command-line into argc/argv) does not remove double quotes if characters not fit for 8-bit (?) are present. Broken in: CYGWIN_NT-6.1 Bu 3.3.4(0.341/5/3) 2022-01-31 19:35 x86_64 Cygwin Works in: CYGWIN_NT-6.1-WOW Bu 2.2.1(0.289/5/3) 2015-08-20 11:40 i686 Cygwin To reproduce, do in CMD’s command line: D:\> D:\Programs\cygwin2022\bin\perl -wle "print for @ARGV" . "/i/" "/и/" . . /i/ "/и/" . Likewise for other characters which do not fit into 8-bits. (I have a TrueType font enabled in a console.) (If you do not know how to type this, just copy-and paste it into a console with CMD.) (The same happens on Win10.) Hope this helps, Ilya P.S. In the instructions for sending bugs https://www.cygwin.com/problems.html it says: If you can't run cygcheck for some reason (and why shouldn't you be able to? cygcheck is just a standard windows program which does not use the Cygwin dll) By the snide tone, I presume this is a rhetorical question… ( Hmm, anyway, after less than 1 hour, it actually terminated… So I attach the censored output. ) P.P.S. It seems that the mailing list does not allow attaching the output! So I retry with a shortened version… Cygwin Configuration Diagnostics Current System Time: Mon Nov 14 21:37:17 2022 Windows 7 Home Premium Ver 6.1 Build 7601 Service Pack 1 Path: d:\Programs\cygwin2022\bin C:\Program Files (x86)\Common Files\Oracle\Java\javapath C:\Program Files\Common Files\Microsoft Shared\Windows Live C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live C:\Program Files (x86)\Intel\iCLS Client\ C:\Program Files\Intel\iCLS Client\ C:\Windows\system32 C:\Windows C:\Windows\System32\Wbem C:\Windows\System32\WindowsPowerShell\v1.0\ C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x86 C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x64 C:\Program Files\Intel\WiFi\bin\ C:\Program Files\Common Files\Intel\WirelessCommon\ C:\Program Files\Lenovo\Bluetooth Software\ C:\Program Files\Lenovo\Bluetooth Software\syswow64 . C:\Program Files\Intel\Intel(R) Management Engine Components\DAL C:\Program Files\Intel\Intel(R) Management Engine Components\IPT C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT C:\Program Files (x86)\Windows Live\Shared Output from d:\Programs\cygwin2022\bin\id.exe UID: 197608(***) GID: 197121(None) 197121(None) 545(Users) 4(INTERACTIVE) 66049(CONSOLE LOGON) 11(Authenticated Users)15(This Organization) 113(Local account) 4095(CurrentSession) 66048(LOCAL) 262154(NTLM Authentication) 401408(Medium Mandatory Level) SysDir: C:\Windows\system32 WinDir: C:\Windows Path = 'd:\Programs\cygwin2022\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x86;C:\Program Files (x86)\Intel\OpenCL SDK\2.0\bin\x64;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\Lenovo\Bluetooth Software\;C:\Program Files\Lenovo\Bluetooth Software\syswow64;;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Windows Live\Shared' ALLUSERSPROFILE = 'C:\ProgramData' APPDATA = 'C:\Users\***\AppData\Roaming' CommonProgramFiles = 'C:\Program Files\Common Files' CommonProgramFiles(x86) = 'C:\Program Files (x86)\Common Files' CommonProgramW6432 = 'C:\Program Files\Common Files' COMPUTERNAME = '***' ComSpec = 'C:\Windows\system32\cmd.exe' configsetroot = 'C:\Windows\ConfigSetRoot' FP_NO_HOST_CHECK = 'NO' HOMEDRIVE = 'C:' HOMEPATH = '\Users\***' JD2_HOME = 'D:\Programs-Installed\JDownloader' LOCALAPPDATA = 'C:\Users\***\AppData\Local' LOGONSERVER = '\\***' NUMBER_OF_PROCESSORS = '4' OS = 'Windows_NT' PATHEXT = '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC' PROCESSOR_ARCHITECTURE = 'AMD64' PROCESSOR_IDENTIFIER = 'Intel64 Family * Model * Stepping *' PROCESSOR_LEVEL = '*' PROCESSOR_REVISION = '***' ProgramData = 'C:\ProgramData' ProgramFiles = 'C:\Program Files' ProgramFiles(x86) = 'C:\Program Files (x86)' ProgramW6432 = 'C:\Program
