Re: [msysGit] Re: [PATCH] git-compat-util.h: reduce optimization level to 1 on MinGW env.

2013-09-25 Thread Johannes Schindelin
Hi,

On Wed, 25 Sep 2013, Jonathan Nieder wrote:

> Wataru Noguchi wrote:
> 
> >   - actually file name is decoded.]
> >   
> > %E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%201-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%202-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%203-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%204-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%205-long-long-long-dirname/%E3%81%AF%E3%81%98%E3%82%81%E3%81%AB%E3%81%8A%E8%AA%AD%E3%81%BF%E3%81%8F%E3%81%A0%E3%81%95%E3%81%84.txt
> >
> > This commit reduce gcc optimization level from O2 to O1 when MinGW
> > Windows environment.
> 
> Do you know why reducing the optimization level avoids a crash?

I suspect that the optimization level causes smaller (or no) buffer bytes
between data structures and the crash is the symptom of a buffer overflow.
In that light, I think that reducing the optimization level is most likely
just working around this particular issue, and other scenarios might still
crash until we fix the underlying bug.

Most likely the problem is with our Windows-specific UTF-8 handling, I
would not be surprised if it is my "favorite" bug: an off-by-one.

To find out what the real cause is, I suggest using a tool similar to
valgrind. Valgrind does not run on Windows, but I DuckDuckWent e.g.
http://code.google.com/p/drmemory/ as an alternative that could be tried
to identify the problem.

Ciao,
Johannes
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [msysGit] Re: [PATCH] git-compat-util.h: reduce optimization level to 1 on MinGW env.

2013-09-25 Thread Wataru Noguchi
Hi,

I'm sorry resend email many times...
My GMail setting default to HTML format.
So mail sent to Git ML is rejected...


Thanks for your reply.

> If changing the optimization level turns out to be the right fix, I
> would prefer to see this change made in the Makefile instead of
> git-compat-util.h.

Yes, I see.
I think so.
I shuld write optimization level change code to Makefile instead of
git-compat-util.h.



2013/9/26 Jonathan Nieder :
> (cc-ing the Git for Windows maintainers)
> Hi,
>
> Wataru Noguchi wrote:
>
>> Git for Windows crashes when clone Japanese multibyte repository.
>> - Japanese Base Encoding is Shift-JIS.
>> - It happens Japanese multibyte directory name and too-long directory path
>> - Linux(ex. Ubuntu 13.04 amd64) can clone normally.
>> - example repository is here:
>>
>> git clone https://github.com/wnoguchi/mingw-checkout-crash.git
>>
>> - The reproduce crash repository contains following file only.
>>   - following directory and file name is encoded for this commit log.
>>   - actually file name is decoded.]
>>   
>> %E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%201-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%202-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%203-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%204-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%205-long-long-long-dirname/%E3%81%AF%E3%81%98%E3%82%81%E3%81%AB%E3%81%8A%E8%AA%AD%E3%81%BF%E3%81%8F%E3%81%A0%E3%81%95%E3%81%84.txt
>> - only one commit.
>>
>> This commit reduce gcc optimization level from O2 to O1 when MinGW Windows 
>> environment.
>>
>> Signed-off-by: Wataru Noguchi 
>
> Thanks.
>
>> ---
>>  git-compat-util.h | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/git-compat-util.h b/git-compat-util.h
>> index a31127f..394c23b 100644
>> --- a/git-compat-util.h
>> +++ b/git-compat-util.h
>> @@ -90,6 +90,8 @@
>>  #define WIN32_LEAN_AND_MEAN  /* stops windows.h including winsock.h */
>>  #include 
>>  #include 
>> +/* reduce gcc optimization level to 1 */
>> +#pragma GCC optimize ("O1")
>>  #define GIT_WINDOWS_NATIVE
>>  #endif
>
> Do you know why reducing the optimization level avoids a crash?
> Perhaps this is just masking the symptoms and the problem is still
> lurking.
>
> If changing the optimization level turns out to be the right fix, I
> would prefer to see this change made in the Makefile instead of
> git-compat-util.h.  That way, the user building can still easily
> override the optimization settings to -O0 if they want to during a
> debugging session.
>
> What do you think?
>
> Hope that helps,
> Jonathan




Hi,

Thanks for your advise.

> To find out what the real cause is, I suggest using a tool similar to
> valgrind. Valgrind does not run on Windows, but I DuckDuckWent e.g.
> http://code.google.com/p/drmemory/ as an alternative that could be tried
> to identify the problem.

I will try debugging by using any tools more real cause.
* Valgrind and drmemory, etc.


2013/9/26 Johannes Schindelin :
> Hi,
>
> On Wed, 25 Sep 2013, Jonathan Nieder wrote:
>
>> Wataru Noguchi wrote:
>>
>> >   - actually file name is decoded.]
>> >   
>> > %E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%201-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%202-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%203-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%204-long-long-long-dirname/%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%83%87%E3%82%A3%E3%83%AC%E3%82%AF%E3%83%88%E3%83%AA%205-long-long-long-dirname/%E3%81%AF%E3%81%98%E3%82%81%E3%81%AB%E3%81%8A%E8%AA%AD%E3%81%BF%E3%81%8F%E3%81%A0%E3%81%95%E3%81%84.txt
>> >
>> > This commit reduce gcc optimization level from O2 to O1 when MinGW
>> > Windows environment.
>>
>> Do you know why reducing the optimization level avoids a crash?
>
> I suspect that the optimization level causes smaller (or no) buffer bytes
> between data structures and the crash is the symptom of a buffer overflow.
> In that light, I think that reducing the optimization level is most likely
> just working around this particular issue, and other scenarios might still
> crash until we fix the underlying bug.
>
> Most likely the problem is with our Windows-specific UTF-8 handling, I
> would not be surprised if it is my "favorite" bug: an off-by-one.
>
> To find out what the real cause is, I suggest using a tool similar to
> valgrind. Valgrind does not run on Windows, but I DuckDuckWent e.g.
> http://code.google.com/p/drmemory/ as an alternative that could be tried
> to identify the problem.
>
> Ciao,
> Johannes


Thanks!


--