Re: [Wireshark-dev] wslog, windows, pytest, and heap corruption

[João Valverde]

On 30/12/21 23:46, John Thacker wrote:

On Thu, Dec 30, 2021 at 5:55 PM Gerald Combs  wrote:

On 12/29/21 5:15 PM, John Thacker wrote:
> I was working on a MR for moving the text2pcap/text_import debug
over to the ws_log features and I ran into a seemingly bizarre
problem. Setting the log level to a non-default value causes the
pytest procedures to fail with heap corruption on the Gitlab
Windows CI.
>
> Some of the text2pcap pytests depend on grepping through the
stderr output for some of the debug information. Those tests
originally passed the -d flag to text2pcap, so I replaced it with
setting the log level to "debug" (and later "info") with the
standard "--log-level debug" argument read by ws_log_parse_args().
>
> On Windows (but not Linux or MacOS, not clang or gcc, nor with
either using ASAN), those tests which set the log level (and only
those tests) started failing with a return code of 0xc374,
heap corruption.
>
> As I looked into it closer, all the debug information that those
tests used ought to be logged at "warning" or "message," which are
at the default log level, so I was able to remove that flag, and
then it passed.
>
> It looks like it might be related to some of the things
discussed here, though I'm not 100% sure because I'm not a Windows
programmer:
>
> https://discuss.wxpython.org/t/heap-corruption-on-windows/35583

> https://bugs.python.org/issue36792

> https://bugs.python.org/issue37945

>
> There's some kind of issue seen in Python 3.8 and higher, with
Windows 10 build 1809 (which is a long term support build that is
what the CI build server uses), with UTF-8 locales, with log
systems that get system locale information and print dates, the
Windows 10 Universal CRT, and heap corruption.
>
> It might have something to do with the tests spawning a lot of
subprocesses in parallel and setting the log level to a different
value eventually calling free_log_filter() from
ws_log_set_debug_filter().

Is https://gitlab.com/wireshark/wireshark/-/pipelines/438735249
one of the pipelines that failed? If so, it looks like Wireshark
is crashing and Python is complaining about its return code:


Traceback (most recent call last):
   File "C:\builds\wireshark\wireshark\test\fixtures.py", line 54,
in wrapped
     test_fn(self, *fixtures)
   File "C:\builds\wireshark\wireshark\test\suite_text2pcap.py",
line 186, in test_text2pcap_ikev1_certs_pcap
     check_text2pcap(self, 'ikev1-certs.pcap', 'pcap')
   File "C:\builds\wireshark\wireshark\test\suite_text2pcap.py",
line 144, in check_text2pcap_real
     self.assertRun(text2pcap_cmd, shell=True)
   File "C:\builds\wireshark\wireshark\test\subprocesstest.py",
line 304, in assertRun
     self.assertEqual(process.returncode, expected_return)
AssertionError: 3221226356 != 0


Yes, that's one of them. 3221226356= 0xc374 and is apparently a 
special Windows return code for heap corruption.




It is noticeable in that CI log that the UTF-8 console output is 
corrupted. It works flawlessly in my Windows VM, which is running all 
the latest updates, so it looks like you are on the right track.___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] wslog, windows, pytest, and heap corruption

[João Valverde]

On 30/12/21 23:36, Jirka Novak wrote:

Dear John,

I was working on a MR for moving the text2pcap/text_import debug over 
to the ws_log features and I ran into a seemingly bizarre problem. 
Setting the log level to a non-default value causes the pytest 
procedures to fail with heap corruption on the Gitlab Windows CI.

...


I'm touching similar issue just on Windows platform when running 
extcap from Wireshark. Extcap dies when logging level is set to file 
and file size increases by subsequent ws_log related writes over 4192 
bytes.


I found that if I disable part of code, it works fine including 
writing messages to log. Can you try it too:


wsutil/wslog.c in log_write_dispatch() comment out second call of 
log_write_do_log():


diff --git a/wsutil/wslog.c b/wsutil/wslog.c
index f0f87eaa6e..8e49b6200c 100644
--- a/wsutil/wslog.c
+++ b/wsutil/wslog.c
@@ -965,11 +965,14 @@ static void log_write_dispatch(const char 
*domain, enum ws_log_level level,
 user_format, user_ap, 
registered_log_writer_data);

 }
 else {
+   // TODO: Fix it
+    /*
 log_write_do_work(console_file(level), 
console_color_enabled(level),

 get_localtime(tstamp.tv_sec, ),
 tstamp.tv_nsec,
 domain, level, file, line, func,
 user_format, user_ap);
+   */
 }

A few weeks ago I worked with João Valverde on changes in logging for 
extcaps, but I don't think we touched this part.

João Valverde, can you check our findings?



Sure, I can try. I think it would be better if you opened an issue in 
the bug tracker for that.


I don't see any setlocale() calls in extcaps so that is worth checking, 
per Gerald's suggestion.

___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] wslog, windows, pytest, and heap corruption

[John Thacker]

On Thu, Dec 30, 2021 at 5:55 PM Gerald Combs  wrote:

> On 12/29/21 5:15 PM, John Thacker wrote:
> > I was working on a MR for moving the text2pcap/text_import debug over to
> the ws_log features and I ran into a seemingly bizarre problem. Setting the
> log level to a non-default value causes the pytest procedures to fail with
> heap corruption on the Gitlab Windows CI.
> >
> > Some of the text2pcap pytests depend on grepping through the stderr
> output for some of the debug information. Those tests originally passed the
> -d flag to text2pcap, so I replaced it with setting the log level to
> "debug" (and later "info") with the standard "--log-level debug" argument
> read by ws_log_parse_args().
> >
> > On Windows (but not Linux or MacOS, not clang or gcc, nor with either
> using ASAN), those tests which set the log level (and only those tests)
> started failing with a return code of 0xc374, heap corruption.
> >
> > As I looked into it closer, all the debug information that those tests
> used ought to be logged at "warning" or "message," which are at the default
> log level, so I was able to remove that flag, and then it passed.
> >
> > It looks like it might be related to some of the things discussed here,
> though I'm not 100% sure because I'm not a Windows programmer:
> >
> > https://discuss.wxpython.org/t/heap-corruption-on-windows/35583 <
> https://discuss.wxpython.org/t/heap-corruption-on-windows/35583>
> > https://bugs.python.org/issue36792 
> > https://bugs.python.org/issue37945 
> >
> > There's some kind of issue seen in Python 3.8 and higher, with Windows
> 10 build 1809 (which is a long term support build that is what the CI build
> server uses), with UTF-8 locales, with log systems that get system locale
> information and print dates, the Windows 10 Universal CRT, and heap
> corruption.
> >
> > It might have something to do with the tests spawning a lot of
> subprocesses in parallel and setting the log level to a different value
> eventually calling free_log_filter() from ws_log_set_debug_filter().
>
> Is https://gitlab.com/wireshark/wireshark/-/pipelines/438735249 one of
> the pipelines that failed? If so, it looks like Wireshark is crashing and
> Python is complaining about its return code:
>
> 
> Traceback (most recent call last):
>File "C:\builds\wireshark\wireshark\test\fixtures.py", line 54, in
> wrapped
>  test_fn(self, *fixtures)
>File "C:\builds\wireshark\wireshark\test\suite_text2pcap.py", line 186,
> in test_text2pcap_ikev1_certs_pcap
>  check_text2pcap(self, 'ikev1-certs.pcap', 'pcap')
>File "C:\builds\wireshark\wireshark\test\suite_text2pcap.py", line 144,
> in check_text2pcap_real
>  self.assertRun(text2pcap_cmd, shell=True)
>File "C:\builds\wireshark\wireshark\test\subprocesstest.py", line 304,
> in assertRun
>  self.assertEqual(process.returncode, expected_return)
> AssertionError: 3221226356 != 0
> 
>
> Yes, that's one of them. 3221226356 = 0xc374 and is apparently a
special Windows return code for heap corruption.

Just a wild guess, but maybe we need to call setlocale at the beginning of
> text2pcap similar to our other executables?
>

That probably makes sense to do anyway. However, I tried another draft
merge request adding "--log-level debug" to tshark (and dumpcap)
executables, with no other changes, and saw the same issue:

https://gitlab.com/wireshark/wireshark/-/jobs/1931595222

All the tests where I added the log-level command fail with heap corruption
on Windows (and nowhere else). Since tshark already has the setlocale
command, I guess that's not it. The various bug reports seem to indicate
that it only happens on UTF-8 code pages, and was fixed in a later Windows
release.

John Thacker
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] wslog, windows, pytest, and heap corruption

[Jirka Novak]

Dear John,

I was working on a MR for moving the text2pcap/text_import debug over to 
the ws_log features and I ran into a seemingly bizarre problem. Setting 
the log level to a non-default value causes the pytest procedures to 
fail with heap corruption on the Gitlab Windows CI.

...


I'm touching similar issue just on Windows platform when running extcap 
from Wireshark. Extcap dies when logging level is set to file and file 
size increases by subsequent ws_log related writes over 4192 bytes.


I found that if I disable part of code, it works fine including writing 
messages to log. Can you try it too:


wsutil/wslog.c in log_write_dispatch() comment out second call of 
log_write_do_log():


diff --git a/wsutil/wslog.c b/wsutil/wslog.c
index f0f87eaa6e..8e49b6200c 100644
--- a/wsutil/wslog.c
+++ b/wsutil/wslog.c
@@ -965,11 +965,14 @@ static void log_write_dispatch(const char *domain, 
enum ws_log_level level,

 user_format, user_ap, registered_log_writer_data);
 }
 else {
+   // TODO: Fix it
+/*
 log_write_do_work(console_file(level), 
console_color_enabled(level),

 get_localtime(tstamp.tv_sec, ),
 tstamp.tv_nsec,
 domain, level, file, line, func,
 user_format, user_ap);
+   */
 }

A few weeks ago I worked with João Valverde on changes in logging for 
extcaps, but I don't think we touched this part.

João Valverde, can you check our findings?

Best regards,

Jirka Novak
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] wslog, windows, pytest, and heap corruption

[Gerald Combs]

On 12/29/21 5:15 PM, John Thacker wrote:

I was working on a MR for moving the text2pcap/text_import debug over to the 
ws_log features and I ran into a seemingly bizarre problem. Setting the log 
level to a non-default value causes the pytest procedures to fail with heap 
corruption on the Gitlab Windows CI.

Some of the text2pcap pytests depend on grepping through the stderr output for some of the debug information. 
Those tests originally passed the -d flag to text2pcap, so I replaced it with setting the log level to 
"debug" (and later "info") with the standard "--log-level debug" argument read 
by ws_log_parse_args().

On Windows (but not Linux or MacOS, not clang or gcc, nor with either using 
ASAN), those tests which set the log level (and only those tests) started 
failing with a return code of 0xc374, heap corruption.

As I looked into it closer, all the debug information that those tests used ought to be logged at 
"warning" or "message," which are at the default log level, so I was able to 
remove that flag, and then it passed.

It looks like it might be related to some of the things discussed here, though 
I'm not 100% sure because I'm not a Windows programmer:

https://discuss.wxpython.org/t/heap-corruption-on-windows/35583 

https://bugs.python.org/issue36792 
https://bugs.python.org/issue37945 

There's some kind of issue seen in Python 3.8 and higher, with Windows 10 build 
1809 (which is a long term support build that is what the CI build server 
uses), with UTF-8 locales, with log systems that get system locale information 
and print dates, the Windows 10 Universal CRT, and heap corruption.

It might have something to do with the tests spawning a lot of subprocesses in 
parallel and setting the log level to a different value eventually calling 
free_log_filter() from ws_log_set_debug_filter().

John Thacker


Is https://gitlab.com/wireshark/wireshark/-/pipelines/438735249 one of the 
pipelines that failed? If so, it looks like Wireshark is crashing and Python is 
complaining about its return code:


Traceback (most recent call last):
  File "C:\builds\wireshark\wireshark\test\fixtures.py", line 54, in wrapped
test_fn(self, *fixtures)
  File "C:\builds\wireshark\wireshark\test\suite_text2pcap.py", line 186, in 
test_text2pcap_ikev1_certs_pcap
check_text2pcap(self, 'ikev1-certs.pcap', 'pcap')
  File "C:\builds\wireshark\wireshark\test\suite_text2pcap.py", line 144, in 
check_text2pcap_real
self.assertRun(text2pcap_cmd, shell=True)
  File "C:\builds\wireshark\wireshark\test\subprocesstest.py", line 304, in 
assertRun
self.assertEqual(process.returncode, expected_return)
AssertionError: 3221226356 != 0


Just a wild guess, but maybe we need to call setlocale at the beginning of 
text2pcap similar to our other executables?
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

Re: [Wireshark-dev] wslog, windows, pytest, and heap corruption

[chuck c]

I opened an issue to upgrade but maybe not in the right place?

https://gitlab.com/wireshark/wireshark-containers/-/issues/5
Update Windows servercore to newer version

On Wed, Dec 29, 2021 at 7:15 PM John Thacker  wrote:

> I was working on a MR for moving the text2pcap/text_import debug over to
> the ws_log features and I ran into a seemingly bizarre problem. Setting the
> log level to a non-default value causes the pytest procedures to fail with
> heap corruption on the Gitlab Windows CI.
>
> Some of the text2pcap pytests depend on grepping through the stderr output
> for some of the debug information. Those tests originally passed the -d
> flag to text2pcap, so I replaced it with setting the log level to "debug"
> (and later "info") with the standard "--log-level debug" argument read by
> ws_log_parse_args().
>
> On Windows (but not Linux or MacOS, not clang or gcc, nor with either
> using ASAN), those tests which set the log level (and only those tests)
> started failing with a return code of 0xc374, heap corruption.
>
> As I looked into it closer, all the debug information that those tests
> used ought to be logged at "warning" or "message," which are at the default
> log level, so I was able to remove that flag, and then it passed.
>
> It looks like it might be related to some of the things discussed here,
> though I'm not 100% sure because I'm not a Windows programmer:
>
> https://discuss.wxpython.org/t/heap-corruption-on-windows/35583
> https://bugs.python.org/issue36792
> https://bugs.python.org/issue37945
>
> There's some kind of issue seen in Python 3.8 and higher, with Windows 10
> build 1809 (which is a long term support build that is what the CI build
> server uses), with UTF-8 locales, with log systems that get system locale
> information and print dates, the Windows 10 Universal CRT, and heap
> corruption.
>
> It might have something to do with the tests spawning a lot of
> subprocesses in parallel and setting the log level to a different value
> eventually calling free_log_filter() from ws_log_set_debug_filter().
>
> John Thacker
> ___
> Sent via:Wireshark-dev mailing list 
> Archives:https://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
>  mailto:wireshark-dev-requ...@wireshark.org
> ?subject=unsubscribe
>
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe

[Wireshark-dev] Failed build on SLED 15

[Anders Broman via Wireshark-dev]

Hi,

We get 

glib-compat.h:31:7: note: include '' or provide a declaration of
'memcpy'

when building on SLED 15.

Regards

Anders



smime.p7s
Description: S/MIME cryptographic signature
___
Sent via:Wireshark-dev mailing list 
Archives:https://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://www.wireshark.org/mailman/options/wireshark-dev
 mailto:wireshark-dev-requ...@wireshark.org?subject=unsubscribe