[issue36670] test suite broken due to cpu usage feature on win 10/ german

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Steve Dower]

Change by Steve Dower :


--
keywords: +easy, newcomer friendly

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Steve Dower]

Change by Steve Dower :


--
stage:  -> needs patch
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Sebastian Koslowski]

Sebastian Koslowski  added the comment:

I ran into this issue on a Win10 German box running the tests for version 3.7.4

Your changes fixes the issue for me.

--
nosy: +skoslowski
versions: +Python 3.7 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Lorenz Mende]

Lorenz Mende  added the comment:

Did some minor changes with win_utils.
Encoding changed to 'oem'.
@heckad: does this bugfix work for u?

@all: please verify working of the changes with different localizations.
thx

--
Added file: https://bugs.python.org/file48406/win_utils.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Lorenz Mende]

Change by Lorenz Mende :


Removed file: https://bugs.python.org/file48285/win_utils.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Андрей Казанцев]

Андрей Казанцев  added the comment:

I have the same problem in Russian locale.
Adding "oem" encoding in the decode method solve problem with decoding but got:
File 
"C:\Users\User\Documents\Projects\cpython\lib\test\libregrtest\win_utils.py", 
line 98, in getloadavg
load = float(toks[1].replace('"', ''))
ValueError: could not convert string to float

In typeperf_output text with description of the error:
'\r\nВыполняется выход, подождите... \r\nОшибка: 
Счетчики не указаны.\r\n\r\r'
Translation on English is "Exiting, wait... Error: Counters are not specified."

Is it possible to check if the counters are found in advance?

--
nosy: +heckad

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Eryk Sun]

Eryk Sun  added the comment:

It's "oem", not "mbcs".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Lorenz Mende]

Lorenz Mende  added the comment:

Sorry, was off some days. I tried to decode the output with mbcs, solves the 
issue partly - the counter name is still wrong.
Was able to pick the localization specific counter name from registry and use 
it for the typeperf.

But the tests fail after several seconds with Broken Pipe Error of the 
command_stdout pipe.

@Ammar - why is the handle closed in start()? If I uncomment it, the load 
tracker workks fine.

Added the fixed file, may someone evaluate the solution on another windows 
localization?

--
Added file: https://bugs.python.org/file48285/win_utils.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Steve Dower]

Steve Dower  added the comment:

If the code page is set to UTF-8 then mbcs will still be the right encoding :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Eryk Sun]

Eryk Sun  added the comment:

> It'll probably be mbcs or oem encoding. Certainly not UTF-8

Unless the system locale's codepage is set to UTF-8 in Windows 10. 

I stepped through this in the debugger. When writing to a pipe, typeperf.exe 
calls WideCharToMultiByte(CP_OEMCP, ...), so it's "oem".

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Steve Dower]

Steve Dower  added the comment:

It'll probably be mbcs or oem encoding. Certainly not UTF-8

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Terry J. Reedy]

Terry J. Reedy  added the comment:

'crash' mean *nix coredump or Windows equivelent, rather than traceback.

--
components: +Windows
nosy: +paul.moore, steve.dower, terry.reedy, tim.golden, zach.ware
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Ammar Askar]

Ammar Askar  added the comment:

Thank you, could you also share the output if you just give it the English name 
of the counter?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Lorenz Mende]

Lorenz Mende  added the comment:

Hi Ammar, you are correct.
typeperf returns:
P:\Repos\CPython\cpython>typeperf "\System\Prozessor-Warteschlangenlänge" -si 1

"(PDH-CSV 4.0)","\\ZERO\System\Prozessor-Warteschlangenlänge"
"04/19/2019 19:09:14.510","0.00"
"04/19/2019 19:09:15.514","0.00"

So even with correct counter name the outpu needs to be decoded correctly.
I already got a solution to get the location specific counter name from 
registry - if it helps I'll commit it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Ammar Askar]

Ammar Askar  added the comment:

What does `typeperf "\System\Processor Queue Length" -si 1` actually return on 
your non-English system?

Does it just return an error with the counter's name or is the umalet just in 
the first header line, i.e this one for me:

  "(PDH-CSV 4.0)","\\MSI\System\Processor Queue Length"

If it's the latter then I think the correct fix would be to figure out what 
encoding typeperf is outputting in and then just decode with that.

--
nosy: +ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Lorenz Mende]

Change by Lorenz Mende :


--
nosy:  -ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Lorenz Mende]

Change by Lorenz Mende :


--
nosy: +ammar2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Lorenz Mende]

New submission from Lorenz Mende :

The test suite fails with the first tests (I assume 1st call of getloadavg of 
WindowsLoadTracker).
Traceback (most recent call last):
  File "P:\Repos\CPython\cpython\lib\runpy.py", line 192, in _run_module_as_main
return _run_code(code, main_globals, None,
  File "P:\Repos\CPython\cpython\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
  File "P:\Repos\CPython\cpython\lib\test\__main__.py", line 2, in 
main()
  File "P:\Repos\CPython\cpython\lib\test\libregrtest\main.py", line 653, in 
main
Regrtest().main(tests=tests, **kwargs)
  File "P:\Repos\CPython\cpython\lib\test\libregrtest\main.py", line 586, in 
main
self._main(tests, kwargs)
  File "P:\Repos\CPython\cpython\lib\test\libregrtest\main.py", line 632, in 
_main
self.run_tests()
  File "P:\Repos\CPython\cpython\lib\test\libregrtest\main.py", line 515, in 
run_tests
self.run_tests_sequential()
  File "P:\Repos\CPython\cpython\lib\test\libregrtest\main.py", line 396, in 
run_tests_sequential
self.display_progress(test_index, text)
  File "P:\Repos\CPython\cpython\lib\test\libregrtest\main.py", line 150, in 
display_progress
load_avg_1min = self.getloadavg()
  File "P:\Repos\CPython\cpython\lib\test\libregrtest\win_utils.py", line 81, 
in getloadavg
typeperf_output = self.read_output()
  File "P:\Repos\CPython\cpython\lib\test\libregrtest\win_utils.py", line 78, 
in read_output
return response.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x81 in position 67: 
invalid start byte
##
The windows 'typeperf "\System\Processor Queue Length" -si 1' command unluckily 
returns an string with an umlaut which leads to the Decode-Error. This comes up 
because the  for the typeperf is location dependend. (In german the 
counter would read \System\Prozessor-Warteschlangenlänge)

I see two possible solutions to this issue.
1. Raising an exception earlier on creation of WindowsLoadTracker resulting in 
the same behaviour as if there is no typeperf available (german pythoneers 
would have a drawback with this)
2. Getting the typeperf counter correctly from registry 
(HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows 
NT\CurrentVersion\Perflib\CurrentLanguage, described here 
https://social.technet.microsoft.com/Forums/de-DE/25bc6907-cf2c-4dc8-8687-974b799ba754/powershell-ausgabesprache-umstellen?forum=powershell_de)

environment:
Windows 10 x64, 1809, german
cpython @e16467af0bfcc9f399df251495ff2d2ad20a1669
commit of assumed root cause of https://bugs.python.org/issue34060

--
components: Tests
messages: 340547
nosy: LorenzMende
priority: normal
severity: normal
status: open
title: test suite broken due to cpu usage feature on win 10/ german
type: crash
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com