[issue42938] [security][CVE-2021-3177] ctypes double representation BoF

2021-02-22 Thread Alexander Riccio


Alexander Riccio  added the comment:

Yes, I definitely should. I work on https://bugs.python.org/issue25878 
sometimes, which encompasses this.

--

___
Python tracker 
<https://bugs.python.org/issue42938>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42938] [security][CVE-2021-3177] ctypes double representation BoF

2021-02-22 Thread Alexander Riccio


Alexander Riccio  added the comment:

Petition to remove all uses of the unchecked string handling functions from 
CPython?

Sidenote: if C4996 was on, this would be a warning.

--
nosy: +Alexander Riccio

___
Python tracker 
<https://bugs.python.org/issue42938>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40145] Pyshellext room for binary size improvement

2020-04-03 Thread Alexander Riccio


Alexander Riccio  added the comment:

Oh, uh, also, do you prefer I add a commit or a new branch & PR?

--

___
Python tracker 
<https://bugs.python.org/issue40145>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40145] Pyshellext room for binary size improvement

2020-04-03 Thread Alexander Riccio


Alexander Riccio  added the comment:

Ahh, ok. Even though I question the usefulness of manually maintaining MSBuild 
files instead of something like CMake, I can work with that. Is there a 
preferred way to do it? It looks like I can do a 
Condition="'$(Configuration)|$(Platform)'=='Release|x64'" and merge a bunch of 
things under each one.

--

___
Python tracker 
<https://bugs.python.org/issue40145>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40161] Name collisions in pythoncore, preventing unity/jumbo build

2020-04-02 Thread Alexander Riccio


New submission from Alexander Riccio :

This isn't a priority issue I'd say. However, fixing it could yield nice 
benefits. I ran into this while experimenting with JUMBO/Unity builds as part 
of a bit of fun I've been having tweaking build options across the CPython 
ecosystem.

Theoretically, a JUMBO/Unity build could reduce code size, improve performance, 
and maybe even help code analysis detect more bugs by building everything in a 
single compilation unit. Link Time Code Generation is great, but usually isn't 
as good as building everything in a single compilation unit.


An example of an interesting thing noticed while compiling as a Unity build:

This exact variable is defined twice in two separate source files, 
itertoolsmodule.c:4303, and and collectionsmodule.c:1774:

PyDoc_STRVAR(length_hint_doc, "Private method returning an estimate of 
len(list(it)).");

...the default Release configuration includes this exact string 12 (!) times.

There's a lot of stuff like that. It's not actually broken, and sometimes it's 
probably inconvenient to fix it (what are you gonna do? put it in a header?), 
but it would be nice.

--
components: Interpreter Core
messages: 365636
nosy: Alexander Riccio
priority: normal
severity: normal
status: open
title: Name collisions in pythoncore, preventing unity/jumbo build
type: compile error
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40161>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40151] _overlapped room for improvement

2020-04-01 Thread Alexander Riccio


Change by Alexander Riccio :


--
keywords: +patch
pull_requests: +18658
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19298

___
Python tracker 
<https://bugs.python.org/issue40151>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40145] Pyshellext room for binary size improvement

2020-04-01 Thread Alexander Riccio


Change by Alexander Riccio :


--
pull_requests: +18659
pull_request: https://github.com/python/cpython/pull/19298

___
Python tracker 
<https://bugs.python.org/issue40145>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40151] _overlapped room for improvement

2020-04-01 Thread Alexander Riccio


New submission from Alexander Riccio :

Similarly to bpo-40145, I've tweaked build options to reduce the size of the
binary.

This patch turns on (for release builds) Whole Program Optimization, MinSpace
optimization, /Ob2 AnySuitable function inlining, /Zo (so that people can still
debug it when lots of code has been optimized out), /OPT:REF dead code
elimination, /OPT:ICF COMDAT folding, Link Time Code Generation, and DebugFull
debugging information creation.

For debug builds, it enables all of that, except instead of the MinSpace
optimization, it's only /Ob2 with custom optimization. My intent is to not
optimize any asserts or similar checks, while still getting the benefit of
dead and duplicate code elimination.


_overlapped.pyd   32 bit unimproved release size: 31KB
_overlapped.pyd   32 bit   improved release size: 29KB
size reduction:   -3KB
 %  size reduction:10%

_overlapped_d.pyd 32 bit unimproved release size: 55KB
_overlapped_d.pyd 32 bit   improved release size: 34KB
size reduction:  -21KB
 %  size reduction:38%




_overlapped.pyd   64 bit unimproved release size: 39KB
_overlapped.pyd   64 bit   improved release size: 36KB
size reduction:   -3KB
 %  size reduction: 8%


_overlapped_d.pyd 64 bit unimproved release size: 74KB
_overlapped_d.pyd 64 bit   improved release size: 41KB
size reduction:  -33KB
 %  size reduction:45%


If this patch is merged, and all 7 million (estimated) Python developers update 
their installation, I calculate that I just saved the PSF 21GB worth of 
bandwidth costs :)

--
components: Windows
messages: 365567
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: _overlapped room for improvement
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40151>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40150] (minor) mismatched argument in overlapped_RegisterWaitWithQueue call to RegisterWaitForSingleObject

2020-04-01 Thread Alexander Riccio


New submission from Alexander Riccio :

This popped out at me while looking for something else. It's probably not much 
of an actual problem, since the wrong datatype is larger than the correct one, 
but it's worth fixing.

The problem is in overlapped_RegisterWaitWithQueue, at overlapped.c:297 (in 
_overlapped):

if (!RegisterWaitForSingleObject(
, Object, (WAITORTIMERCALLBACK)PostToQueueCallback,
pdata, Milliseconds,
WT_EXECUTEINWAITTHREAD | WT_EXECUTEONLYONCE))


...it stood out to me immediately while I was paging past, since function 
pointer casts of this sort are almost always wrong, and I've seen it too many 
times.

WAITORTIMERCALLBACK is a typedef of WAITORTIMERCALLBACKFUNC:

typedef VOID (NTAPI * WAITORTIMERCALLBACKFUNC) (PVOID, BOOLEAN );

...and PostToQueueCallback is defined as:

static VOID CALLBACK
PostToQueueCallback(PVOID lpParameter, BOOL TimerOrWaitFired)

...where BOOL is an int, and BOOLEAN is an unsigned char. I guess there could 
be some kind of issue down the line if the generated code tries to read the 
BOOLEAN/int from the register/memory location where there's actually an 
unsigned char, but after about an hour of debugging, I can't see it. The 
documentation also states explicitly that this should be either TRUE or FALSE. 
Also, it doesn't matter what we actually pass to PostQueuedCompletionStatus: 
https://devblogs.microsoft.com/oldnewthing/20070525-00/?p=26693

By changing the (incorrect) BOOL declaration to BOOLEAN, then we don't need the 
cast.

--
components: IO
messages: 365565
nosy: Alexander Riccio
priority: normal
severity: normal
status: open
title: (minor) mismatched argument in overlapped_RegisterWaitWithQueue call to 
RegisterWaitForSingleObject
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40150>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40145] Pyshellext room for binary size improvement

2020-04-01 Thread Alexander Riccio


Alexander Riccio  added the comment:

If this patch is merged, and all 7 million (estimated) Python developers update 
their installation, I calculate that I just saved the PSF 119GB worth of 
bandwidth costs :)

I'll take my 10 cents in the mail please :D

--

___
Python tracker 
<https://bugs.python.org/issue40145>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40145] Pyshellext room for binary size improvement

2020-04-01 Thread Alexander Riccio


Change by Alexander Riccio :


--
keywords: +patch
pull_requests: +18642
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19284

___
Python tracker 
<https://bugs.python.org/issue40145>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40145] Pyshellext room for binary size improvement

2020-04-01 Thread Alexander Riccio


New submission from Alexander Riccio :

I've tweaked the pcbuild options for pyshellext to reduce the size of the 
binary.
Since this is a very simple component, there really isn't much benefit of
optimizing for speed, likely the slowest part of this component's lifetime is
simply loading it.

This patch turns on Whole Program Optimization, MinSpace optimization, /Ob2
AnySuitable function inlining (to expose more code to elimination, this does
actually help here), /Zo (so that people can still debug it when lots of code
has been optimized out), turns of C++ RTTI (nobody is using it), /OPT:REF dead
code elimination, /OPT:ICF COMDAT folding, Link Time Code Generation, and
DebugFull debugging information creation.

/Gw global data optimization seemed to do nothing, which makes sense since
there isn't much going on in this project, very little in the way of actual
global data. Disabling C++ exceptions both in the project config (i.e. /EH-),
and disabling stdlib exceptions via _HAS_EXCEPTIONS=0, had no effect.
Strangely, with exceptions disabled and _HAS_EXCEPTIONS=0, exception functions
are still emitted in the binary (as seen in IDA). I presume it has something
to do with the fact that its a dll.

Enabling optimizations (even for Debug builds) should have no effect. The debug
builds were not actually using debugging featuers (not even assert), so nothing
should change.

pyshellext.dll   32 bit unimproved release size: 42KB
pyshellext.dll   32 bit   improved release size: 25KB
size reduction: -17KB
 %  size reduction:   40%

pyshellext_d.dll 32 bit unimproved debug size:   85KB
pyshellext_d.dll 32 bit   improved debug size:   32KB
   size reduction:  -53KB
 %  size reduction:   62%


pyshellext.dll   64 bit unimproved release size: 52KB
pyshellext.dll   64 bit   improved release size: 30KB
   size reduction:  -22KB
 %  size reduction:   42%

pyshellext_d.dll 64 bit unimproved debug size:  105KB
pyshellext_d.dll 64 bit   improved debug size:   38KB
   size reduction:  -67KB
 %  size reduction:   63%

--
components: Windows
messages: 365527
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Pyshellext room for binary size improvement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40145>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40143] shutil.rmtree will frequently fail on Windows under heavy load due to racy deletion

2020-04-01 Thread Alexander Riccio


New submission from Alexander Riccio :

The "obvious" way to delete a directory tree on Windows is wrong. It's 
inherently racy, since deleting a file on Windows *doesn't actually delete it*, 
instead it marks the file for deletion. The system will eventually get around 
to deleting it, but under heavy load, this might be sometime after an attempt 
is made to delete the parent directory. I've seen this (windows error 145, 
directory is not empty) many times when running the testsuite, and it causes 
all kinds of intermittent failures.

The best way to do things correctly is kinda nuts, and is explained well by 
Niall Douglass here: https://www.youtube.com/watch?v=uhRWMGBjlO8=8m54s

In short, the correct way to do it involves moving each file to a randomly 
named file in %TEMP%, then deleting that, and then doing the same with each 
newly-empty directory.

--
components: Windows
messages: 365520
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: shutil.rmtree will frequently fail on Windows under heavy load due to 
racy deletion
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40143>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2020-03-31 Thread Alexander Riccio


Alexander Riccio  added the comment:

Ok, so a draft of this produces 34 warnings, but makes way more changes to the 
.vcxproj and .filters files than I think it should: 
https://github.com/ariccio/cpython/commit/60152aa065a3ad861f0359a8ada7f2fbc83a3933

Before I submit a PR, I think I should figure out how to change the defaults 
without Visual Studio adding a bunch of noise to the config - is that 
reasonable?

The warnings appear essentially innocuous - mixing signed/unsigned bytes, some 
benign truncation of constant values, and 3 places where local variables shadow 
the globals, where it looks like the global should've been used - but should be 
checked nonetheless.

The warnings are in the attached file.

--
Added file: https://bugs.python.org/file49017/34PythonWarnings.txt

___
Python tracker 
<https://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40020] growable_comment_array_add leaks, causes crash

2020-03-31 Thread Alexander Riccio


Alexander Riccio  added the comment:

Sure, should I open a new issue?

--
nosy:  -vstinner
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue40020>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40082] Assertion failure in trip_signal

2020-03-27 Thread Alexander Riccio


Alexander Riccio  added the comment:

Hmmm, happens every time I interrupt while attached. Is there some obvious 
gotcha in the docs that I'm missing?

--

___
Python tracker 
<https://bugs.python.org/issue40082>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40082] Assertion failure in trip_signal

2020-03-27 Thread Alexander Riccio


Alexander Riccio  added the comment:

Lmao the name mangling comes up as a mailto. That's interesting.

--

___
Python tracker 
<https://bugs.python.org/issue40082>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40082] Assertion failure in trip_signal

2020-03-27 Thread Alexander Riccio


New submission from Alexander Riccio :

While trying to make sense of some static analysis warnings for the Windows 
console IO module, I Ctrl+C'd in the middle of an intentionally absurd __repr__ 
output, and on proceeding in the debugger (which treated it as an exception), I 
immediately hit the assertion right here:

/* Get the Python thread state using PyGILState API, since
   _PyThreadState_GET() returns NULL if the GIL is released.
   For example, signal.raise_signal() releases the GIL. */
PyThreadState *tstate = PyGILState_GetThisThreadState();
assert(tstate != NULL);

...With the stacktrace: 
ucrtbased.dll!issue_debug_notification(const wchar_t * const message) 
Line 28   C++
ucrtbased.dll!__acrt_report_runtime_error(const wchar_t * message) Line 
154 C++
ucrtbased.dll!abort() Line 51   C++
ucrtbased.dll!common_assert_to_stderr_direct(const wchar_t * const 
expression, const wchar_t * const file_name, const unsigned int line_number) 
Line 161C++
ucrtbased.dll!common_assert_to_stderr(const wchar_t * const 
expression, const wchar_t * const file_name, const unsigned int line_number) 
Line 175  C++
ucrtbased.dll!common_assert(const wchar_t * const expression, 
const wchar_t * const file_name, const unsigned int line_number, void * const 
return_address) Line 420   C++
ucrtbased.dll!_wassert(const wchar_t * expression, const wchar_t * 
file_name, unsigned int line_number) Line 443C++
>   python39_d.dll!trip_signal(int sig_num) Line 266C
python39_d.dll!signal_handler(int sig_num) Line 342 C
ucrtbased.dll!ctrlevent_capture(const unsigned long ctrl_type) Line 206 
C++
KernelBase.dll!_CtrlRoutine@4()Unknown
kernel32.dll!@BaseThreadInitThunk@12() Unknown
ntdll.dll!__RtlUserThreadStart()Unknown
ntdll.dll!__RtlUserThreadStart@8() Unknown


...I'm not entirely sure why this happened, but I can tell a few things. 
_PyRuntime.gilstate.autoInterpreterState is NOT null, in fact the gilstate 
object is as displayed in my watch window: 

-   _PyRuntime.gilstate {check_enabled=1 
tstate_current={_value=0 } getframe=0x79e3a570 
{python39_d.dll!threadstate_getframe(_ts *)} ...}   _gilstate_runtime_state
check_enabled   1   int
+   tstate_current  {_value=0 } _Py_atomic_address
getframe0x79e3a570 
{python39_d.dll!threadstate_getframe(_ts *)} _frame *(*)(_ts *)
-   autoInterpreterState0x00e5eff8 {next=0x  
tstate_head=0x00e601c0 {prev=0x  next=0x  ...} ...} 
 _is *
+   next0x_is *
+   tstate_head 0x00e601c0 {prev=0x  
next=0x  interp=0x00e5eff8 {next=0x  ...} ...}   
_ts *
+   runtime 0x7a0e2118 {python39_d.dll!pyruntimestate _PyRuntime} 
{preinitializing=0 preinitialized=1 core_initialized=...} pyruntimestate *
id  0   __int64
id_refcount -1  __int64
requires_idref  0   int
id_mutex0x  void *
finalizing  0   int
+   ceval   {tracing_possible=0 eval_breaker={_value=0 } 
pending={finishing=0 lock=0x00e59390 calls_to_do={_value=...} ...} }   
_ceval_state
+   gc  {trash_delete_later=0x  
trash_delete_nesting=0 enabled=1 ...} _gc_runtime_state
+   modules 0x00bf1228 {ob_refcnt=3 ob_type=0x7a0b1178 
{python39_d.dll!_typeobject PyDict_Type} {ob_base={ob_base=...} ...} }   
_object *
+   modules_by_index0x00750058 {ob_refcnt=1 
ob_type=0x7a0b8210 {python39_d.dll!_typeobject PyList_Type} 
{ob_base={ob_base=...} ...} }   _object *
+   sysdict 0x00bf1298 {ob_refcnt=2 ob_type=0x7a0b1178 
{python39_d.dll!_typeobject PyDict_Type} {ob_base={ob_base=...} ...} }   
_object *
+   builtins0x00bf1f48 {ob_refcnt=88 ob_type=0x7a0b1178 
{python39_d.dll!_typeobject PyDict_Type} {ob_base={ob_base=...} ...} }  
_object *
+   importlib   0x00c0df60 {ob_refcnt=28 ob_type=0x7a0b92d0 
{python39_d.dll!_typeobject PyModule_Type} {ob_base={ob_base=...} ...} }
_object *
num_threads 0   long
pythread_stacksize  0   unsigned int
+   codec_search_path   0x00c4a260 {ob_refcnt=1 
ob_type=0x7a0b8210 {python39_d.dll!_typeobject PyList_Type} 
{ob_base={ob_base=...} ...} }   _object *
+   codec_search_cache  0x00c1f0d8 {ob_refcnt=1 
ob_type=0x7a0b1178 {python39_d.dll!_typeobject PyDict_Type} 
{ob_base={ob_base=...} ...} }   _object *
+   codec_error_registry0x00c14f10 {ob_refcnt=1 
ob_type=0x7a0b1178 {python39_d.dll!_typeobject PyDict_Type} 
{ob_base={ob_base=...} ...} }   _obj

[issue40079] NULL pointer deref on error path in _ssl debughelpers.c

2020-03-26 Thread Alexander Riccio


New submission from Alexander Riccio :

At line 138 in debughelpers.c, ssl_obj, which was set to NULL on line 122, is 
dereferenced. 

I think the original intent was to actually bubble the error up through the ssl 
object.



Full function:

static void
_PySSL_keylog_callback(const SSL *ssl, const char *line)
{
PyGILState_STATE threadstate;
PySSLSocket *ssl_obj = NULL;  /* ssl._SSLSocket, borrowed ref */
int res, e;
static PyThread_type_lock *lock = NULL;

threadstate = PyGILState_Ensure();

/* Allocate a static lock to synchronize writes to keylog file.
 * The lock is neither released on exit nor on fork(). The lock is
 * also shared between all SSLContexts although contexts may write to
 * their own files. IMHO that's good enough for a non-performance
 * critical debug helper.
 */
if (lock == NULL) {
lock = PyThread_allocate_lock();
if (lock == NULL) {
PyErr_SetString(PyExc_MemoryError, "Unable to allocate lock");
PyErr_Fetch(_obj->exc_type, _obj->exc_value,
_obj->exc_tb);
return;
}
}

ssl_obj = (PySSLSocket *)SSL_get_app_data(ssl);
assert(PySSLSocket_Check(ssl_obj));
if (ssl_obj->ctx->keylog_bio == NULL) {
return;
}

PySSL_BEGIN_ALLOW_THREADS
PyThread_acquire_lock(lock, 1);
res = BIO_printf(ssl_obj->ctx->keylog_bio, "%s\n", line);
e = errno;
(void)BIO_flush(ssl_obj->ctx->keylog_bio);
PyThread_release_lock(lock);
PySSL_END_ALLOW_THREADS

if (res == -1) {
errno = e;
PyErr_SetFromErrnoWithFilenameObject(PyExc_OSError,
 ssl_obj->ctx->keylog_filename);
PyErr_Fetch(_obj->exc_type, _obj->exc_value, _obj->exc_tb);
}
PyGILState_Release(threadstate);
}

--
assignee: christian.heimes
components: SSL
messages: 365114
nosy: Alexander Riccio, christian.heimes
priority: normal
severity: normal
status: open
title: NULL pointer deref on error path in _ssl debughelpers.c
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40079>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40020] growable_comment_array_add leaks, causes crash

2020-03-19 Thread Alexander Riccio


Change by Alexander Riccio :


--
keywords: +patch
pull_requests: +18442
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19083

___
Python tracker 
<https://bugs.python.org/issue40020>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40020] growable_comment_array_add leaks, causes crash

2020-03-19 Thread Alexander Riccio


Alexander Riccio  added the comment:

Sidenote: visual studio was misleading and made this look like a use-after-free 
for a little while, which was interesting.

--
nosy: +pablogsal

___
Python tracker 
<https://bugs.python.org/issue40020>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40020] growable_comment_array_add leaks, causes crash

2020-03-19 Thread Alexander Riccio


New submission from Alexander Riccio :

growable_comment_array_add in parsetok.c incorrectly uses realloc, which leaks 
the array when allocation fails, and then causes a null pointer deref crash 
later when the array is freed in growable_comment_array_deallocate (the array 
pointer is dereferenced, passing null to free is fine).

It's unlikely that this codepath is reached in normal use, since type comments 
need to be turned on (via the PyCF_TYPE_COMMENTS compiler flag), but I've 
managed to replicate the issue by injecting faults with Application Verifier. 
It's easiest to cause it to fail with a very large number of type comments, but 
presumably this could also happen with some form of heap fragmentation.

The buggy code is:

static int
growable_comment_array_add(growable_comment_array *arr, int lineno, char 
*comment) {
if (arr->num_items >= arr->size) {
arr->size *= 2;
arr->items = realloc(arr->items, arr->size * sizeof(*arr->items));
if (!arr->items) {
return 0;
}
}

arr->items[arr->num_items].lineno = lineno;
arr->items[arr->num_items].comment = comment;
arr->num_items++;
return 1;
}


and the correct code would be something like:

static int
growable_comment_array_add(growable_comment_array *arr, int lineno, char 
*comment) {
if (arr->num_items >= arr->size) {
arr->size *= 2;
void* new_items_array = realloc(arr->items, arr->size * 
sizeof(*arr->items));
if (!new_items_array) {
return 0;
}
arr->items = new_items_array;
}

arr->items[arr->num_items].lineno = lineno;
arr->items[arr->num_items].comment = comment;
arr->num_items++;
return 1;
}

--
components: Interpreter Core
messages: 364644
nosy: Alexander Riccio, benjamin.peterson
priority: normal
severity: normal
status: open
title: growable_comment_array_add leaks, causes crash
type: crash
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40020>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2020-03-19 Thread Alexander Riccio


Alexander Riccio  added the comment:

Ok, so I finally have some proper time to work on this. How would people (who 
are higher up in python than me, obviously) feel about suppressing most of the 
warnings via a user macro in Visual Studio? I've found that it's quite easy to 
add a macro to the project properties (i.e. pyproject), and have it import into 
the "Disable specific warnings" build options. This keeps our build files 
hand-crafted (let's keep the hipsters happy!), and consistent. By doing this, I 
can get it down to ~100 warnings, which are essentially code that we may want 
to keep an eye on.

The following warnings are essentially always spurious for python. C4100 is
everywhere, and is nearly always benign, but would still require a lot of
careful inspection to determine it its truly on purpose. C4127 is fine since C
doesn't have `constexpr`, let alone `if constexpr`, and it's just easier to
use if statements than  `#ifdef`s everywhere. C4152 is used in every module
exports array. 

C4100 (unreferenced formal parameter) 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4100?view=vs-2019
C4115 'type' : named type definition in parentheses 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-levels-1-and-4-c4115?view=vs-2019
C4127 conditional expression is constant 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4127?view=vs-2019
C4131 'function' : uses old-style declarator 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4131?view=vs-2019
C4152 non standard extension, function/data ptr conversion in expression 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4152?view=vs-2019

These are usually spurious, since zero-sized arrays are common across the C
world, and is one of several ways of implementing dynamic structs (e.g. unsized
arrays, ANYSIZE_ARRAY, etc...). C4204 and C4221 exist all over the place, and
are probably fine as long as we don't make any lifetime mistakes, which of
course is a solved problem in C. Instances of C4244 should each be individually
inspected carefully since they could be bugs, but there are way too many of
them to allow the warning right now. 

C4200 nonstandard extension used : zero-sized array in struct/union 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-levels-2-and-4-c4200?view=vs-2019
C4201 nonstandard extension used : nameless struct/union 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4201?view=vs-2019
C4204 nonstandard extension used : non-constant aggregate initializer 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4204?view=vs-2019
C4221 nonstandard extension used : 'identifier' : cannot be initialized using 
address of automatic variable 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4221?view=vs-2019
C4232 nonstandard extension used : 'identifier' : address of dllimport 
'dllimport' is not static, identity not guaranteed 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4232?view=vs-2019
 
C4244 'conversion' conversion from 'type1' to 'type2', possible loss of data 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-levels-3-and-4-c4244?view=vs-2019
C4245 'conversion' : conversion from 'type1' to 'type2', signed/unsigned 
mismatch 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4245?view=vs-2019


Instances of C4389 should each be individually inspected carefully since they
could be bugs, but there are way too many of them to allow the warning right 
now.

C4389 'operator' : signed/unsigned mismatch 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4389?view=vs-2019


Instances of C4456 and C4457 are likely ok given the length of many CPython
functions, but should be inspected for mistakes. There are lots of places
where `i` and such are used as variables, and it's there's no way to know
if the author intended it.
C4456 declaration of 'identifier' hides previous local declaration 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4456?view=vs-2019
C4457 declaration of 'identifier' hides function parameter 
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-4-c4457?view=vs-2019


Instances of C4701 occur pretty frequently around code that initializes C
objects with the funky PyArg_ParseTuple format string mechanism, since the
compiler has no built in support for understanding custom format strings.
Personally, I would support zero-initializing these variables. Woul

[issue36790] test_asyncio fails with application verifier!

2019-05-14 Thread Alexander Riccio


Alexander Riccio  added the comment:

It's part of the Windows SDK, and is installed with it. To enable for this 
error, add the Python executable in Application Verifier, and check the Handles 
box.

--

___
Python tracker 
<https://bugs.python.org/issue36790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36790] test_asyncio fails with application verifier!

2019-05-03 Thread Alexander Riccio


Alexander Riccio  added the comment:

Hmm, proceeding a bit further pointed to finish_recv in windows_events.py

--
Added file: https://bugs.python.org/file48299/python_invalid_handle.PNG

___
Python tracker 
<https://bugs.python.org/issue36790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36790] test_asyncio fails with application verifier!

2019-05-03 Thread Alexander Riccio


New submission from Alexander Riccio :

I compiled PCBuild Debug x64 from an updated clone of upstream, and when 
running the testsuite under Application Verifier with handle verification, the 
test triggers an invalid handle access by passing an invalid overlapped handle 
to CancelIoEx with this code: Py_CancelIoEx(self->handle, >overlapped) 
(where self->handle appears to be the offending variable).

I have no idea who's calling _overlapped.cancel, and a quick spelunking through 
the codebase only confuses me more.

--
components: Tests
files: python_test_invalid_handle_failure.TXT
messages: 341365
nosy: Alexander Riccio
priority: normal
severity: normal
status: open
title: test_asyncio fails with application verifier!
versions: Python 3.8
Added file: 
https://bugs.python.org/file48298/python_test_invalid_handle_failure.TXT

___
Python tracker 
<https://bugs.python.org/issue36790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2019-04-18 Thread Alexander Riccio


Alexander Riccio  added the comment:

One more thing, after I ran code analysis:

This is obviously a potential memory leak:
Warning C6308   'realloc' might return null pointer: assigning null pointer to 
'arr->items', which is passed as an argument to 'realloc', will cause the 
original memory block to be leaked.
cpython\parser\parsetok.c   38  


I found some sketchy code that isn't obviously correct. Here are a few of the 
warnings:


Warning C6294   Ill-defined for-loop:  initial condition does not satisfy test. 
 Loop body not executed.
\cpython\modules\gcmodule.c 1377


Warning C6011   Dereferencing NULL pointer 'cmdline'. See line 230 for an 
earlier location where this can occur
\cpython\python\preconfig.c 242 
(cmdline is checked for nullness after several uses?)


And finally there's one warning where I have no clue what's going on:

Warning C6386   Buffer overrun while writing to 'x_digits':  the writable size 
is '10' bytes, but '286331156' bytes might be written.   
\cpython\objects\longobject.c   2972

--
Added file: https://bugs.python.org/file48276/cpython_xdigits_overrun.PNG

___
Python tracker 
<https://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2019-04-18 Thread Alexander Riccio


Alexander Riccio  added the comment:

I decided to come back to this after a python meetup last night. By messing 
with this a bit, building in VS2019 with /W4, I see that fully 2/3rds of the 
total warnings are from two specific warnings:

C4100 (unreferenced formal parameter)
C4127 (conditional expression is constant)

...This seems to be a stylistic thing across the codebase. If it were a new 
codebase, I'd simply recommend not giving unreferenced formal parameters a 
variable name - the typical way of doing it - but there's no way anybody is 
gonna care enough to comment them out across the whole codebase. C4127 is not A 
Big Deal, since dispatching based on data type sizes in conditionals is just 
the easiest way to do The Right Thing in C.

The rest of the warnings are mostly datatype coercions ('=': conversion from 
'int' to 'char', possible loss of data), old style declarators, and a bunch of 
type indirection mismatches ('function': 'volatile int *' differs in 
indirection to slightly different base types from 'volatile long *'), type cast 
truncation ('type cast': truncation from 'volatile __int64' to 'PyThreadState 
*'), named type declarations in parenthesis ('timeval': named type definition 
in parentheses), and assignments in conditionals (which I don't like, but are 
not a huge deal).

There really are only a few things that actually look sketchy. For example, in 
run_child (launcher.c), SetInformationJobObject is passing sizeof(info) as 
cbJobObjectInformationLength, where it should instead be the local variable rc.

--

___
Python tracker 
<https://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26137] [idea] use the Microsoft Antimalware Scan Interface

2016-07-02 Thread Alexander Riccio

Alexander Riccio added the comment:

We might want to use some kind of Group Policy setting, for the same reason 
that many Windows security configuration options are there, and that DoD STIGs 
for Windows https://www.stigviewer.com/stig/windows_8_8.1/ are almost totally 
about configuring Group Policy settings.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26137>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26137] [idea] use the Microsoft Antimalware Scan Interface

2016-07-02 Thread Alexander Riccio

Alexander Riccio added the comment:

It's not just Stuxnet, as at least one other Advanced Persistent Threat uses 
that tactic. An APT (likely Russian intelligence) recently used encoded 
PowerShell to break into the Democratic National Committe: 
https://www.crowdstrike.com/blog/bears-midst-intrusion-democratic-national-committee/

>From that article:

> This one-line powershell command, stored only in WMI database, establishes an 
> encrypted connection to C2 and downloads additional powershell modules from 
> it, executing them in memory.

(As a fun coincidence, they also used py2exe to distribute other modules, which 
is kinda like a separate interpreter using safe_exec)

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26137>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26139] libmpdec: disable /W4 warning (non-standard dllimport behavior)

2016-01-18 Thread Alexander Riccio

Alexander Riccio added the comment:

> Please stop educating us.

Sorry, not what was intended! Tone transmits poorly.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26139>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2016-01-17 Thread Alexander Riccio

Alexander Riccio added the comment:

If there are few enough instances, then using a #pragma warning(suppress:4232) 
is probably the best idea.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26137] [idea] use the Microsoft Antimalware Scan Interface

2016-01-17 Thread Alexander Riccio

New submission from Alexander Riccio:

I'm really not sure what it'd look like, or how it'd work, but CPython should 
take advantage of Microsoft's Antimalware Scan Interface, which is new to 
Windows 10. It's designed for applications like interpreters, which can execute 
u trusted code that may not be visible to antimalware.

--
components: Interpreter Core, Windows
messages: 258455
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: [idea] use the Microsoft Antimalware Scan Interface
type: enhancement

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26137>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26137] [idea] use the Microsoft Antimalware Scan Interface

2016-01-17 Thread Alexander Riccio

Alexander Riccio added the comment:

See also: "Security Focus: Defending PowerShell with the Anti-Malware Scan 
Interface (AMSI)" 
http://blogs.technet.com/b/poshchap/archive/2015/10/16/security-focus-defending-powershell-with-windows-defender.aspx

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26137>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26137] [idea] use the Microsoft Antimalware Scan Interface

2016-01-17 Thread Alexander Riccio

Alexander Riccio added the comment:

When I say "I'm really not sure what it'd look like, or how it'd work" I mean 
at the C level. At a higher level, there are many places that I imagine are 
good places to use AMSI: Perhaps expressions passed in from the command line 
(-c) should be scanned; maybe pip packages should be scanned on installation, 
and maybe untrusted packages should be scanned right before execution...

Then it's a matter of calling the AMSI APIs in the right places; I don't know 
where those are.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26137>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26137] [idea] use the Microsoft Antimalware Scan Interface

2016-01-17 Thread Alexander Riccio

Alexander Riccio added the comment:

See "Windows 10 to offer application developers new malware defenses" 
https://blogs.technet.microsoft.com/mmpc/2015/06/09/windows-10-to-offer-application-developers-new-malware-defenses/
 for an example of how AMSI works with PowerShell.

I think the applicability to CPython is self-evident.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26137>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2015-12-21 Thread Alexander Riccio

Alexander Riccio added the comment:

> This should be about a 2 line change, but the current patch is several 
> hundred lines of spam.

I agree, but wasn't immediately sure how to do so.

Unfortunately, I've been working on other things, and I'm not sure when I'll be 
able to finish this.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25891] Stray variable meth_idx in enable_symlink

2015-12-16 Thread Alexander Riccio

New submission from Alexander Riccio:

See: https://hg.python.org/cpython/file/tip/Modules/posixmodule.c#l12383

The variable int meth_idx is initialized but not used. I have no idea how it 
got there.

--
components: Windows
messages: 256553
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Stray variable meth_idx in enable_symlink
type: compile error

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25891>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25893] Second variable DWORD reqdSize in getpythonregpath is initialized but not used

2015-12-16 Thread Alexander Riccio

New submission from Alexander Riccio:

See: https://hg.python.org/cpython/file/tip/PC/getpathp.c#l324

The second variable named `reqdSize` in getpythonregpath is initialized but not 
used.

--
components: Windows
messages: 256555
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Second variable DWORD reqdSize in getpythonregpath is initialized but 
not used
type: compile error

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25893>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25890] PyObject *po in _listdir_windows_no_opendir is initialized but not used

2015-12-16 Thread Alexander Riccio

New submission from Alexander Riccio:

See: https://hg.python.org/cpython/file/tip/Modules/posixmodule.c#l3466

The variable PyObject *po in _listdir_windows_no_opendir is initialized but not 
used. Given that there's a variable named po_wchars, and two PyObject 
variables, I'm going to guess that the programmer forgot about `v`.

--
components: Windows
messages: 256549
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: PyObject *po in _listdir_windows_no_opendir is initialized but not used
type: compile error

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25890] PyObject *po in _listdir_windows_no_opendir is initialized but not used

2015-12-16 Thread Alexander Riccio

Alexander Riccio added the comment:

(in the same function, char *bufptr is ALSO unused)

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25890>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25892] PyObject *exc in encode_code_page_strict is initialized but not used

2015-12-16 Thread Alexander Riccio

New submission from Alexander Riccio:

See: https://hg.python.org/cpython/file/tip/Objects/unicodeobject.c#l7335

The variable PyObject *exc in encode_code_page_strict is initialized but not 
used.

--
components: Windows
messages: 256554
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: PyObject *exc in encode_code_page_strict is initialized but not used
type: compile error

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25892>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2015-12-16 Thread Alexander Riccio

Alexander Riccio added the comment:

> The problem with this bug report is that there is little chance that it gets 
> resolved in the near term, and it's quite possible that it will stay open for 
> years. Somebody would have to sit down and start producing patches to fix 
> these warnings correctly, before the actual patch can be applied.


That's perfectly reasonable. In the mean time, we may be able to dramatically 
reduce the timetable by suppressing warnings - C4232 for example - which don't 
offer anything on a case-by-case basis.

C4232(*), for example, warns any time we use the address of a `dllimport`ed 
function. mpd_mallocfunc, mpd_free, and friends - see 
https://hg.python.org/cpython/file/tip/Modules/_decimal/libmpdec/memory.c#l44 - 
are assigned malloc, free, etc... but is it safe to ignore?



And then there are warnings that are safe to ignore in certain places, like 
C4310 in the _Py_ERROR_SURROGATEESCAPE case of PyUnicode_DecodeASCII in 
unicodeobject.c 
(https://hg.python.org/cpython/file/tip/Objects/unicodeobject.c#l6897):

if (error_handler == _Py_ERROR_REPLACE)
PyUnicode_WRITE(kind, data, writer.pos, 0xfffd);

...(here, the constant value 0xfffd is cast to a Py_UCS1, truncating it, inside 
a macro) but are not valid in others, like these insertint calls in 
PyInit_msvcrt, in msvcrtmodule.c 
(https://hg.python.org/cpython/file/tip/PC/msvcrtmodule.c#l538):

insertint(d, "CRTDBG_FILE_STDERR", (int)_CRTDBG_FILE_STDERR);
insertint(d, "CRTDBG_FILE_STDOUT", (int)_CRTDBG_FILE_STDOUT);
insertint(d, "CRTDBG_REPORT_FILE", (int)_CRTDBG_REPORT_FILE);


...(here, _CRTDBG_FILE_STDERR, _CRTDBG_FILE_STDOUT, and _CRTDBG_REPORT_FILE, 
are _HFILEs, which is a void*, here {((_HFILE)(intptr_t)-4), 
((_HFILE)(intptr_t)-5), and ((_HFILE)(intptr_t)-6)} respectively; the void* is 
truncated to an int in 64 bit builds)

(*)C4232, "nonstandard extension used : 'identifier' : address of dllimport 
'dllimport' is not static, identity not guaranteed",  
(https://msdn.microsoft.com/en-us/library/9a1sy630.aspx)

I will try to sort a few of the easy categories, so we can discuss more 
generally.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25889] Find_BOM accepts a char*, but is passed an unsigned char*; and related usage

2015-12-16 Thread Alexander Riccio

New submission from Alexander Riccio:

This is safe, but warns on /W4. In maybe_handle_shebang, an unsigned char* is 
passed to find_BOM, which accepts a char* 
(https://hg.python.org/cpython/file/tip/PC/launcher.c#l1139).

Without an explicit cast, this generates a warning:

10>..\PC\launcher.c(1139): warning C4057: 'function': 'char *' differs in 
indirection to slightly different base types from 'unsigned char [256]'

for `bom = find_BOM(buffer);`

Similarly, assigning start to buffer generates warnings:

10>..\PC\launcher.c(1141): warning C4057: '=': 'char *' differs in indirection 
to slightly different base types from 'unsigned char *'

for `start = buffer;`


10>..\PC\launcher.c(1148): warning C4057: '=': 'char *' differs in indirection 
to slightly different base types from 'unsigned char *'
for `start = [bom->length];`

--
components: Windows
messages: 256547
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Find_BOM accepts a char*, but is passed an unsigned char*; and related 
usage
type: compile error

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25889>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2015-12-16 Thread Alexander Riccio

Alexander Riccio added the comment:

Cut out more noisy warnings.

--
Added file: http://bugs.python.org/file41333/W4_v3.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-15 Thread Alexander Riccio

Alexander Riccio added the comment:

I'll open up a new issue for /W4, and deal with that first.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2015-12-15 Thread Alexander Riccio

New submission from Alexander Riccio:

This issue is related to Issue25847.

Compiling at /W4 is generally a good idea. It's an industry best practice, and 
even though I don't expect disagreement, I'll throw in a few coding standard 
links:

https://www.securecoding.cert.org/confluence/display/c/MSC00-C.+Compile+cleanly+at+high+warning+levels

https://books.google.com/books?id=pDsFCAAAQBAJ=PA557=PA557=compile+at+highest+warning+level=bl=RlKoHFuuWW=mcz-wqdpf3MhiyGZSYKFvpTmd9A=en=X=0ahUKEwjkhN2Rh9_JAhXIth4KHTTbCiMQ6AEIgwEwEQ#v=onepage=compile%20at%20highest%20warning%20level=false

http://programmers.stackexchange.com/questions/232626/is-it-a-good-practice-to-choose-the-highest-warning-level-in-c-programming

http://ptgmedia.pearsoncmg.com/images/0321113586/items/sutter_item1.pdf


...so I was surprised to discover that the Windows build of CPython compiles at 
/W3!

Bumping it up to /W4 produces ~9,000 warnings (up from ~20).


The patch that I'll include with this issue (first iteration) bumps the warning 
level up to /W4, and I've disabled warnings that (appear) aren't useful. That 
suppresses ~8,000 of those warnings.

The patch isn't quite yet perfect, as Visual Studio made changes that I didn't 
ask for, such as:

-
+

...but that's about it. I had to use `hg diff -w`, because Visual Studio also 
decided to change the spacing in all of the .vcxproj files.

--
messages: 256494
nosy: Alexander Riccio
priority: normal
severity: normal
status: open
title: CPython on Windows builds with /W3, not /W4

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-15 Thread Alexander Riccio

Alexander Riccio added the comment:

See Issue25878.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2015-12-15 Thread Alexander Riccio

Alexander Riccio added the comment:

The warnings that I've disabled are:

C4054, "'conversion' : from function pointer 'type1' to data pointer 'type2'": 
https://msdn.microsoft.com/en-us/library/07d15ax5(v=vs.90).aspx

I disabled 4054because there are lots of void* to (somefuncptr) conversions 
in CPython. I couldn't see any problems with them, and they all seemed 
necessary.


C4100, "'identifier' : unreferenced formal parameter": 
https://msdn.microsoft.com/en-us/library/26kb9fy0.aspx

I disabled C4100 because there are thousands of places in CPython where 
function parameters aren't referenced. Some of these could actually be bugs 
(!!), but right now there are too many benign cases of this to be of use. 
Commenting out parameter names where they're intentionally not referenced is an 
elegant way of suppressing this warning while documenting intent.


C4115, "'type' : named type definition in parentheses": 
https://msdn.microsoft.com/en-us/library/hhzc0806(v=vs.90).aspx

I disabled C4115 because CPython appears to trigger it in benign 
conditions. This warning triggers when a function accepts a structure as an 
argument, and that structure type has not yet been properly declared.


C4127, "conditional expression is constant": 
https://msdn.microsoft.com/en-us/library/6t66728h(v=vs.90).aspx

I disabled C4127 because the do { } while (1) pattern is quite prevalent in 
CPython, and is unlikely to yield any useful warnings.


C4131, "'function' : uses old-style declarator": 
https://msdn.microsoft.com/en-us/library/b92s55e9(v=vs.90).aspx

I disabled C4131 because CPython includes (third party) code that uses the 
silly old style C function declaration form.


C4152, "non standard extension, function/data ptr conversion in expression": 
https://msdn.microsoft.com/en-us/library/bacb5038(v=vs.90).aspx

I disabled C4152 for the same reason that I disabled C4054.


C4200, "nonstandard extension used : zero-sized array in struct/union": 
https://msdn.microsoft.com/en-us/library/79wf64bc.aspx

I disabled C4200 in a few projects, because this is a very common way of 
reducing memory allocations. Block allocations are trickier to correctly use, 
and don't have a standardized, safe, mechanism (not until C++17, at least), but 
they're just too darn useful & common to warn about every single use.


C4204, "nonstandard extension used : non-constant aggregate initializer": 
https://msdn.microsoft.com/en-us/library/6b73z23c.aspx

I disabled C4204 because CPython frequently initializes a struct with local 
variables. This is perfectly reasonable, despite ANSI C incompatibility.


C4244, "'conversion' conversion from 'type1' to 'type2', possible loss of 
data": https://msdn.microsoft.com/en-us/library/th7a07tz.aspx

I disabled C4244, if I remember correctly, because all instances appeared 
safe. I should revisit this one.


C4267, "'var' : conversion from 'size_t' to 'type', possible loss of data": 
https://msdn.microsoft.com/en-us/library/6kck0s93.aspx

I disabled C4267, if I remember correctly, because all instances appeared 
safe. I should revisit this one.


C4706, "assignment within conditional expression": 
https://msdn.microsoft.com/en-us/library/7hw7c1he.aspx

I disabled C4706 because there's lots of code in CPython that allocates 
memory inside the conditional (i.e. `if (!(self = PyObject_New(Pdata, 
_Type)))` from Pdata_New in _pickle.c), which is perfectly valid code. 
Even if it makes me nauseous. 



Running `build.bat -d -v --no-ssl --no-tkinter -c Debug -p x64` produces 524 
warnings.

--
keywords: +patch
Added file: http://bugs.python.org/file41321/W4_v2.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2015-12-15 Thread Alexander Riccio

Alexander Riccio added the comment:

I've added the text build output.

--
Added file: http://bugs.python.org/file41322/W4_v2_build_output

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2015-12-15 Thread Alexander Riccio

Changes by Alexander Riccio <test35...@gmail.com>:


--
nosy: +paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25878] CPython on Windows builds with /W3, not /W4

2015-12-15 Thread Alexander Riccio

Changes by Alexander Riccio <test35...@gmail.com>:


--
components: +Windows

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25878>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-14 Thread Alexander Riccio

Alexander Riccio added the comment:

> That is, (as I undersatnd it) we've done a lot of work to not have compiler 
> warnings generated during compilation, and we don't want to backtrack on that.

Well, as-is, simply building as x64 generates a bunch of warnings, so it's not 
*quite* clean. I totally understand the need to keep warning-clean, but, well:

..\Modules\_pickle.c(5087): warning C4244: 'function': conversion from 
'Py_ssize_t' to 'int', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Modules\posixmodule.c(3321): warning C4267: 'function': conversion from 
'size_t' to 'int', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Modules\_tracemalloc.c(67): warning C4359: '': Alignment 
specifier is less than actual alignment (8), and will be ignored. 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Modules\zipimport.c(914): warning C4244: 'function': conversion from 
'Py_ssize_t' to 'long', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Objects\codeobject.c(111): warning C4244: '=': conversion from 'Py_ssize_t' 
to 'unsigned char', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Objects\funcobject.c(635): warning C4244: 'function': conversion from 
'Py_ssize_t' to 'int', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Objects\funcobject.c(636): warning C4244: 'function': conversion from 
'Py_ssize_t' to 'int', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\PC\winreg.c(885): warning C4311: 'type cast': pointer truncation from 'void 
*' to 'DWORD' [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\PC\getpathp.c(144): warning C4267: '=': conversion from 'size_t' to 'int', 
possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\PC\msvcrtmodule.c(391): warning C4312: 'type cast': conversion from 'int' to 
'_HFILE' of greater size [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\PC\msvcrtmodule.c(391): warning C4311: 'type cast': pointer truncation from 
'_HFILE' to 'long' [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\PC\msvcrtmodule.c(538): warning C4311: 'type cast': pointer truncation from 
'_HFILE' to 'int' [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\PC\msvcrtmodule.c(539): warning C4311: 'type cast': pointer truncation from 
'_HFILE' to 'int' [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\PC\msvcrtmodule.c(540): warning C4311: 'type cast': pointer truncation from 
'_HFILE' to 'int' [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\ceval.c(4826): warning C4244: '=': conversion from 'Py_ssize_t' to 
'int', possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\ceval.c(5021): warning C4244: '=': conversion from 'Py_ssize_t' to 
'int', possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\compile.c(480): warning C4312: 'type cast': conversion from 'unsigned 
int' to 'void *' of greater size [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\compile.c(481): warning C4312: 'type cast': conversion from 'unsigned 
int' to 'void *' of greater size [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\compile.c(482): warning C4312: 'type cast': conversion from 'unsigned 
int' to 'void *' of greater size [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\compile.c(3155): warning C4244: 'initializing': conversion from 
'Py_ssize_t' to 'int', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\compile.c(3385): warning C4244: 'initializing': conversion from 
'Py_ssize_t' to 'int', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]


..\Python\peephole.c(482): warning C4244: '=': conversion from 'Py_ssize_t' to 
'int', possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\peephole.c(535): warning C4244: '=': conversion from 'Py_ssize_t' to 
'int', possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\peephole.c(553): warning C4244: '=': conversion from 'Py_ssize_t' to 
'int', possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\peephole.c(581): warning C4244: '=': conversion from 'Py_ssize_t' to 
'int', possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\peephole.c(592): warning C4244: '=': conversion from 'Py_ssize_t' to 
'unsigned char', possible loss of data 
[C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\peephole.c(625): warning C4244: '=': conversion from 'Py_ssize_t' to 
'int', possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Python\peephole.c(639): warning C4244: '-=': conversion from 'Py_ssize_t' to 
'int', possible loss of data [C:\PythonDev\repo\PCbuild\pythoncore.vcxproj]

..\Modules\expat\xmlparse.c(1844): warning C4244: 'return': conversion from 
'__int64' to 'XML_Index', possible loss of data 
[C:\PythonDev\repo\PCbuild\_elementtree.vcxproj

[issue25847] CPython not using Visual Studio code analysis!

2015-12-14 Thread Alexander Riccio

Alexander Riccio added the comment:

> In which direction do you find us to be mad?

That's really quite a low warning level! For a large project, I can't imagine 
anything less than /W4!

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-14 Thread Alexander Riccio

Alexander Riccio added the comment:

Actually, hmm... the very naive version *DOES NOT* work. Grr.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-14 Thread Alexander Riccio

Alexander Riccio added the comment:

Hold on... CPython builds at /W3???!? What is this madness??!?

--
Added file: http://bugs.python.org/file41312/CPythonW3.PNG

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-14 Thread Alexander Riccio

Alexander Riccio added the comment:

> OK, let's move this to patch needed, then, and see if anyone is ambitious 
> enough to do the work needed to make it useful to us :)


I can try and hack it in, just as proof of concept. I think I should just be 
able to add something like:

/p:EnablePREfast=%EnablePREfast%^

...along with something like:

if "%~1"=="--enable-code-analysis" (set EnablePREfast=true) & shift & goto 
CheckOpts




God I hate batch scripting.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-14 Thread Alexander Riccio

Alexander Riccio added the comment:

Yup, the very naive version works.

--
keywords: +patch
Added file: http://bugs.python.org/file41311/EnableCodeAnalysis.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25846] Use of Py_ARRAY_LENGTH on pointer in posixmodule.c, win32_wchdir

2015-12-13 Thread Alexander Riccio

Alexander Riccio added the comment:

> Are you aware of the Coverity program? Last time I heard about Coverity, 
> CPython had 0 bug found by Coverity ;-)

Yup, see Issue25847.

> The sad part is that Py_ARRAY_LENGTH() is written for static analysis

Sadly, yeah. MSVC, when compiling as C++, should actually catch this, but 
CPython precludes that.

--
resolution: fixed -> 
type: behavior -> 

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25846>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-13 Thread Alexander Riccio

Alexander Riccio added the comment:

> Is analyze something that can be used from the command line only, or does it 
> require the GUI?

You can do it from the command line - Chrome/chromium makes use of it as such.
See: https://code.google.com/p/chromium/issues/detail?id=427616


The /analyze option is documented here:
https://msdn.microsoft.com/en-us/library/ms173498.aspx

/analyze:WX- Prevents compilation failure when compiling with /WX (warn as 
errors) and /analyze warnings are disabled the same way that normal warnings 
are.

For example, /analyze an extremely large number of variable shadowing issues, 
which I think should be suppressed (as CPython's code base tolerates them?), to 
get to the more important issues. 


> Also, we aren't likely to make the code more complex in order to deal with 
> shortcomings in analyze's algorithms

I assume you're referring to the out-of-bounds in complex conditions? I can't 
imagine how making the code *more* complex would help :)


> I'm surprised it is catching things that coverity doesn't.

Every tool has its strengths and weaknesses; I am, however surprised that 
coverity didn't catch these issues, as they're common, and platform agnostic.

/analyze can pick up many issues that coverity doesn't, simply because /analyze 
understands SAL, so it understands how the Windows API is supposed to be used.

Also: Of the three issues that I opened, one is already fix, and two are in the 
pipeline. Impressive!

--
components: +Build -Windows

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25844] Pylauncher, launcher.c: Assigning NULL to a pointer instead of testing against NULL

2015-12-11 Thread Alexander Riccio

New submission from Alexander Riccio:

I found this while writing up a separate bug (CPython doesn't use static 
analysis!).

In PC/launcher.c, get_env has a bug:

/* Large environment variable. Accept some leakage */
wchar_t *buf2 = (wchar_t*)malloc(sizeof(wchar_t) * (result+1));
if (buf2 = NULL) {
error(RC_NO_MEMORY, L"Could not allocate environment buffer");
}
GetEnvironmentVariableW(key, buf2, result);
return buf2;

See: https://hg.python.org/cpython/file/tip/PC/launcher.c#l117


Instead of `buf2 == NULL`, Vinay Sajip wrote `buf2 = NULL`. The commit where 
the error was introduced: https://hg.python.org/cpython/rev/4123e002a1af

Thus, whatever value was in buf2 is lost, the branch is NOT taken (because buf2 
evaluates to false), and GetEnvironmentVariableW will (probably) cause an 
access violation. 


Compiling with /analyze found this quite easily:

c:\pythondev\repo\pc\launcher.c(117): warning C6282: Incorrect operator:  
assignment of constant in Boolean context. Consider using '==' instead.

--
components: Windows
messages: 256254
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, vinay.sajip, 
zach.ware
priority: normal
severity: normal
status: open
title: Pylauncher, launcher.c: Assigning NULL to a pointer instead of testing 
against NULL

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25844>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25844] Pylauncher, launcher.c: Assigning NULL to a pointer instead of testing against NULL

2015-12-11 Thread Alexander Riccio

Changes by Alexander Riccio <test35...@gmail.com>:


--
type:  -> crash

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25844>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25846] Use of Py_ARRAY_LENGTH on pointer in posixmodule.c, win32_wchdir

2015-12-11 Thread Alexander Riccio

New submission from Alexander Riccio:

I found this while writing up a separate bug (CPython doesn't use static 
analysis!).

In modules/posixmodule.c, win32_wchdir uses Py_ARRAY_LENGTH on a wchar_t*:

wchar_t _new_path[MAX_PATH], *new_path = _new_path;
int result;
wchar_t env[4] = L"=x:";

if(!SetCurrentDirectoryW(path))
return FALSE;
result = GetCurrentDirectoryW(Py_ARRAY_LENGTH(new_path), new_path);


...instead of using Py_ARRAY_LENGTH(_new_path), the programmer wrote 
Py_ARRAY_LENGTH(new_path), doesn't work on pointers:

/* Get the number of elements in a visible array

   This does not work on pointers, or arrays declared as [], or function
   parameters. With correct compiler support, such usage will cause a build
   error (see Py_BUILD_ASSERT_EXPR).

   Written by Rusty Russell, public domain, http://ccodearchive.net/
*/
#define Py_ARRAY_LENGTH(array) \
(sizeof(array) / sizeof((array)[0]))


The same issue occurs two lines later:

if (result > Py_ARRAY_LENGTH(new_path)) {



Compiling with /analyze found this quite easily:

c:\pythondev\repo\modules\posixmodule.c(1354): warning C6384: Dividing sizeof a 
pointer by another value.

--
components: Windows
messages: 256260
nosy: Alexander Riccio, larry, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Use of Py_ARRAY_LENGTH on pointer in posixmodule.c, win32_wchdir

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25846>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25845] _ctypes\cfield.c identical subexpressions in Z_set

2015-12-11 Thread Alexander Riccio

New submission from Alexander Riccio:

I found this while writing up a separate bug (CPython doesn't use static 
analysis!).


In _ctypes/cfield.c, Z_set has a bug of some sort:

if (PyLong_Check(value) || PyLong_Check(value)) {

See: https://hg.python.org/cpython/file/tip/Modules/_ctypes/cfield.c#l1378

...which has been there for at least 5 years: 
https://hg.python.org/cpython/rev/cab14be0ada1


I'm not sure what the original programmer meant - it's been around forever & I 
don't know what was there before it - but PyLong_Check(value) is evaluated 
twice. Which doesn't really make sense.

Compiling with /analyze found this quite easily:

c:\pythondev\repo\modules\_ctypes\cfield.c(1378): warning C6287: Redundant 
code:  the left and right sub-expressions are identical.



There's a similar issue in P_set, at line 1486.

--
components: ctypes
messages: 256256
nosy: Alexander Riccio, amaury.forgeotdarc, belopolsky, meador.inge
priority: normal
severity: normal
status: open
title: _ctypes\cfield.c identical subexpressions in Z_set

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25845>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25847] CPython not using Visual Studio code analysis!

2015-12-11 Thread Alexander Riccio

New submission from Alexander Riccio:

Visual Studio comes with static analysis, enabled by /analyze (command line) or 
"Code analysis" in the project configuration dialog. Currently, none of the 
CPython projects in PCbuild have Code Analysis turned on, in any configuration.

I was going to write my first patch, for issue25386, but noticed this, ran a 
(partial) build with /analyze, and ended up filing three bugs instead 
(Issue25844, Issue25845, Issue25846) from bugs /analyze found.

There's quite a bad signal-to-noise ratio at the moment, as there's lots of 
variable shadowing, and there's lots of code that /analyze doesn't understand 
is benign (parsing a tuple into a variable confuses /analyze), but there is 
also lots of code that isn't *obviously* incorrect.

Of the code that's not obviously incorrect, /analyze usually complains about 
possibly out-of-bounds reads in very complex conditions, and I really can't 
tell. Some assertions would probably help.


Thoughts?

--
components: Build
messages: 256265
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: CPython not using Visual Studio code analysis!

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25386] msvcrt_putch/msvcrt_putwch don't check the return value of _putch/_putwch

2015-10-22 Thread Alexander Riccio

Alexander Riccio added the comment:

Sorry for the delay: Gmail actually directed the update emails to my spam 
folder! Gmail said (something like): "It is in violation of Google's 
recommended email sender guidelines."

...and it's apparently not the first time this has happened with the python 
bugtracker: 
https://mail.python.org/pipermail/tracker-discuss/2015-January/003989.html


Anyways:

> That's a great test, thanks.

That IS a great test :)

> I'm curious: did you [Alexander] have some code where you're actually 
> expecting an exception, or did you just stumble across this case?

Nope, I was just poking around the sources. I was curious to see what CPython's 
C implementation looked like, and decided to poke around some of the dustier, 
and the less used corners.

That, and I'm really OCD about checking return values.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25386>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25386] msvcrt_putch/msvcrt_putwch don't check the return value of _putch/_putwch

2015-10-12 Thread Alexander Riccio

Alexander Riccio added the comment:

For your convenience, the MSDN docs for the _putch/_putwch functions: 
https://msdn.microsoft.com/en-us/library/azb6c04e.aspx

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25386>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25386] msvcrt_putch/msvcrt_putwch don't check the return value of _putch/_putwch

2015-10-12 Thread Alexander Riccio

New submission from Alexander Riccio:

A minor issue (probably qualifies for the "easy" keyword):

All functions in msvcrtmodule.c (I'm looking at 
http://svn.python.org/projects/python/trunk/PC/msvcrtmodule.c) except 
msvcrt_putch and msvcrt_putwch properly check return values against error 
codes, and call one of the PyErr_Set* functions to properly bubble the error up 
the call stack.

_putch returns EOF on failure, and _putwch returns WEOF on failure.

Like the rest of the functions in that file, I imagine that the fix would 
involve something like:


if (_putch(ch) == EOF)
return PyErr_SetFromErrno(PyExc_IOError);


Note: the Python msvcrt documentation 
(https://docs.python.org/3/library/msvcrt.html) says:

"Changed in version 3.3: Operations in this module now raise OSError where 
IOError was raised."

...so if you were to backport this minor fix, you need to consider that (not 
sure what difference that makes).

Should I take a stab at patching this myself (I've never done this for the 
Python project) or shall I leave that to the devs?

--
components: Windows
messages: 252899
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: msvcrt_putch/msvcrt_putwch don't check the return value of _putch/_putwch
type: behavior
versions: Python 3.6

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25386>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25387] sound_msgbeep doesn't check the return value of MessageBeep

2015-10-12 Thread Alexander Riccio

New submission from Alexander Riccio:

A really minor issue (probably qualifies for the "easy" keyword):

sound_msgbeep (in http://svn.python.org/projects/python/trunk/PC/winsound.c) 
doesn't check the return value of MessageBeep 
(https://msdn.microsoft.com/en-us/library/windows/desktop/ms680356.aspx). This 
is a very minor issue, but the possibility of failure is not mentioned in the 
winsound documentation. Invisible failures are quite surprising.


A message in the Gnome archives 
(https://mail.gnome.org/archives/commits-list/2010-November/msg01938.html) has 
an example "fix".

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25387>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25387] sound_msgbeep doesn't check the return value of MessageBeep

2015-10-12 Thread Alexander Riccio

Changes by Alexander Riccio <test35...@gmail.com>:


--
components: Windows
nosy: Alexander Riccio, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: sound_msgbeep doesn't check the return value of MessageBeep
type: behavior
versions: Python 3.6

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25387>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com