Re: [Cython] [cython-users] Re: Welcome David Woods as a Cython core developer

[Robert Bradshaw]

+1, thank you David!

On Mon, Aug 1, 2022 at 1:29 PM Blue  wrote:

> Congrats!
>
> I have had the pleasure of working with David in reviewing some their PRs,
> nice to see this. Well deserved!
>
> On Sun, 31 Jul 2022, 21:47 Matti Picus,  wrote:
>
>> As someone watching from the sidelines, it is nice to see the Cython
>> team grow, especially with such a talented and committed contributor.
>>
>> Matti
>>
>>
>> On 31/7/22 12:15, Stefan Behnel wrote:
>> > Hi everyone,
>> >
>> > with the release of the first 3.0 alpha that supports Python 3.11
>> > (aptly named "alpha 11"), I'm happy to announce that David Woods has
>> > been promoted to a Cython core developer.
>> >
>> > David has shown an extraordinary commitment and dedication over the
>> > last years. His first merged commits were already back in 2015, mostly
>> > related to the C++ support. But within the last two years, he
>> > voluntarily took over more and more responsibility for bugs and issues
>> > and developed several major new features for the project. This
>> > includes the Walrus operator (PEP 572), cdef dataclasses (modelled
>> > after PEP 557), internal "std::move()" usage in C++ mode or support
>> > for Unicode identifiers and module names, all of which form a major
>> > part of the 3.0 feature set. David has more than deserved a place in
>> > the circle of present and prior core devs.
>> >
>> > David, thank you for your impressive work on Cython,
>> > and welcome to the core team!
>> >
>> > Stefan
>> > ___
>> > cython-devel mailing list
>> > cython-devel@python.org
>> > https://mail.python.org/mailman/listinfo/cython-devel
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/cython-users/CAHnUJPaWhwfyN%2BH32zFGSSX2BVo39T2O5yxc9Rmy0LNyWpvvAQ%40mail.gmail.com
> 
> .
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Auto-generation of wrapper types

[Robert Bradshaw]

I am in general of more automated wrapping. Two things that make C++
classes more difficulty are.

(1) How to handle inheritance (including multiple inheritance)? Would
the wrapped types mirror the inheritance?
(2) We can do auto-conversion of most types because they are passed by
value. This breaks down a bit with char*, but generally we assume we
can take a copy. Similarly with structs. However C++ classes are (in
practice) much more stateful and this gets into all sorts of thorny
issues with what the conventions should be about ownership and common
references.

There's also a fair number of existing libraries out there that
specifically target wrapping C++ libraries (often directly from the
source files). As well as making it easier to wrap from C++ classes
from Cython, it'd be good to have more seamless (and efficient)
integration with those.

On Sun, Mar 15, 2020 at 1:34 PM da-woods  wrote:
>
> On 15/03/2020 08:14, Stefan Behnel wrote:
> > What makes C structs and C++ classes special enough to make this a separate
> > language feature? That should be part of a motivation somewhere (probably
> > in a ticket
>
> Most of the other major things you could want to wrap are covered
> through other mechanisms - you can create a wrapper round a `cdef
> extern` function or enum with `cpdef`. The basic C types (int, float)
> etc largely have a direct Python equivalent that converts freely.
>
> This is mainly targetted at C++ classes which seem to be the last major
> feature that doesn't have this kind of wrapper. C structs just seemed
> easy to deal with by the same mechanism - although they obviously have
> the conversion to/from dict, which is fine but does involve a
> conversion, and so doesn't propagate modifications back.
>
> I'll create a ticket fairly shortly.
>
> > How does this relate to the support for @dataclass in GH-2903? Is the only
> > difference that you would write "x.member" instead of "x.structattr.member"?
>
> It's slightly difficult to answer that because we don't know exactly
> what @dataclass will look like in Cython. In Python it's mostly about
> generating `__init__`, `__repr__`, `__eq__` etc from a "struct-like"
> description. The Cython version I've submitted basically just follows
> the Python scheme, but it isn't obvious how Python-level access to
> attributes should work. "x.structattr" would probably end up doing the
> usual Cython conversion to/from a dict.
>
> This feature is mostly for creating a quick wrapper for some external
> C/C++ thing that already exists. For a struct it probably isn't too
> different from a dataclass. For a C++ class it should hopefully be able
> to fill in a lot of the functions too.
>
> > Why "autowrap[S]" and not "autowrap(S)"? I'm asking also because there
> > should probably be a way for this to work in Python files, for which a
> > function call seems more natural.
> >
> > It also feels like a @decorator would somehow fit quite well here. Even
> > when defined in a .pxd file, the struct would still only be exported in the
> > corresponding module (with the same name). Something like this:
> >
> >  @cclass
> >  cdef struct S:
> >  int x
> >
> > Not sure if reusing @cclass is a good idea, but it doesn't seem wrong.
>
> Definitely open to different syntax - now you point it out I think a
> function call might be better than an index (especially with optional
> keyword arguments to control it).
>
> One other option (for consistency with how functions/enums are already
> handled) would be `cpdef struct S/cpdef cppclass ClassName`. The
> difficulty here is that it overrides the name which I suspect could be a
> challenge (in most cases you'd probably want to access the original
> definition and the wrapper extension type separately in Cython). This
> might be an issue with a decorator too.
>
> > Defining this in a .pxd file could then even allow cimporting modules to
> > understand the extension type wrapper as well, by simply reconstructing the
> > type representation on their side.
>
> I hadn't thought of this, but yes. This should be pretty simple - the
> type representation is really just:
>
> cdef class SomeName:
>  cdef c_type* obj
>  @staticmethod
>  cdef factor_func(obj x) # possibly reference or r-value here
> reference
>
> (Almost) everything else it defines is a Python interface of def
> functions and properties so a cimporting module would have no special
> knowledge - for fast access the user would go through obj.
>
> David
>
>
> >
> > Stefan
> > ___
> > cython-devel mailing list
> > cython-devel@python.org
> > https://mail.python.org/mailman/listinfo/cython-devel
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cytho

Re: [Cython] Size of output

[Robert Bradshaw]

On Sat, Feb 1, 2020 at 6:33 AM John Skaller2  wrote:
>
> > given target. Removing the final 0.5% of code that is "unused in some
> > cases" is really not something I would want to invest days into, each time
> > we make a change in Cython.
>
> But its not 0.5%. My problem is trying to read the generated code.
>
> A possibility might be an option which puts the boilerplate in one file
> and the variant code in another that then #includes the boilerplate.
> So the code is the same, but the main file is much easier to inspect
> to see what got generated.

The primary target of the generated code is a C compiler, though we do
try to generate code that's reasonably comprehensible for a human as
well. However, there's often a tradeoff to be made in terms of
conciseness and readability of the generated C code and that of the
Cython codebase itself, and the latter is arguably more important.

Also, as mentioned earlier, the generated code also makes heavy use of
macros (including much of the refnanny stuff) so that various
decisions can be deferred to C compile time.

I'm also sure there's room for improvement, but that might not be the
highest priority for where to spend limited time. Hopefully the -a
option is useful.

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Size of output

[Robert Bradshaw]

On Fri, Jan 31, 2020 at 3:17 PM Greg Ewing  wrote:
>
> On 1/02/20 3:17 am, John Skaller2 wrote:
> > When I ran Cython on a two line Python function I got this from wc:
> >
> >  4276   13798  161338 oldtest.c
>
> That seems a bit excessive.
>
> > A lot of the emitted code appeared to be run time and compile
> > time support code which wasn’t actually used.
>
> Not sure what's going on there. Pyrex made efforts to only
> include support code that was actually used, but Cython
> has changed a lot since then and I haven't been following
> its development closely. Either it's slipped on that, or
> the support code has become more bloated.

Cython attempts to do the same.

Taking a quick glance at an auto-generated file for an empty .pyx, we have

~200 lines of macros normalizing various C compiler issues
~500 lines defining macros to normalize across Python 2.7-3.9
~200 lines of providing defaults for various CYTHON_ macros
~300 lines of macros for optional optimizations for CPython details
(vs. using more public/pypy compatible, ... APIs)
~300 lines module setup code. Even for trivial modules, we still
declare and call functions for creating globals, preparing types, etc.
even if we don't have any globals, types, etc.
~300 lines exception handling and traceback creation
~700 lines conversion for basic int and string types (which we assume
to be available in various utilities).

Much of this is macro-heavy code, to allow maximum flexibility at C
compile time, but much would get elided by the preprocessor for any
particular environment.

Extra utility code is inserted on an as-needed bases, e.g. function
creation, various dataytype optimizations, other type conversions,
etc. These are re-used within a module. A two line function could add
a lot (e.g. just defining a function and its wrapper is a good chunk
of code, and whatever the function does of course).

I agree there's some fat that could be trimmed there, but not sure
it'd be worth the effort.

> Can you remove any of it and still have it compile? If
> so, filing a bug report might be useful.

+1

> > Is there an option to use an #include for the standard stuff?
>
> There are upsides and downsides to that as well. The way
> things are, the generated file is self-contained, and can
> be shipped without worrying about it becoming disconnected
> from a compatible version of the include file. This is
> important when details of the support code can change
> without notice between Cython releases.

+1

We have an option "common_utility_include_dir" that would create a
shared utility folder into which the compiler could create (versioned)
#includable files to possibly be shared across many modules, but it
was never completely finished (and in particular was difficult to
reconcile with cycache, which is like ccache for Cython, due to the
outside references a cython artifact could then produce). We've
thought of going even further and providing a shared runtime library,
but that has some of the same issues (plus more, though in some cases
we use the pattern where every module declares type X, but before
creating its own looks to see if one was already loaded to let modules
share the same internal type at runtime).
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] longintrepr.h issues with MinGW

[Robert Bradshaw]

On Tue, Oct 23, 2018 at 10:22 PM Stefan Behnel  wrote:
>
> Robert Bradshaw schrieb am 22.10.2018 um 11:03:
> > Given that https://bugs.python.org/issue4709 results in extension
> > modules that seem to work, but silently produce completely incorrect
> > answers, I'm thinking we should either disable our long-unpacking code
> > on these platforms, or at the very least give a runtime error if we
> > detect issues like sizeof(void*) != SIZEOF_VOID_P (or both).
>
> We could try to detect MinGW-64 and set the "MS_WIN64" macro before
> importing the CPython headers. That seems to be the right fix. Can't say if
> we also need to define the "MS_WIN32" macro, but would probably be better
> to have it. CPython defines both for MSVC.
>
> There is a "__MINGW64__" macro to detect MinGW-64, it seems.

I created a minimal https://github.com/cython/cython/pull/2679

> > I'm not sure what has made this issue pop up more lately (Python 3),
>
> My guess is that MinGW is just fairly rarely used to build CPython
> extensions overall.
>
> > but perhaps we should even consider releasing bugfixes for previous
> > Cython versions.
>
> You mean a 0.28.6, or even older than that?

I wish there were a good way to see what users are actually using in the wild...

> As I said, MinGW does not seem to be used very widely…

Yes, but it is used, and this is (IMHO) a really severe bug.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] longintrepr.h issues with MinGW

[Robert Bradshaw]

Given that https://bugs.python.org/issue4709 results in extension
modules that seem to work, but silently produce completely incorrect
answers, I'm thinking we should either disable our long-unpacking code
on these platforms, or at the very least give a runtime error if we
detect issues like sizeof(void*) != SIZEOF_VOID_P (or both).

I'm not sure what has made this issue pop up more lately (Python 3),
but perhaps we should even consider releasing bugfixes for previous
Cython versions.

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Enhancing "ctyepdef class numpy.ndarray" with getter properties

[Robert Bradshaw]

On Thu, Sep 27, 2018 at 11:36 PM Matti Picus  wrote:

> On 27/09/18 22:50, Robert Bradshaw wrote:
> >
> > On Thu, Sep 27, 2018 at 10:38 AM Matti Picus
> > mailto:matti.pi...@gmail.com>> wrote:
> > To solve issue #2498, I did some experiments
> > https://github.com/cython/cython/issues/2498#issuecomment-414543549
> > with
> > hiding direct field access in an external extension type (documented
> > here
> >
> https://cython.readthedocs.io/en/latest/src/userguide/extension_types.html#external-extension-types
> ).
> >
> > The idea is to write `a.ndims` in cython (in plain python code),
> > and in
> > C magically get the attribute lookup converted into a
> > `PyArray_NDIMS(a)`
> > getter, which could be a macro or a c-function.
> >
> > The experiments proved fruitful, and garnered some positive
> > feedback so
> > I am pushing forward.
> >
> > I would like to get some feedback on syntax before I progress too
> > far.
> > Should the syntax be extended to support
> >
> > ctypedef class numpy.ndarray [object PyArrayObject]:cdef: # Convert
> > python __getattr__ access to c functions. int ndims PyArray_NDIMS |
> >
> >
> > or perhaps a decorator, like Python
> >
> > |ctypedef class numpy.ndarray [object PyArrayObject]: cdef: # Convert
> > python __getattr__ access to c functions. @property  cdef int
> > ndims(self): return PyArray_NDIMS(self) or something else? The second
> > seems more wordy but more explicit. I don't know which would be
> > easier
> > to implement or require more effort to test and maintain.
> >
>
> > Matti |
> >
> > Thanks for looking into this!
> >
> > My preference would be to use the @property syntax, as this will be
> > immediately understandable to any Cython user and could contain
> > arbitrary code, rather than just a macro call.
> >
> > There are, however, a couple of downsides. The first is that it may
> > not be clear when accessing an attribute that a full function call may
> > be invoked. (Arguably this is the same issue one has with Python, but
> > there attribute access is already expensive. The function could be
> > inline as well if desired.) The second is that this means that this
> > attribute is no longer an lvalue. The last is that it's a bit special
> > to be defining methods on an extern class. Maybe it would have to be
> > inline if it's in the pxd?
> >
> > If we're going to be defining a special syntax, I might prefer
> > something like
> >
> > cdef extern class ...:
> > int ndims "PyArray_NDIMS(*)"
> >
> > which more resembles
> >
> > int ndims "nd"
> >
> > Open to bikeshedding on what the "self" placeholder should be. As
> > before, should the ndims lose its lvalue status in this case, or not
> > (in case the accessor is really a macro intended to be used like this)?
> >
> >
> Sorry about the formatting messup, the original proposal was supposed to
> be (this time using double spacing to make sure it works):
>
>
> -
>
> cdef extern class ...:
>
>  @property
>
>  cdef int ndims(self):
>
>  return PyArray_NDIMS(self)
>
> --
>
> vs
>
> 
>
> cdef extern class ...:
>
>  cdef int ndims PyArray_NDIMS
>
> 
>
> The proposal  is for a getter via a C function or a macro. NumPy's
> current public API uses a mix. Currently I am interested in getters that
> would not allow lvalue at all. Maybe in the future we will have fast
> rvalue setter functions in NumPy, but the current API does not support
> them. It remains to be seem how much slowdown we see in real-life
> benchmarks when calling a small C function from a different shared
> object to access attributes rather than directly accessing them via
> struct fields.
>

Hmm...so in this case it upgrading Cython would cause an unconditional
switch from direct access to a function call without any code change (or
choice) for users of numpy.pxd. I am curious what kind of a slowdown this
would represent (though would assume this kind of analysis was done by the
NumPy folks when choosing macro vs. function for the public API).

As I point out in the "experiment" com

Re: [Cython] Enhancing "ctyepdef class numpy.ndarray" with getter properties

[Robert Bradshaw]

Thanks for looking into this!

My preference would be to use the @property syntax, as this will be
immediately understandable to any Cython user and could contain arbitrary
code, rather than just a macro call.

There are, however, a couple of downsides. The first is that it may not be
clear when accessing an attribute that a full function call may be invoked.
(Arguably this is the same issue one has with Python, but there attribute
access is already expensive. The function could be inline as well if
desired.) The second is that this means that this attribute is no longer an
lvalue. The last is that it's a bit special to be defining methods on an
extern class. Maybe it would have to be inline if it's in the pxd?

If we're going to be defining a special syntax, I might prefer something
like

cdef extern class ...:
int ndims "PyArray_NDIMS(*)"

which more resembles

int ndims "nd"

Open to bikeshedding on what the "self" placeholder should be. As before,
should the ndims lose its lvalue status in this case, or not (in case the
accessor is really a macro intended to be used like this)?


On Thu, Sep 27, 2018 at 10:38 AM Matti Picus  wrote:

> To solve issue #2498, I did some experiments
> https://github.com/cython/cython/issues/2498#issuecomment-414543549 with
> hiding direct field access in an external extension type (documented
> here
>
> https://cython.readthedocs.io/en/latest/src/userguide/extension_types.html#external-extension-types).
>
> The idea is to write `a.ndims` in cython (in plain python code), and in
> C magically get the attribute lookup converted into a `PyArray_NDIMS(a)`
> getter, which could be a macro or a c-function.
>
> The experiments proved fruitful, and garnered some positive feedback so
> I am pushing forward.
>
> I would like to get some feedback on syntax before I progress too far.
> Should the syntax be extended to support
>
> |ctypedef class numpy.ndarray [object PyArrayObject]: cdef: # Convert
> python __getattr__ access to c functions. int ndims PyArray_NDIMS |
>
>
> or perhaps a decorator, like Python
>
> |ctypedef class numpy.ndarray [object PyArrayObject]: cdef: # Convert
> python __getattr__ access to c functions. @property  cdef int
> ndims(self): return PyArray_NDIMS(self) or something else? The second
> seems more wordy but more explicit. I don't know which would be easier
> to implement or require more effort to test and maintain. Matti |
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Safer exception handling

[Robert Bradshaw]

On Wed, Sep 19, 2018, 5:04 PM Stefan Behnel  wrote:

> Robert Bradshaw schrieb am 17.09.2018 um 15:44:
> > One of the pain points in Cython is that one must explicitly annotate
> > non-object returning functions with except clauses. Would it be worth
> > trying to change the default here, making exception-suppressing opt-in
> > rather than opt-out?
> >
> > There are a couple of open questions, e.g.
> >   * What would the syntax be?
> >   * What about extern functions?
> >   * What would the performance impact be? Could it be mitigated?
>
> One more point: what about nogil functions? They can, in theory, raise
> exceptions, but they almost never do in practice.
>

I think feels like it should be orthogonal, I wonder what the overhead of
this check really is in practice...

Given that the default for return type annotations is exception propagation
> (i.e. Python semantics) now, mixing that into a nogil function means that
> the caller would almost always check for exceptions needlessly. And we do
> not currently have a way to say "no exception return value" to get rid of
> that check. That's something we'd need, especially if we change the
> default.
>

Yeah, that's what I meant about the syntax question.


> Stefan
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Safer exception handling

[Robert Bradshaw]

On Tue, Sep 18, 2018, 10:12 AM Jeroen Demeyer  wrote:

> On 2018-09-17 15:44, Robert Bradshaw wrote:
> > One of the pain points in Cython is that one must explicitly annotate
> > non-object returning functions with except clauses. Would it be worth
> > trying to change the default here, making exception-suppressing opt-in
> > rather than opt-out?
>
> Just to clarify things: are you proposing that the default would be
> "except *"?
>

We may choose an implicit default exception value.

An alternative would be to give *warnings* for functions where
> exceptions could occur but could not be propagated. For example, this
> function is totally fine:
>
> cdef int foo(int x):
>  return x
>
> but this function would give a warning:
>
> cdef int foo(x):
>  return x   # implicit conversion Python -> int
>

Given that essentially every Python operation can raise exceptions, I don't
know how useful this warning would be. Unless we expect everyone to change
their code.

and this should probably be a compile-time error:
>
> cdef int foo(x):
>  raise NotImplementedError
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Safer exception handling

[Robert Bradshaw]

One of the pain points in Cython is that one must explicitly annotate
non-object returning functions with except clauses. Would it be worth
trying to change the default here, making exception-suppressing opt-in
rather than opt-out?

There are a couple of open questions, e.g.
  * What would the syntax be?
  * What about extern functions?
  * What would the performance impact be? Could it be mitigated?
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 3.0 and "unicode_literals"

[Robert Bradshaw]

On Sun, Sep 16, 2018 at 5:24 PM Stefan Behnel  wrote:

> Jeroen Demeyer schrieb am 22.08.2018 um 13:10:
> > On 2018-08-19 08:26, Stefan Behnel wrote:
> >> Should we make that a new directive rather than a language level? Like
> >> "py2_str=str"? That would allow its use together with language_level=3
> >> already in the next release.
> >
> > With a new new directive, you also run into compatibility problems. What
> > should the default be? py2_str=str or py2_str=unicode? The former breaks
> > code assuming that it's unicode and the latter doesn't really solve
> > anything: stuff will still break when language_level=3 becomes the
> default.
> >
> > My proposal is a new setting language_level=3str (meaning: everything
> that
> > language_level=3 does, except unicode_literals) and make that the
> default.
> > That way, you keep full compatibility with code already setting the
> > language_level. You also have reasonably good chances that code that
> > currently uses the implicit language_level=2 will continue to work with
> > language_level=3str.
>
> I thought about this some more and I like the idea. There's still the
> true-division issue, but strings are certainly the biggest blocker in
> migrations. As long as people want to support Python 2.x, "3str" is a way
> to help them with it, and for Py3-only users, it won't make a difference.
>

Yes, this does seem worth handling differently.

I've also been wondering if the c_string_type directive would work for
this. (Unfortunately, we let c_string_type=str mean c_string_type=bytes,
where a more natural interpretation for c_string_type=str would be "bytes
in py2, unicode in py3." Would it be good enough to disallow str for a
release before going to 3.0? (Or would a warning be good enough, given that
Cython 3.0 is backwards incompatible?)
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.29 – or 29.0 ?

[Robert Bradshaw]

On Sat, Aug 18, 2018 at 10:24 AM, Jeroen Demeyer  wrote:
> On 2018-08-17 11:44, Stefan Behnel wrote:
>>
>> Hej, that gives us an alternative for the versioning switch. We could
>> release Cython 3.0 when we change the default language level (and require
>> users to select "language_level=2" for legacy code). Definitely a breaking
>> change that merits inceasing the major version, and 3.0 seems very
>> suitable.
>
>
> One annoying point with language_level=3 is that all string literals become
> unicode (like from __future__ import unicode_literals). Unlike the other
> changes that language_level=3 makes, this is a major breaking change on
> Python 2.
>
> So I would very much prefer enabling everything that language_level=3 does,
> but keeping strings of type "str" on Python 2. Maybe you could invent a new
> option for that, say "language_level=3str"?

Do the existing c_string_type and c_string_encoding directives not
cover this usecase? I suppose what you're asking for is str being the
str of the runtime, even if language_level=3 is set.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.29 – or 29.0 ?

[Robert Bradshaw]

It'd be nice to change the default of cython.binding as well (breaking
change).

On Fri, Aug 17, 2018 at 9:19 AM, Robert Bradshaw  wrote:

> I like the jump to 3.0 to to change the language level. It still might be
> safer to *require* this flag for a period though, and 3.0 would make it
> optional (with the opposite default than it has now). (I'm honestly on the
> fence with this one, as it may be a major pain to make this required...)
>
> Letting the next version be 2.9 feels like it risks typos and confusion
> (e.g. people who are used to the current numbering will call it 0.29).
> Jumping to 29.0 makes it feel like we'll be updating the major version
> often (or we'll be at an arbitrary 29.x for a long time).
>
> As for semantic versioning, I'm with Julian that it's more about
> communicating intent. Patch releases should be safe no-brainers, minor
> versions should be safe by default, but make sure you think about them, and
> major changes will likely require code changes. Not unlike how Python has
> handled numbers (though hopefully we'll release with more frequency).
>
> We may want to reserve a major version bump for issues like
> https://github.com/cython/cython/issues/1382 which are simply not done
> for fear of breaking users code, before 3.0.
>
> So, I might release the next release by default (0.29.6? Or is it too
> major. I guess going 0.30 would be bad.), get a short list of breaking
> changes we're actually going to change by the next major release (such as
> requiring a language level, double-underscore mangling, ??? called 1.x or
> 2.x) and go from there.
>
> I like a plan to get out of the 0.x naming scheme, but it's hard to argue
> it must be done *now* vs. in a release or two.
>
> - Robert
>
>
>
> On Fri, Aug 17, 2018 at 8:42 AM, Yury V. Zaytsev  wrote:
>
>> On Fri, 17 Aug 2018, Stefan Behnel wrote:
>>
>> Proposal: we'll release a 2.9.0 next, which gives a warning that users
>>> should explicitly enable "language_level=2" in their setup.py when
>>> compiling Py2 code, and then release 3.0 as the next feature release, which
>>> requires this option for Py2 code and defaults to compiling with Py3 syntax
>>> and semantics.
>>>
>>
>> Sounds like an excellent plan, I like it!
>>
>> --
>> Sincerely yours,
>> Yury V. Zaytsev
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>>
>
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.29 – or 29.0 ?

[Robert Bradshaw]

I like the jump to 3.0 to to change the language level. It still might be
safer to *require* this flag for a period though, and 3.0 would make it
optional (with the opposite default than it has now). (I'm honestly on the
fence with this one, as it may be a major pain to make this required...)

Letting the next version be 2.9 feels like it risks typos and confusion
(e.g. people who are used to the current numbering will call it 0.29).
Jumping to 29.0 makes it feel like we'll be updating the major version
often (or we'll be at an arbitrary 29.x for a long time).

As for semantic versioning, I'm with Julian that it's more about
communicating intent. Patch releases should be safe no-brainers, minor
versions should be safe by default, but make sure you think about them, and
major changes will likely require code changes. Not unlike how Python has
handled numbers (though hopefully we'll release with more frequency).

We may want to reserve a major version bump for issues like
https://github.com/cython/cython/issues/1382 which are simply not done for
fear of breaking users code, before 3.0.

So, I might release the next release by default (0.29.6? Or is it too
major. I guess going 0.30 would be bad.), get a short list of breaking
changes we're actually going to change by the next major release (such as
requiring a language level, double-underscore mangling, ??? called 1.x or
2.x) and go from there.

I like a plan to get out of the 0.x naming scheme, but it's hard to argue
it must be done *now* vs. in a release or two.

- Robert



On Fri, Aug 17, 2018 at 8:42 AM, Yury V. Zaytsev  wrote:

> On Fri, 17 Aug 2018, Stefan Behnel wrote:
>
> Proposal: we'll release a 2.9.0 next, which gives a warning that users
>> should explicitly enable "language_level=2" in their setup.py when
>> compiling Py2 code, and then release 3.0 as the next feature release, which
>> requires this option for Py2 code and defaults to compiling with Py3 syntax
>> and semantics.
>>
>
> Sounds like an excellent plan, I like it!
>
> --
> Sincerely yours,
> Yury V. Zaytsev
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.29 – or 29.0 ?

[Robert Bradshaw]

If we're going to ditch the 0.x, I'd go for 1.0 as well. I'm a huge fan of
semantic versioning.

The primary reasons we kept the 0.x scheme were that

* We wanted full compatibility with CPython (we're nearly there, or at
least it's safe to say the differences are on par with those between
different versions and implementations of Python), and
* We wanted the flexibility to possibly jettison archaic features of the
language or otherwise clean things up.

It's worth revisiting to see if either of these are still relevant (*and*
likely to be worked on in the near future).




On Wed, Aug 15, 2018 at 10:40 PM, Jeroen Demeyer  wrote:

> I vote for 1.0
>
> Version 29.0 sounds too much like marketing.
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] #cython on freenode

[Robert Bradshaw]

Thanks for the heads up. I'd be happy to help with this.

On Thu, Aug 9, 2018, 12:55 AM Nathan Goldbaum  wrote:

> Hi all,
>
> The Freenode IRC network is currently undergoing a spam attack that is
> affecting the #cython channel there.
>
> People definitely do use #cython to ask questions. I and some others try
> and help them out. It would be a shame to have that community, such as it
> is, get destroyed by the spam attack.
>
> Freenode has a way for a project to take control of the IRC channel
> associated with their name by mailing proje...@freenode.net and working
> with an admin to prove who you say you are. For matplotlib they asked Tom
> Caswell to push a branch with a randomly chosen name to github. If someone
> associated with the project is willing to do something like that, I would
> be very happy to volunteer to admin the channel and set the channel
> settings to mitigate the spam attack. I'm currently doing a similar process
> to get admin powers for #ipython and #matplotlib and already have admin
> powers in #scipy and #numpy.
>
> Thanks for your help,
>
> -Nathan
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Should we start requiring "language_level=2" for .pyx files?

[Robert Bradshaw]

Language level affects things like integer division (for example).

I agree it makes sense to start putting a warning in.



On Thu, May 31, 2018 at 7:35 AM, Erik Bray  wrote:

> On Mon, May 28, 2018 at 8:56 PM, Stefan Behnel 
> wrote:
> > Hi,
> >
> > Python 3 is clearly taking over the world these days, so it starts
> feeling
> > arcane to require Py2 syntax in .pyx files. Increasingly, it means that
> > people cannot just rename .py files anymore to start optimising them,
> > because the .py file has a high chance of being written in Py3 syntax.
> >
> > Eventually, we will have to switch to Py3 syntax by default in order to
> > follow what most people are (or will be) used to.
> >
> > As a transition, I think we could start warning about cases where the
> > language level is not set explicitly. If people start marking their code
> as
> > being "Cython 2.x code", either with an in-file directive or from their
> > setup.py, we will have less of a problem in the future to change the
> default.
> >
> > What do you think? Any other ideas, comments, objections?
>
>
> Perhaps you could clarify something:  I tried suggesting a while ago
> that Sage start using language_level=3 at least when actually building
> Sage on Python 3.  I know this isn't necessary but it just seemed to
> make logical sense.  But Jeroen was convinced it wasn't necessary
> because, according to him, language_level=3 doesn't really do
> anything.
>
> So what exactly does language_level=3 (or 2) do, such that it would
> impact porting Python 3 code to Cython?
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Multiple inheritance with old-style classes in Python 2?

[Robert Bradshaw]

No, we don't care about supporting this, but we should detect and
reject it informatively when possible.

On Thu, Jan 25, 2018 at 9:40 AM, Elizabeth A. Fischer
 wrote:
> No.  Python2 is obsolete, old style classes even more so.
>
> On Thu, Jan 25, 2018 at 12:22 PM Jeroen Demeyer  wrote:
>>
>> Do we want to support Python 2 old-style classes for multiple
>> inheritance? Personally, I don't think that we should, but it's
>> something that has to be decided.
>>
>> The reason I ask is that my code from
>> https://github.com/cython/cython/pull/2033 is actually broken when given
>> an old-style class.
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Segfault with large cdef'd list

[Robert Bradshaw]

Cython itself doesn't impose any limits, but it does inherit whatever
limit exists in the C complier and runtime. The variance may be due to
whatever else happens to be placed on the stack.

On Sat, Jan 6, 2018 at 10:57 PM, Dan Stromberg  wrote:
> I'm getting a weird segfault from a tiny function (SSCCE) using cython
> with python 2.7.  I'm seeing something similar with cython and python
> 3.5, though I did not create an SSCCE for 3.5.
>
> This same code used to work with slightly older cythons and pythons,
> and a slightly older version of Linux Mint.
>
> The code is at http://stromberg.dnsalias.org/svn/why-is-python-slow/trunk
> (more specifically at
> http://stromberg.dnsalias.org/svn/why-is-python-slow/trunk/tst.pyx )
>
> In short, cdef'ing a list of doubles with about a million elements,
> and using only the 0th element once, segfaults - but cdef'ing a
> slightly smaller array does not segfault under otherwise identical
> conditions.
>
> Any suggestions?  Does Cython have a limit on the max size of a stack frame?
>
> Thanks!  I quite like cython.
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Bug with inlined functions that access globals

[Robert Bradshaw]

On Fri, Nov 24, 2017 at 5:48 AM, Erik Bray  wrote:
> On Fri, Nov 24, 2017 at 2:16 PM, Stefan Behnel  wrote:
>> Erik Bray schrieb am 24.11.2017 um 13:44:
>>> I think maybe I've seen this brought up once or twice before in the
>>> past, but with no real discussion.  There is a slightly unpythonic
>>> problem with using cpdef inline functions defined in a .pxd file that
>>> happen to access global variables.  For example:
>>>
>>> $ cat foo.pxd
>>> FOO_A = 1
>>>
>>> cpdef inline foo():
>>> return FOO_A
>>
>> Hmm, I might be missing something, but it feels like globals in a .pxd file
>> should not be allowed in the first place - unless there is a .pyx file for
>> them. But that's difficult to prohibit, given that the .pyx file is not
>> needed at compile time, and might not be there at all, just the readily
>> compiled module (and maybe not even that).
>>
>> And I don't see why Python globals should be allowed in .pxd files at all.
>>
>> So - it seems probably difficult to improve the first case. Regarding your
>> actual example, however, if there is really no use case for Python globals
>> in .pxd files, then we should better disallow them.
>
> That would be an acceptable resolution, though I believe there is a
> use case for it.  If you take some function that was originally
> defined in a .pyx module, but you want to be able to inline it, then
> one is forced to rewrite it in some way if it in any way relied on
> some module-level globals (which is a normal thing to do).  So it's a
> problem of the implementation details driving what language features
> one is allowed to use.  This is less than ideal...
>
> Anyways there's really two issues here:
>
> 1) Python globals in .pxd files, which you're saying should be
> disallowed--I have to wonder if there isn't some way to make that make
> sense, but I don't feel strongly about it.
> 2) Non-local variables in functions defined in .pxd files--again these
> should either be allowed by some mechanism, or entirely disallowed
> since otherwise the result is either the function breaks with a
> NameError, or behaves wildly unexpectedly if it gets dropped into a
> module that happens to have a global of the same name

+1 to disallowing python globals altogether in .pxd files. One can
cdef any global one needs.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Drop Cython wheels for 2.6?

[Robert Bradshaw]

Wheels are a convenience; I'm OK with not going through extra hoops to
support them for a version of Python retired years ago. Until we
actually drop support for 2.6 entirely one can still install from
source.

On Mon, Nov 20, 2017 at 5:57 PM, Nathaniel Smith  wrote:
> On Mon, Nov 20, 2017 at 4:08 PM, Matthew Brett  
> wrote:
>> Hi,
>>
>> The manylinux1 docker image has just stopped supporting Python 2.6, so
>> we can no longer build Python 2.6 wheels without putting some hacks on
>> top:
>>
>> https://github.com/pypa/manylinux/pull/125#issuecomment-345770870
>>
>> Do y'all care about that?
>
> To clarify, it's not so much that the docker image officially dropped
> support, as that pip dropped support and the last rebuild happened to
> pick that up, so we should figure out whether to drop it officially or
> what: https://github.com/pypa/manylinux/issues/126
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Python callable staticmethods?

[Robert Bradshaw]

On Mon, Sep 25, 2017 at 12:58 PM, John Ehresman  wrote:
> On 9/25/17 3:32 PM, Stefan Behnel wrote:
>>
>> Am 25. September 2017 20:59:01 MESZ schrieb John Ehresman:
>>>
>>> Is there a way to define a Python callable staticmethod in a Cython
>>> cdef class?
>>
>>
>> Did you try it?
>
>
> Not fully, I'll admit.  I found that the following are invalid in a .pxd:
>
> cdef class Cls:
>
>   @staticmethod
>   def method(a)
>
>   @staticmethod
>   cpdef method(a)
>
> I later found that just using @staticmethod\n def method(a) in a .py file
> does work (I'm using pure mode).  That means that cython code in other
> modules is doing a PyObject_GetAttr to get the method and then calling the
> returned object (admittedly I'm not far enough along to profile to see if
> this is a real problem or not).
>
> I think I'm really asking about cpdef staticmethods.

You can write cdef static methods and "def" static methods, but cpdef
is not yet supported.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Big backwards compatibility problem with fused types

[Robert Bradshaw]

On Tue, Sep 19, 2017 at 10:48 AM, Stefan Behnel  wrote:
> Hi devs,
>
> this is really bad:
>
> https://github.com/cython/cython/pull/1873
>
> The problem is that cdef functions with fused types are expanded
> arbitrarily on use, and their Entries removed and re-appended to the list
> of cdef function entries. But that list also defines the order of the
> entries in the vtable!
>
> Currently, when a method with fused types is defined in a .pxd file, it is
> the implementation order in the .pyx (!) file that determines the order in
> the vtable. Even worse, for modules that cimport from that .pxd file, it is
> the order in which these methods are *called* that determines the
> corresponding vtable on the other side, as the entries are expanded at
> first use, i.e. at their first occurrence in the code, and appended to the
> current list of cdef functions at this (arbitrary) point.
>
> What this means is that there is no way to safely infer the vtable order of
> an extension type with fused methods from a .pxd file.
>
> Now, the correct way to handle this would have been to *replace* the
> original fused entry with the expanded list of specialised entries. But
> it's too late for that now. All existing modules out there would need to
> get recompiled if we changed their vtable order, at least if they use fused
> any types methods anywhere in their hierarchy.
>
> However, I think we can assume code that uses a different method order in
> the .pxd and .pyx files to be broken already, so a way out of this misery
> would be to make the .pxd order the one and only source, with the twist
> that we must keep the "delete and re-append" algorithm to keep the order
> that existing translated modules are using.
>
> Thus the proposal:
>
> - we switch to a scheme now that is only defined by the first declaration
> order, i.e. the order in the .pxd file if there is one.
>
> - we make sure all fused methods are expanded in that same order, but
> continue to follow all non-fused methods in the vtable.
>
> - we break all code now that uses a different order of fused methods in
> their pyx and pxd files
>
> Thoughts?

+1. We should let the .pxd file, as it is declared, be the source of
truth. Code that doesn't already follow this is and does cross-module
fused-cdef function calls is already broken.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Safer default exception handling with return type annotations?

[Robert Bradshaw]

On Sat, Sep 9, 2017 at 10:36 AM, Stefan Behnel  wrote:
> Robert Bradshaw schrieb am 06.09.2017 um 08:28:
>> On Tue, Sep 5, 2017 at 10:44 PM, Stefan Behnel wrote:
>>> Robert Bradshaw schrieb am 06.09.2017 um 07:21:
>>>> I'm not a huge fan of behaving differently depending on what syntax
>>>> was used to annotate the return type--I'd rather they be 100% aliases
>>>> of each other.
>>>
>>> Regarding this bit - I already chose to implement some differences for
>>> annotation typing. Mainly, if you say
>>>
>>> def f(x: int) -> float:
>>> return x
>>>
>>> then the (plain "def") function will actually be typed as "double
>>> f(object)"., assuming that you probably meant the Python types and not the
>>> C types. If you want the C types "int" and "float", you have to use either
>>> of these:
>>>
>>> def f1(x: cython.int) -> cython.float:
>>> return x
>>>
>>> cpdef float f2(int x):
>>> return x
>>>
>>> That is because the main use case of signature annotations is Python code
>>> compatibility, so I tried to change the semantics as little as possible
>>> from what the code would be expected to do in Python.
>>
>> What about
>>
>> def f(x: float) -> int
>>   return x * 2
>>
>> would that throw an error if x was, say, a str? I think float -> c
>> double but int -> python object will be surprising. I also worry a bit
>> about x: float being enforced but x: List[float] not being so.
>>
>>> I also considered if distinguishing between .py and .pyx files would make
>>> sense here, but adapting the type interpretation to that felt much worse
>>> than the above,
>>
>> Agreed, -1 to doing this.
>>
>>> which is at least consistent regarding the typing scheme
>>> that you use.
>>>
>>> I think this type interpretation is a reasonable, use case driven
>>> difference to make. Thus my question if we should extend this to the
>>> exception declaration.
>>
>> I suppose you've already made a case for deviating...
>>
>> I guess I think it'd be nice to change the default universally, but
>> that's perhaps a bigger conversation.
>
> What do you think of this?
>
> https://github.com/cython/cython/pull/1865

Makes sense to me.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Safer default exception handling with return type annotations?

[Robert Bradshaw]

On Wed, Sep 6, 2017 at 12:06 AM, Stefan Behnel  wrote:
> Robert Bradshaw schrieb am 06.09.2017 um 08:28:
>> On Tue, Sep 5, 2017 at 10:44 PM, Stefan Behnel wrote:
>>> Robert Bradshaw schrieb am 06.09.2017 um 07:21:
>>>> I'm not a huge fan of behaving differently depending on what syntax
>>>> was used to annotate the return type--I'd rather they be 100% aliases
>>>> of each other.
>>>
>>> Regarding this bit - I already chose to implement some differences for
>>> annotation typing. Mainly, if you say
>>>
>>> def f(x: int) -> float:
>>> return x
>>>
>>> then the (plain "def") function will actually be typed as "double
>>> f(object)"., assuming that you probably meant the Python types and not the
>>> C types. If you want the C types "int" and "float", you have to use either
>>> of these:
>>>
>>> def f1(x: cython.int) -> cython.float:
>>> return x
>>>
>>> cpdef float f2(int x):
>>> return x
>>>
>>> That is because the main use case of signature annotations is Python code
>>> compatibility, so I tried to change the semantics as little as possible
>>> from what the code would be expected to do in Python.
>>
>> What about
>>
>> def f(x: float) -> int
>>   return x * 2
>>
>> would that throw an error if x was, say, a str?
>
> It would raise an exception on input, but there would not currently be an
> error on return.
>
>
>> I think float -> c double but int -> python object will be surprising.
>
> I agree. There are two reasons: we don't currently use the int/long Python
> types anywhere in Cython (which could obviously be changed), and in Python
> 2, this would exclude "long" objects, which is most likely not intended.
> So, "int" would probably best refer to "int object or long object" in
> Python 2, which definitely complicates things.
>
> Besides, how many functions can really deal with both "int" and "str" input
> completely? Your example above is actually a good one, because it would
> currently return a float object, not "int". But, since it would do the same
> thing when run in Python, that's probably acceptable. It would need an
> explicit conversion in both cases, in which case Cython wouldn't need to
> enforce the type anymore.
>
> If "int" is used as input type (or variable declaration), then enforcing it
> somehow on assignment would be much more relevant.

Yes, this was my point.

def f(x: float)
return x * 2
f("str")

would throw an error but

def f(x: int)
return x * 2
f("str")

would not, which seems inconsistent. Also,

def f(x: List[float])
return x * 2
f("str")  # error
f(["str"])  # error?
f([1.5]) # ok
f([1]) # ok?

from some_module import X
def g(x: X)
return x * 2
g("str") # error?

There is a whole can of worms here once one starts "enforcing" these
types, partially. I think we need a very clear list of what exactly is
supported. Perhaps we should start out with just supporting cython.*
and perhaps cimported classes.

> Maybe we should reconsider this whole business when we drop support for
> Python 2.7. ;)

I don't think we'll be able to get rid of 2.7 for quite some time...

>> I also worry a bit
>> about x: float being enforced but x: List[float] not being so.
>
> Interpreting "List[float]" as Cython type "list" would definitely be nice,
> but note that it would disallow subtypes. In Python, it does not.
>
> I think the right way to deal with that, eventually, will be optionally
> allowing subtypes also in Cython, and handling the distinction more at a
> case by case basis. Then you could declare a variable as "list" or
> "List[Any]", and enforce the exact type in the first case but not in the
> second.
>
> And we should definitely use "List[itemtype]" hints also in type inference
> for loops and indexing at some point.
>
>>> I think this type interpretation is a reasonable, use case driven
>>> difference to make. Thus my question if we should extend this to the
>>> exception declaration.
>>
>> I suppose you've already made a case for deviating...
>>
>> I guess I think it'd be nice to change the default universally, but
>> that's perhaps a bigger conversation.
>
> I think so, too. For this specific case, we can change the default without
> breaking backwards compatibility.
>
> I also added a new decorator "@exceptval(x, check=False)" for pure mode. If
> used without arguments as "@exceptval()" or even "@exceptval(check=False)",
> which seems more readable, users could still get the "write unraisable but
> don't propagate" behaviour, if they really need it.

+1. Let's make the value required, perhaps with None meaning no value
(as it'll never be used). Perhaps check should default to True as
well, as the overhead is small (it's an additional check only for
exceptions) but more conservative.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Error handling during buffer reassignment

[Robert Bradshaw]

On Wed, Sep 6, 2017 at 3:53 AM, Stefan Behnel  wrote:
> Hi,
>
> consider this code:
>
> cdef np.ndarray[int32, ndim=1] a
> a = new_int32_buffer()
> a = new_float32_buffer()
>
> This fails during the second assignment, but only during validation, after
> unpacking the buffer. The code that handles this case is generated here:
>
> https://github.com/cython/cython/blob/c95ca9f21a3524718a83c3415bb7102a508154be/Cython/Compiler/Buffer.py#L376
>
> Note the twist where it says:
>
> # If acquisition failed, attempt to reacquire the old buffer
> # before raising the exception.
>
> I faintly remember a discussion about that case (apparently back in 2008),
> but can't find it in my mail archive. Might have been off-list at the time.
>
> Question: Instead of re-acquiring the buffer (and thus faking the
> non-assignment), wouldn't it be better to acquire the new buffer in a temp,
> and only overwrite the old "Py_buffer" if all is fine with it?

That makes a lot more sense to me, assuming overwriting is cheap
enough. (Can an error happen in releasing the old value?)
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Safer default exception handling with return type annotations?

[Robert Bradshaw]

On Tue, Sep 5, 2017 at 10:44 PM, Stefan Behnel  wrote:
> Robert Bradshaw schrieb am 06.09.2017 um 07:21:
>> I'm not a huge fan of behaving differently depending on what syntax
>> was used to annotate the return type--I'd rather they be 100% aliases
>> of each other.
>
> Regarding this bit - I already chose to implement some differences for
> annotation typing. Mainly, if you say
>
> def f(x: int) -> float:
> return x
>
> then the (plain "def") function will actually be typed as "double
> f(object)"., assuming that you probably meant the Python types and not the
> C types. If you want the C types "int" and "float", you have to use either
> of these:
>
> def f1(x: cython.int) -> cython.float:
> return x
>
> cpdef float f2(int x):
> return x
>
> That is because the main use case of signature annotations is Python code
> compatibility, so I tried to change the semantics as little as possible
> from what the code would be expected to do in Python.

What about

def f(x: float) -> int
  return x * 2

would that throw an error if x was, say, a str? I think float -> c
double but int -> python object will be surprising. I also worry a bit
about x: float being enforced but x: List[float] not being so.

> I also considered if distinguishing between .py and .pyx files would make
> sense here, but adapting the type interpretation to that felt much worse
> than the above,

Agreed, -1 to doing this.

> which is at least consistent regarding the typing scheme
> that you use.
>
> I think this type interpretation is a reasonable, use case driven
> difference to make. Thus my question if we should extend this to the
> exception declaration.

I suppose you've already made a case for deviating...

I guess I think it'd be nice to change the default universally, but
that's perhaps a bigger conversation.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Safer default exception handling with return type annotations?

[Robert Bradshaw]

On Mon, Sep 4, 2017 at 12:32 PM, Stefan Behnel  wrote:
> Hi all,
>
> since PEP 484 type annotations are scheduled for the next release now, I
> wonder if we should take the opportunity to change the defaults for the
> exception handling of typed cdef functions.
>
> Currently, if you define a function like this:
>
> cdef int func():
> raise ValueError()
>
> It will not actually raise the error but print and ignore it. In order to
> make it safe, you have to say
>
> cdef int func() except -1:
> raise ValueError()

A huge +1 to making exception propagation implicitly the default.
(Would we want to introduce syntax to preserve the old behavior
though?) There is the downside of a slight performance regression (we
should measure exactly how slight) and leaving the exception set
(which could actually be desirable) but exception swallowing is almost
never what a user wants.

> It has always been like this, but with PEP 484 type annotations, it becomes
> much nicer to keep Cython code Python compatible and actually running in
> Python, so that the following is safe when interpreted but not when compiled:
>
> @cython.cfunc
> def func(x: cython.int) -> cython.int:
> if x < 0:
> raise ValueError()
> return x * 2
>
> Since this is new syntax, and no code exists yet that uses it, it would be
> safe to change the default here, and to automatically declare this function
> as "except? -MIN_INT", for example. The same applies to functions returning
> pointers ("except? NULL") or even structs, which could be declared "except *".
>
> This would lead to inconsistencies within Cython, but I think the use case
> of Python compatible behaviour is important enough to consider this, and
> for code using this syntax, it also feels less likely that the caller is
> actual C code that couldn't handle that exception.

I'm not a huge fan of behaving differently depending on what syntax
was used to annotate the return type--I'd rather they be 100% aliases
of each other.

I would be willing to consider letting un-annotated extern functions
have different defaults than non-extern ones. This would, of course,
make the two type signatures incompatible (though presumably an
except? could be used wherever a noexcept one is needed.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Release candidate for 0.26.1

[Robert Bradshaw]

FYI, see https://travis-ci.org/cython-testbed which could be useful.

On Fri, Aug 25, 2017 at 9:52 AM, Stefan Behnel  wrote:
> Hi all,
>
> I've tagged a release candidate for the 0.26.1 bug fix release. Please give
> it some final testing.
>
> https://github.com/cython/cython/archive/0.26.1rc1.tar.gz
>
> Here is the changelog:
>
> https://github.com/cython/cython/blob/0.26.1rc1/CHANGES.rst
>
> Stefan
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Dropping support for Python 3.2

[Robert Bradshaw]

+1 to this.

On Fri, Aug 25, 2017 at 1:14 PM, Stefan Behnel  wrote:
> Hi all,
>
> the release branch still builds ok with Python 3.2, but master doesn't, and
> I'd rather not spend my time on fixing that. Python 3.2 has reached its
> final end-of-life in February 2016, so it's really dead for good now.
>
> https://www.python.org/dev/peps/pep-0392/
>
> I'm not really expecting any objections. ;) Unless someone stands up to
> invest time into making all tests pass in Py3.2 again, I'll rip out the
> Py3.2-only usage of 2to3 from setup.py during the next weeks, and also the
> small set of special cases here and there in the C code.
>
> Stefan
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Preparations for Cython 0.26.1 and 0.27

[Robert Bradshaw]

On Thu, Aug 17, 2017 at 5:18 AM, Stefan Behnel  wrote:
> Stefan Behnel schrieb am 02.08.2017 um 11:44:
>> Given the current set of bugs and regressions in 0.26, I think we should
>> start preparing a 0.26.1 release. I've already updated the release branch
>> with recent fixes. It's currently the same as master, but I'd like to
>> separate the branches soon in order to start merging features for 0.27 to
>> avoid bit-rot.
>>
>> The following regressions seem worth looking into for 0.26.1:
>>
>> https://github.com/cython/cython/issues/1790
>>
>> https://github.com/cython/cython/issues/1788
>
> Robert managed to fix these. I think we're ready for a release candidate
> then, or is there anything still open?

Any updates on this?
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Preparations for Cython 0.26.1 and 0.27

[Robert Bradshaw]

+1, I've been thinking the same. I'll have time to look into these
issues next week (if I don't manage to get to them sooner).

On Wed, Aug 2, 2017 at 2:44 AM, Stefan Behnel  wrote:
> Hi!
>
> Given the current set of bugs and regressions in 0.26, I think we should
> start preparing a 0.26.1 release. I've already updated the release branch
> with recent fixes. It's currently the same as master, but I'd like to
> separate the branches soon in order to start merging features for 0.27 to
> avoid bit-rot.
>
> The following regressions seem worth looking into for 0.26.1:
>
> https://github.com/cython/cython/issues/1790
>
> https://github.com/cython/cython/issues/1788
>
> Anything else of importance?
>
> Stefan
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 beta

[Robert Bradshaw]

The release is out. Thanks everyone for the feedback on and off list.

On Jul 12, 2017 1:05 PM, "Jeroen Demeyer"  wrote:

> On 2017-07-12 17:13, Robert Bradshaw wrote:
>
>> Could also be https://github.com/cython/cython/pull/1728 , though of
>> course correctness trumps speed.
>>
>
> That would be unlikely since we currently do *NOT* use binding=True in
> SageMath, so that patch should not affect anything.
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] [cython-users] Cython 0.26 release candidate.

[Robert Bradshaw]

That change has been merged, as well as a couple of (should be
completely safe) additional declarations and warning fixes. New
release candidate up at
https://github.com/cython/cython/archive/0.26rc2.zip . I plan to make
this the actual release tomorrow.

On Sun, Jul 16, 2017 at 12:22 AM, Stefan Behnel  wrote:
> Robert Bradshaw schrieb am 16.07.2017 um 01:19:
>> On Sat, Jul 15, 2017 at 2:44 AM, Jeroen Demeyer wrote:
>>> Do you intend to do something about the speed regression due to
>>> https://github.com/cython/cython/pull/1719
>>>
>>> I'm happy with a yes/no answer just to know what to expect.
>>>
>> Had a chance to take a look at this:
>>
> https://github.com/cython/cython/commit/5e2dc4f78e2c9b560b07f73e664f3e70f7a8cfc8
>> I think this is safe enough to get into the next release.
>
> I'd suggest this change:
>
> https://github.com/cython/cython/pull/1773
>
> Stefan
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups 
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to cython-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 release candidate.

[Robert Bradshaw]

Some last-minute commits to unbreak Py3.7. New release candidate at

http://bradshawfamily.net/cython/Cython-0.26rc1.tar.gz (sdist)

or

https://github.com/cython/cython/archive/0.26rc1.zip (git)

On Fri, Jul 14, 2017 at 5:28 PM, Robert Bradshaw  wrote:
> Looks like everything's gone smoothly with the beta. I have prepared a
> release candidate, likely to become the final release.
>
> The git sources are available at
> https://github.com/cython/cython/archive/0.26rc0.zip
> I have also created an sdist, up at
> http://bradshawfamily.net/cython/Cython-0.26rc0.tar.gz
>
> - Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Cython 0.26 release candidate.

[Robert Bradshaw]

Looks like everything's gone smoothly with the beta. I have prepared a
release candidate, likely to become the final release.

The git sources are available at
https://github.com/cython/cython/archive/0.26rc0.zip
I have also created an sdist, up at
http://bradshawfamily.net/cython/Cython-0.26rc0.tar.gz

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 beta

[Robert Bradshaw]

On Wed, Jul 12, 2017 at 4:58 AM, Jeroen Demeyer  wrote:
> On 2017-07-10 23:51, Robert Bradshaw wrote:
>>
>> https://github.com/cython/cython/archive/0.26b2.zip
>
>
> Everything fine in SageMath!

Thanks!

> I did notice a speed regression and I suspect that
> https://github.com/cython/cython/pull/1719 might be the reason.

Could also be https://github.com/cython/cython/pull/1728 , though of
course correctness trumps speed.

As for timing, I hope to get an RC out this week, releasing shortly after.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 beta

[Robert Bradshaw]

https://github.com/cython/cython/archive/0.26b2.zip

On Mon, Jul 10, 2017 at 2:01 PM, Robert Bradshaw  wrote:
> Scratch that, please try https://github.com/cython/cython/archive/0.26b1.zip
>
> On Mon, Jul 10, 2017 at 12:49 PM, Robert Bradshaw  wrote:
>> Another beta: https://github.com/cython/cython/archive/0.26b1.zip
>>
>> On Mon, Jul 10, 2017 at 10:59 AM, Robert Bradshaw  wrote:
>>> https://github.com/scikit-learn/scikit-learn/pull/9311 looks like a
>>> good resolution. The infinite recursion issue has been fixed as well.
>>>
>>> On Fri, Jul 7, 2017 at 2:10 PM, Andreas Mueller  wrote:
>>>> Hey folks.
>>>> So scikit-learn master doesn't build with 0.26b0:
>>>> https://github.com/scikit-learn/scikit-learn/issues/9272
>>>>
>>>> Robert already looked into this, but it hasn't been resolved yet:
>>>> https://github.com/scikit-learn/scikit-learn/pull/9133
>>>>
>>>> We want to create a release candidate today and release soon(ish).
>>>> It would be great if you could give us hand so that our release
>>>> builds with your release :)
>>>>
>>>> Cheers,
>>>> Andy
>>>>
>>>>
>>>> On 07/03/2017 04:29 PM, Robert Bradshaw wrote:
>>>>>
>>>>> There have been a lot of changes since Cython 0.25.2, please test and
>>>>> report. https://github.com/cython/cython/archive/0.26b0.zip
>>>>>
>>>>> At this point I anticipate only bugfixes will be merged before the
>>>>> final release.
>>>>> ___
>>>>> cython-devel mailing list
>>>>> cython-devel@python.org
>>>>> https://mail.python.org/mailman/listinfo/cython-devel
>>>>
>>>>
>>>> ___
>>>> cython-devel mailing list
>>>> cython-devel@python.org
>>>> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 beta

[Robert Bradshaw]

Scratch that, please try https://github.com/cython/cython/archive/0.26b1.zip

On Mon, Jul 10, 2017 at 12:49 PM, Robert Bradshaw  wrote:
> Another beta: https://github.com/cython/cython/archive/0.26b1.zip
>
> On Mon, Jul 10, 2017 at 10:59 AM, Robert Bradshaw  wrote:
>> https://github.com/scikit-learn/scikit-learn/pull/9311 looks like a
>> good resolution. The infinite recursion issue has been fixed as well.
>>
>> On Fri, Jul 7, 2017 at 2:10 PM, Andreas Mueller  wrote:
>>> Hey folks.
>>> So scikit-learn master doesn't build with 0.26b0:
>>> https://github.com/scikit-learn/scikit-learn/issues/9272
>>>
>>> Robert already looked into this, but it hasn't been resolved yet:
>>> https://github.com/scikit-learn/scikit-learn/pull/9133
>>>
>>> We want to create a release candidate today and release soon(ish).
>>> It would be great if you could give us hand so that our release
>>> builds with your release :)
>>>
>>> Cheers,
>>> Andy
>>>
>>>
>>> On 07/03/2017 04:29 PM, Robert Bradshaw wrote:
>>>>
>>>> There have been a lot of changes since Cython 0.25.2, please test and
>>>> report. https://github.com/cython/cython/archive/0.26b0.zip
>>>>
>>>> At this point I anticipate only bugfixes will be merged before the
>>>> final release.
>>>> ___
>>>> cython-devel mailing list
>>>> cython-devel@python.org
>>>> https://mail.python.org/mailman/listinfo/cython-devel
>>>
>>>
>>> ___
>>> cython-devel mailing list
>>> cython-devel@python.org
>>> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 beta

[Robert Bradshaw]

Another beta: https://github.com/cython/cython/archive/0.26b1.zip

On Mon, Jul 10, 2017 at 10:59 AM, Robert Bradshaw  wrote:
> https://github.com/scikit-learn/scikit-learn/pull/9311 looks like a
> good resolution. The infinite recursion issue has been fixed as well.
>
> On Fri, Jul 7, 2017 at 2:10 PM, Andreas Mueller  wrote:
>> Hey folks.
>> So scikit-learn master doesn't build with 0.26b0:
>> https://github.com/scikit-learn/scikit-learn/issues/9272
>>
>> Robert already looked into this, but it hasn't been resolved yet:
>> https://github.com/scikit-learn/scikit-learn/pull/9133
>>
>> We want to create a release candidate today and release soon(ish).
>> It would be great if you could give us hand so that our release
>> builds with your release :)
>>
>> Cheers,
>> Andy
>>
>>
>> On 07/03/2017 04:29 PM, Robert Bradshaw wrote:
>>>
>>> There have been a lot of changes since Cython 0.25.2, please test and
>>> report. https://github.com/cython/cython/archive/0.26b0.zip
>>>
>>> At this point I anticipate only bugfixes will be merged before the
>>> final release.
>>> ___
>>> cython-devel mailing list
>>> cython-devel@python.org
>>> https://mail.python.org/mailman/listinfo/cython-devel
>>
>>
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 beta

[Robert Bradshaw]

https://github.com/scikit-learn/scikit-learn/pull/9311 looks like a
good resolution. The infinite recursion issue has been fixed as well.

On Fri, Jul 7, 2017 at 2:10 PM, Andreas Mueller  wrote:
> Hey folks.
> So scikit-learn master doesn't build with 0.26b0:
> https://github.com/scikit-learn/scikit-learn/issues/9272
>
> Robert already looked into this, but it hasn't been resolved yet:
> https://github.com/scikit-learn/scikit-learn/pull/9133
>
> We want to create a release candidate today and release soon(ish).
> It would be great if you could give us hand so that our release
> builds with your release :)
>
> Cheers,
> Andy
>
>
> On 07/03/2017 04:29 PM, Robert Bradshaw wrote:
>>
>> There have been a lot of changes since Cython 0.25.2, please test and
>> report. https://github.com/cython/cython/archive/0.26b0.zip
>>
>> At this point I anticipate only bugfixes will be merged before the
>> final release.
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Cython 0.26 beta

[Robert Bradshaw]

There have been a lot of changes since Cython 0.25.2, please test and
report. https://github.com/cython/cython/archive/0.26b0.zip

At this point I anticipate only bugfixes will be merged before the
final release.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 release

[Robert Bradshaw]

I have made the bad vtable generation a warning rather than an error
for now to ease the transition. I was hoping to be able to reproduce
(and fix) https://github.com/cython/cython/issues/1744 but still
having no luck so I'm going to proceed with a beta to not stall the
process.

On Fri, Jun 23, 2017 at 6:32 PM, David Roe  wrote:
>
>
> On Wed, Jun 21, 2017 at 12:41 PM, Jeroen Demeyer 
> wrote:
>>
>> On 2017-06-21 20:25, Robert Bradshaw wrote:
>>>
>>> I suppose in that case we just got lucky that no one cimported these
>>> classes and tried to use methods after this point in the vtable...
>>
>>
>> I think that's the case indeed.
>
>
> In fact, Julian Rueth and I eventually did cimport one of these classes
> (Polynomial_generic_dense), eventually leading to
> https://github.com/cython/cython/issues/1732.
>
> A warning does seem like it would make fixing existing code easier.
> David
>>
>>
>>
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Next Cython Release

[Robert Bradshaw]

I'm happy with including it at this early stage.

On Sun, Jun 25, 2017 at 2:58 PM, Stefan Behnel  wrote:
> Adrien Guinet schrieb am 17.06.2017 um 11:39:
>> As everyone is making his "wishlist" for the next Cython release, I
>> think it could be nice if https://github.com/cython/cython/pull/1607
>> (Pythran support for Numpy operation) could be integrated. I just pushed
>> a commit that makes the usage easier, just need travis to confirme and
>> squash everything to a single commit.
>
> Looks good to me now. I'll leave it to Robert to decide if this should be
> included in this release. I think it's ready, but I'm not in the target
> group and haven't tested it myself.
>
> Stefan
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] On cdef and cpdef discrepencies

[Robert Bradshaw]

On Thu, Jun 22, 2017 at 1:25 AM, Jeroen Demeyer  wrote:
> On 2017-06-21 21:51, Robert Bradshaw wrote:
>>
>> I'll see if I can put together a patch.
>
>
> For what? For Sage? For Cython?

I was thinking for Sage. I was assuming there was just an oddball
collection of oversights accumulated over the yeras.

> Anyway, I think that coping with this new error message is not particularly
> hard, just a lot of work. That's why I would prefer a warning instead of an
> error. That way, we can easily see how bad the problem is and what needs to
> be fixed and we fix it gradually (say, before Cython 0.27 which could make
> this a hard error).

I was digging into this a bit more, and the issue seems to be entirely
the arithmetic methods like _add_, and was introduced one year and one
day ago: 
https://github.com/sagemath/sage/commit/e1d2ba46574273ec4d358e654ed195a82b81a654
. It'd be cleaner if these were in fact cpdef methods, but our
category method injection framework requires that the class not have
an _add_ defined at the Python level. But any overrides must be cpdef
methods lest a (Python) call to x._add_(y) succeed but not find the
override.

Perhaps it'd be more clear to define cpdef Element._add_ whose
implementation directly calls getattr_from_category rather than
(self)._add_.

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 release

[Robert Bradshaw]

On Wed, Jun 21, 2017 at 12:41 PM, Jeroen Demeyer  wrote:
> On 2017-06-21 20:25, Robert Bradshaw wrote:
>>
>> I suppose in that case we just got lucky that no one cimported these
>> classes and tried to use methods after this point in the vtable...
>
> I think that's the case indeed.

I'll see if I can put together a patch.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 release

[Robert Bradshaw]

On Wed, Jun 21, 2017 at 9:59 AM, Jeroen Demeyer  wrote:
> On 2017-06-21 17:44, Robert Bradshaw wrote:
>>
>> Is it cdef->cpdef, or something else?
>
> Yes, it is.
>
>> It's possible that the error needs to be made narrower, but I'd really
>> like to avoid incorrect code that could lead to wrong methods being
>> called.
>
> Fine... we can still patch Cython in Sage.

I suppose in that case we just got lucky that no one cimported these
classes and tried to use methods after this point in the vtable...
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 release

[Robert Bradshaw]

On Wed, Jun 21, 2017 at 5:29 AM, Jeroen Demeyer  wrote:
> Could the error from
> https://github.com/cython/cython/issues/1732
> at
> https://github.com/cython/cython/blob/master/Cython/Compiler/Symtab.py#L2099
> be a warning instead?
>
> SageMath has quite a bit of code hitting this error. It would make our life
> easier if we could fix it gradually instead of being forced to fix it all
> before upgrading to Cython 0.26.

Could you give some examples? Is it cdef->cpdef, or something else?
It's possible that the error needs to be made narrower, but I'd really
like to avoid incorrect code that could lead to wrong methods being
called.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.26 release

[Robert Bradshaw]

Thanks. Fixed.

On Tue, Jun 20, 2017 at 7:53 AM, Jeroen Demeyer  wrote:
> This is breaking SageMath:
> https://github.com/cython/cython/issues/1739
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Cython 0.26 release

[Robert Bradshaw]

I'd like to officially start off the release process with an alpha
candidate. This does necessarily contain everything that will be in
the final release, but given the amount of time that has passed I
would recommend everyone test this out with their projects sooner
rather than later.

Download: https://github.com/cython/cython/archive/0.26.alpha0.zip
Initial changelog: https://github.com/cython/cython/blob/0.26.alpha0/CHANGES.rst

Please feel free to respond to this thread or file bugs as necessary.

Thanks,
Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Next Cython Release

[Robert Bradshaw]

On Sat, Jun 10, 2017 at 10:34 AM, Julian Rüth  wrote:
> Hello Cython developers,
>
> some people over at conda-forge would like to use features that are
> already in master but have not been released yet
> (https://github.com/conda-forge/cython-feedstock/pull/20). We would
> prefer not to patch and older release for Cython but just wait for the
> next release. However, since there has not been a release in 6 months,
> we're wondering if there are any plans for a release?

I've been wanting to do a release for a while, just haven't gotten
around to it. Yes, let's get one out soon.

> PS: Not sure if this is the right list for this but the link to the
> bug-tracker on https://mail.python.org/mailman/listinfo/cython-devel
> goes to http://cython.org/trac/cython_trac/ which produces a 404.

As mentioned, we're using github these days. I added redirects
(including specific tickets like http://cython.org/trac/ticket/101 )
but seemed to have missed this link, thanks for pointing that out.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Automatically add include_dirs for externs (#1654)

[Robert Bradshaw]

I'm in favor of the idea (basically attempting to resolve header files
relative to the cython file including them and transitively passing
these paths to modules cimporting them); haven't had time to look at
the code yet or think if there could be negative implications.

On Tue, Apr 11, 2017 at 3:00 AM, Jeroen Demeyer  wrote:
> Dear Cython developers,
>
> as part of a big effort to make SageMath less monolithic and make it part of
> a larger ecosystem, we are splitting off certain modules from SageMath as
> separate packages. cysignals was an early example of this.
>
> Given that we use Cython a lot, it is really important for us that
> inter-package C and Cython interfaces work well. One blocking issue is
> https://github.com/cython/cython/pull/1654
>
> That PR improves the way how "cdef extern from" files are found: it will
> automatically add a disutils include_dirs argument to the Extension if it's
> needed to find to extern file. It is needed whenever you have a package
> which installs .pxd and .h files in site-packages.
>
>
> Cheers,
> Jeroen.
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Treatment of right-hand side after assignment

[Robert Bradshaw]

On Mon, Apr 10, 2017 at 5:57 AM, Clemens Hofreither
 wrote:
>
> On Sun, Apr 9, 2017 at 8:00 AM, Robert Bradshaw  wrote:
> > On Sun, Apr 2, 2017 at 9:42 AM, Clemens Hofreither 
> > wrote:
> >>
> >> Hey all,
> >>
> >> (I hope this list isn't dead.)
> >>
> >> I'm reading up on the Cython codebase because I'm trying to implement
> >> a small feature (more on that later). There's one thing that really
> >> confuses me currently, and that's the distinction between
> >> generate_post_assignment_code and generate_disposal_code. From the
> >> comments, it appears that exactly one of these should be called, and
> >> which one it is depends on whether a reference from the rhs was
> >> absorbed into the lhs.
> >>
> >> So much for the theory, but in practice it seems to work differently.
> >> I'm looking at NameNode.generate_assignment_code() and
> >> IndexNode.generate_assignment_code(), which should do similar things
> >> (generate code for an assignment) except that the lhs has different
> >> structure.
> >>
> >> Yet they treat the cleanup of the rhs completely differently.
> >> IndexNode always calls rhs.generate_disposal_code. NameNode almost
> >> always calls rhs.generate_post_assignment_code.
> >>
> >> Can anyone shed light on why this cleanup is handled differently
> >> depending on the target of the assignment?
> >
> >
> > It's been a while since I've looked at this, but the best explanation is
> > that generate_post_assignment_code is used when the rhs is "moved" into the
> > lhs whereas generate_post_assignment_code is what happens for a "copy." This
> > is mostly relevant for temps, e.g. when a temp holds a reference to a Python
> > object, it can be transferred to the assignee (e.g. with NameNodes) or a new
> > reference can be created for the assignee (e.g. with IndexNodes) which
> > changes whether the temp lhs needs to be decref'ed.
> >
> > ___
> > cython-devel mailing list
> > cython-devel@python.org
> > https://mail.python.org/mailman/listinfo/cython-devel
> >
>
> Thanks for your answer. I understood as much about the difference with
> temporaries and copies.
>
> My question was why the RIGHT-hand side is treated differently in
> NameNode.generate_assignment_code() and
> IndexNode.generate_assignment_code(). My understanding is that
> NameNode/IndexNode here refers to the type of the left-hand side. But
> I don't see why whether or not the RHS is a temporary should depend on
> the type of the LHS. Therefore it seems confusing that the code in
> these two methods uses different cleanup for the RHS.


One can generally "transfer" ownership to a NameNode, but this is not
the case with an IndexNode (as it depends on what the [] operator was
overloaded to do). One could always incref the NameNode and use
generate_disposal_code, but using generate_post_assignment_code saves
an unnecessary incref/decref pair.

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Treatment of right-hand side after assignment

[Robert Bradshaw]

On Sun, Apr 2, 2017 at 9:42 AM, Clemens Hofreither 
wrote:

> Hey all,
>
> (I hope this list isn't dead.)
>
> I'm reading up on the Cython codebase because I'm trying to implement
> a small feature (more on that later). There's one thing that really
> confuses me currently, and that's the distinction between
> generate_post_assignment_code and generate_disposal_code. From the
> comments, it appears that exactly one of these should be called, and
> which one it is depends on whether a reference from the rhs was
> absorbed into the lhs.
>
> So much for the theory, but in practice it seems to work differently.
> I'm looking at NameNode.generate_assignment_code() and
> IndexNode.generate_assignment_code(), which should do similar things
> (generate code for an assignment) except that the lhs has different
> structure.
>
> Yet they treat the cleanup of the rhs completely differently.
> IndexNode always calls rhs.generate_disposal_code. NameNode almost
> always calls rhs.generate_post_assignment_code.
>
> Can anyone shed light on why this cleanup is handled differently
> depending on the target of the assignment?


It's been a while since I've looked at this, but the best explanation is
that generate_post_assignment_code is used when the rhs is "moved" into the
lhs whereas generate_post_assignment_code is what happens for a "copy."
This is mostly relevant for temps, e.g. when a temp holds a reference to a
Python object, it can be transferred to the assignee (e.g. with NameNodes)
or a new reference can be created for the assignee (e.g. with IndexNodes)
which changes whether the temp lhs needs to be decref'ed.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] verify_resolution_GH1533

[Robert Bradshaw]

Ah, if the enum is unsigned and has the same size as unsigned long, this
could happen. I pushed
https://github.com/cython/cython/commit/d92a718a26c9354fbf35f31a17de5c069865a447
for future release.

On Tue, Jan 24, 2017 at 4:53 PM, Robert Bradshaw  wrote:

> Disabling only works on full test suites, e.g. you'd have to disable all
> of cpdef_enums.
>
> What kind of machine turns this into a long?
>
> On Tue, Jan 24, 2017 at 8:56 AM, Yaroslav Halchenko 
> wrote:
>
>> Dear Cython gurus,
>>
>> How to skip this test?
>>
>> It is causing failures on i386:
>> https://github.com/cython/cython/issues/1548
>> and I have tried to skip it, so my test running line now is
>>
>> set -e; for P in 2.7 3.5; do \
>>  PYTHONPATH=`/bin/ls -d /build/cython-0.25.2/build/lib.*-$P` \
>>   /usr/bin/python$P runtests.py --shard_count=16 \
>> --no-refnanny -v -v --exclude="(parallel|Debugger|
>> annotate_html|numpy_test|verify_resolution_GH1533)"
>> --work-dir=build/work-dir 2>&1; \
>> done
>>
>> NB numpy_test skip due to https://github.com/cython/cython/issues/1589
>>
>> but the test still runs and fails:
>>
>>
>>
>> FAIL: verify_resolution_GH1533 (line 90) (cpdef_enums.__test__)
>> Doctest: cpdef_enums.__test__.verify_resolution_GH1533 (line 90)
>> --
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
>> raise self.failureException(self.format_failure(new.getvalue()))
>> AssertionError: Failed doctest test for 
>> cpdef_enums.__test__.verify_resolution_GH1533
>> (line 90)
>>   File 
>> "/build/cython-0.25.2/build/work-dir/15/run/c/cpdef_enums/cpdef_enums.so",
>> line unknown line number, in verify_resolution_GH1533 (line 90)
>>
>> --
>> File 
>> "/build/cython-0.25.2/build/work-dir/15/run/c/cpdef_enums/cpdef_enums.so",
>> line ?, in cpdef_enums.__test__.verify_resolution_GH1533 (line 90)
>> Failed example:
>> verify_resolution_GH1533()
>> Expected:
>> 3
>> Got:
>> 3L
>>
>>
>> --
>> Ran 676 tests in 247.426s
>>
>> FAILED (failures=1)
>>
>>
>>
>> quick reply would be appreciate -- running out of time to upload 0.25.2
>> into
>> debian for stretch full freeze
>>
>> Cheers
>> --
>> Yaroslav O. Halchenko
>> Center for Open Neuroscience http://centerforopenneuroscience.org
>> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
>> Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
>> WWW:   http://www.linkedin.com/in/yarik
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>>
>
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] verify_resolution_GH1533

[Robert Bradshaw]

Disabling only works on full test suites, e.g. you'd have to disable all of
cpdef_enums.

What kind of machine turns this into a long?

On Tue, Jan 24, 2017 at 8:56 AM, Yaroslav Halchenko 
wrote:

> Dear Cython gurus,
>
> How to skip this test?
>
> It is causing failures on i386:
> https://github.com/cython/cython/issues/1548
> and I have tried to skip it, so my test running line now is
>
> set -e; for P in 2.7 3.5; do \
>  PYTHONPATH=`/bin/ls -d /build/cython-0.25.2/build/lib.*-$P` \
>   /usr/bin/python$P runtests.py --shard_count=16 \
> --no-refnanny -v -v --exclude="(parallel|Debugger|
> annotate_html|numpy_test|verify_resolution_GH1533)"
> --work-dir=build/work-dir 2>&1; \
> done
>
> NB numpy_test skip due to https://github.com/cython/cython/issues/1589
>
> but the test still runs and fails:
>
>
>
> FAIL: verify_resolution_GH1533 (line 90) (cpdef_enums.__test__)
> Doctest: cpdef_enums.__test__.verify_resolution_GH1533 (line 90)
> --
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
> raise self.failureException(self.format_failure(new.getvalue()))
> AssertionError: Failed doctest test for 
> cpdef_enums.__test__.verify_resolution_GH1533
> (line 90)
>   File 
> "/build/cython-0.25.2/build/work-dir/15/run/c/cpdef_enums/cpdef_enums.so",
> line unknown line number, in verify_resolution_GH1533 (line 90)
>
> --
> File 
> "/build/cython-0.25.2/build/work-dir/15/run/c/cpdef_enums/cpdef_enums.so",
> line ?, in cpdef_enums.__test__.verify_resolution_GH1533 (line 90)
> Failed example:
> verify_resolution_GH1533()
> Expected:
> 3
> Got:
> 3L
>
>
> --
> Ran 676 tests in 247.426s
>
> FAILED (failures=1)
>
>
>
> quick reply would be appreciate -- running out of time to upload 0.25.2
> into
> debian for stretch full freeze
>
> Cheers
> --
> Yaroslav O. Halchenko
> Center for Open Neuroscience http://centerforopenneuroscience.org
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Changing the default value of exception propagation

[Robert Bradshaw]

On Thu, Jan 5, 2017 at 2:31 AM, Stefan Behnel  wrote:
> Hi Robert!
>
> Robert Bradshaw schrieb am 04.01.2017 um 07:46:
>> By default a cdef (or cpdef) function returning C type suppresses all
>> exceptions, which is quite surprising to new (and old) users, and
>> makes debugging difficult. I would propose that we make exception
>> propagation the default.
>>
>> The primary motivation to not do this would be for fear of performance
>> regression, especially for external functions that can't possibly
>> raise exceptions. While the the catch-all "except *" does seem to have
>> non-trivial overhead, the "except? value" version looks
>> indistinguishable from no exception throwing at all (in tight loops,
>> yay branch prediction, would be interested in what other people see
>> [1]).
>>
>> Does this sound like a reasonable proposal?
>
> Sounds good to me.
>
>
>> There are still some open questions, e.g.
>>
>> - What value should we pick. We want something unlikely, but
>> recognizable to external code (e.g. 0xBadBad).
>
> Depends on the value type, obviously. Seems like there should be one error
> value per C type, e.g.
>
> pointer: NULL?  (or maybe 1 ?)
> char: 0xEF
> short: 0xBad
> int/long/etc.: 0xBadBad

It'd be nice if it were consistent, e.g. (type)0xBadBad when possible.
But I think we'll need to do this per type.

> What about C++ in general? Should we also default to catching C++
> exceptions? (Separate proposal, but IIRC that's also still an open issue?)

I think the primary surprise is from Python users who make (or
convert) a couple of c[p]def functions and are surprised in the change
in behavior. Hadn't thought about wrapping every single C function
call in a try...catch statement. Wonder what the overhead (time and
space) of that would be.

>> - What about non-numeric/pointer/reference types; are there other
>> sentinel values we could use?
>
> I can't see a way to handle struct/union. Well, we could handle a union
> that involves an integer, but returning a plain union is fairly unlikely to
> occur in practice, so the main thing here are structs (and pass-by-value
> C++ objects, I guess).
>
>> - Should we default to except * in that case? Presumably these are
>> likely to be more expensive functions anyways (making the relative
>> cost lower).
>
> Depends on how the C compiler implements them. If it inlines a function
> that returns a struct (e.g. because the function is only called in one
> place), the overhead of "returning" that struct would become negligible and
> the exception check overhead could easily show up.
>
> But I agree with your intuition that returning something non-trivial from a
> function suggests a non-trivial implementation of the function.
>
>
>> - Or could we use, essentially, reinterpret_cast to get a possible
>> error value? This could work for PODs (including C structs) but
>> destructors could wreck havoc here.
>
> A solution for C would already help, regardless of anything we could or
> could not do for C++.

We need to consider C++ though, especially as things declared as
structs may actually be C++ classes, or contain C++ class members.

>> - Should we add syntax (e.g. "except -") to manually obtain the old
>> behavior?
>
> Question is: when would we want that? Should we really trade the weird
> WriteUnraisable() behaviour for speed here? I think, an unraisable
> exception is always a bug, even if it doesn't appear in practice. If
> there's not enough memory, exceptions can occur in any place, and shadowing
> them in unexpected places can do arbitrary harm to your system, especially
> if it doesn't crash but continues running in an inconsistent state.

There is the case that the function in question is a callback with no
ways to report errors--one might prefer to at least print something.

>> Or should such be done by surrounding the entire block with
>> an try/bare except statement? This wouldn't catch argument-parsing
>> errors.
>
> Argument parsing errors cannot occur for cdef functions and C level calls
> to cpdef functions, as they would appear on caller side, before even
> calling the function. For Python level calls, I don't think we need to care.
>
>> (Another crazy idea, though I'm not sold, is lifting an
>> initial try and with block that spans the full function body to above
>> the argument parsing--this could simply the gil syntax as well.)
>
> This is unprecedented in Python. If you pass positional arguments into a
> function that only accepts keywo

[Cython] Changing the default value of exception propagation

[Robert Bradshaw]

By default a cdef (or cpdef) function returning C type suppresses all
exceptions, which is quite surprising to new (and old) users, and
makes debugging difficult. I would propose that we make exception
propagation the default.

The primary motivation to not do this would be for fear of performance
regression, especially for external functions that can't possibly
raise exceptions. While the the catch-all "except *" does seem to have
non-trivial overhead, the "except? value" version looks
indistinguishable from no exception throwing at all (in tight loops,
yay branch prediction, would be interested in what other people see
[1]).

Does this sound like a reasonable proposal?

There are still some open questions, e.g.

- What value should we pick. We want something unlikely, but
recognizable to external code (e.g. 0xBadBad).

- What about non-numeric/pointer/reference types; are there other
sentinel values we could use?
- Should we default to except * in that case? Presumably these are
likely to be more expensive functions anyways (making the relative
cost lower).
- Or could we use, essentially, reinterpret_cast to get a possible
error value? This could work for PODs (including C structs) but
destructors could wreck havoc here.

- Should we add syntax (e.g. "except -") to manually obtain the old
behavior? Or should such be done by surrounding the entire block with
an try/bare except statement? This wouldn't catch argument-parsing
errors. (Another crazy idea, though I'm not sold, is lifting an
initial try and with block that spans the full function body to above
the argument parsing--this could simply the gil syntax as well.)


- Robert


[1] https://gist.github.com/robertwb/0ef6bfaa3c9b2ea11ed64d1c1a0bda43

time_with_nothing   0.0322368144989   3.22368144989e-08
time_with_star   0.097442150116   9.7442150116e-08
time_with_value   0.0940091609955   9.40091609955e-08
time_with_maybe_value   0.10076379776   1.0076379776e-07

time_with_nothing   0.0767869949341   3.8393497467e-08
time_with_star   0.189253091812   9.46265459061e-08
time_with_value   0.176230192184   8.81150960922e-08
time_with_maybe_value   0.179567098618   8.97835493088e-08

time_with_nothing   0.110851049423   3.69503498077e-08
time_with_star   0.268408060074   8.9469353358e-08
time_with_value   0.254024982452   8.46749941508e-08
time_with_maybe_value   0.244876861572   8.16256205241e-08

time_with_nothing   0.138132095337   3.45330238342e-08
time_with_star   0.345059156418   8.62647891045e-08
time_with_value   0.329736948013   8.24342370033e-08
time_with_maybe_value   0.341785907745   8.54464769363e-08


time_with_nothing   0.0680961608887   6.80961608887e-10
time_with_star   0.285541057587   2.85541057587e-09
time_with_value   0.0707340240479   7.07340240479e-10
time_with_maybe_value   0.0762150287628   7.62150287628e-10

time_with_nothing   0.148243904114   7.41219520569e-10
time_with_star   0.560220956802   2.80110478401e-09
time_with_value   0.136430978775   6.82154893875e-10
time_with_maybe_value   0.143126964569   7.15634822845e-10

time_with_nothing   0.211837053299   7.06123510997e-10
time_with_star   0.812189102173   2.70729700724e-09
time_with_value   0.197682142258   6.58940474192e-10
time_with_maybe_value   0.203264951706   6.7754983902e-10

time_with_nothing   0.283420085907   7.08550214767e-10
time_with_star   1.09991884232   2.74979710579e-09
time_with_value   0.281104803085   7.02762007713e-10
time_with_maybe_value   0.288016080856   7.20040202141e-10

DONE
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.25.2 released

[Robert Bradshaw]

On Dec 8, 2016 10:29 PM, "Robert Bradshaw"  wrote:

I pushed a bugfix release to PyPi this afternoon. This is a minor
release with the following fixes:

- Fixes several issues with C++ template deduction.
- Fixes a issue with bound method type inference (Github issue #551).
- Fixes a bug with cascaded tuple assignment (Github issue #1523).
- Fixes bug with powers of pure real complex numbers (Github issue #1538).
- Fixes bug with namespace resolution of enum values (Github issue #1533).
- Fixes bug with references and exception handling (Github issue #1519).
- Fixed or silenced many Clang warnings.

It also silences the build deprecation warnings on Python 3. It should
be fully backwards compatible with 0.25.1.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] 0.25.2b0 with 32bit fixes: (sometimes) Interrupted system call in builtin_subtype_methods_T653

[Robert Bradshaw]

When it does reproduce, is it always this particular test?

On Wed, Nov 30, 2016 at 6:46 AM, Yaroslav Halchenko
 wrote:
> A fresh one I ran into while rebuilding with fixes for 32bit (thanks
> BTW! sorry for not prompt followup/gratitude)
>
> ==
> ERROR: runTest (__main__.CythonRunTestCase)
> compiling (cpp) and running builtin_subtype_methods_T653
> --
> Traceback (most recent call last):
>   File "runtests.py", line 998, in run
> self.run_tests(result, ext_so_path)
>   File "runtests.py", line 1013, in run_tests
> self.run_doctests(self.module, result, ext_so_path)
>   File "runtests.py", line 1023, in run_doctests
> run_forked_test(result, run_test, self.shortDescription(), self.fork)
>   File "runtests.py", line 1076, in run_forked_test
> cid, result_code = os.waitpid(child_id, 0)
> OSError: [Errno 4] Interrupted system call
>
> --
> Ran 581 tests in 188.225s
>
> FAILED (errors=1)
>
> it is not 100% reproducible.  Rebuilding the package went smooth without this
> hiccup.  Just wanted to let you know
>
> might be related that the tests were ran as
>
>
> set -e; for P in 2.7 3.5; do \
>  PYTHONPATH=`/bin/ls -d /build/cython-0.25.2~b0/build/lib.*-$P` \
>   /usr/bin/python$P runtests.py --shard_count=16 \
> --no-refnanny -v -v --exclude="(parallel|Debugger|annotate_html)" 
> --work-dir=build/work-dir 2>&1; \
> done
>
> so testing in parallel to speed up the build  of the package.  This one was 
> the
> only failure I have ran into but it built fine before (but there could have
> been some changes in the core packages since then)
>
> --
> Yaroslav O. Halchenko
> Center for Open Neuroscience http://centerforopenneuroscience.org
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] 0.25.1 annotate_html UnicodeDecodeError

[Robert Bradshaw]

You can run just this test with

python runtests.py annotate_html

Could you also run with --no-cleanup so see what non-ascii characters
appear in the html file?

On Sat, Nov 19, 2016 at 7:39 PM, Yaroslav Halchenko
 wrote:
> Ran into it while building a package for Debian
>
> ==
> FAIL: annotate_html ()
> Doctest: annotate_html
> --
> Traceback (most recent call last):
>   File "/usr/lib/python3.5/doctest.py", line 2190, in runTest
> raise self.failureException(self.format_failure(new.getvalue()))
> AssertionError: Failed doctest test for annotate_html
>   File 
> "/build/cython-0.25.1/build/work-dir/run/cpp/annotate_html/annotate_html.cpython-35m-x86_64-linux-gnu.so",
>  line 1, in annotate_html
>
> --
> File 
> "/build/cython-0.25.1/build/work-dir/run/cpp/annotate_html/annotate_html.cpython-35m-x86_64-linux-gnu.so",
>  line 9, in annotate_html
> Failed example:
> with open(module_path + '.html') as html_file:
> html = html_file.read()
> Exception raised:
> Traceback (most recent call last):
>   File "/usr/lib/python3.5/doctest.py", line 1321, in __run
> compileflags, 1), test.globs)
>   File "", line 2, in 
> html = html_file.read()
>   File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
> return codecs.ascii_decode(input, self.errors)[0]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 
> 16904: ordinal not in range(128)
>
> which then leads also to
>
> --
> File 
> "/build/cython-0.25.1/build/work-dir/run/cpp/annotate_html/annotate_html.cpython-35m-x86_64-linux-gnu.so",
>  line 13, in annotate_html
> Failed example:
> assert re.search('', html)
> Exception raised:
> Traceback (most recent call last):
>   File "/usr/lib/python3.5/doctest.py", line 1321, in __run
> compileflags, 1), test.globs)
>   File "", line 1, in 
> assert re.search('', html)
> NameError: name 'html' is not defined
>
>
> I guess some locale setting/assumptions?
>
> btw -- what is the "proper" way to run this doctest alone using
> runtests.py?
>
> --
> Yaroslav O. Halchenko
> Center for Open Neuroscience http://centerforopenneuroscience.org
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Cython 0.25.2

[Robert Bradshaw]

I'm preparing another bugfix release. Try it out at

https://github.com/cython/cython/archive/966a296ac4e4862dee9a571ee412886ca3c61144.zip
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.25 released

[Robert Bradshaw]

I plan on releasing

https://github.com/cython/cython/archive/0.25.1b1.tar.gz

later today. I'd like to figure out the CentOS issue, but can't
reproduce. (To reiterate, if you were broken by this release, this is
what prereleases are for :).



Bugs fixed
--

* Fixes a bug with ``isinstance(o, Exception)`` (Github issue #1496).

* Fixes bug with ``cython.view.array`` missing utility code in some cases
  (Github issue #1502).

Other changes
-

* The distutils extension ``Cython.Distutils.build_ext`` has been reverted,
  temporarily, to be ``old_build_ext`` to give projects time to migrate.
  The new build_ext is available as ``new_build_ext``.

On Wed, Oct 26, 2016 at 9:15 AM, Robert Bradshaw  wrote:
> I'm having trouble reproducing this on any of my setups though. Let's follow
> up on https://github.com/cython/cython/issues/1499
>
> On Wed, Oct 26, 2016 at 2:11 AM, Antoine Martin 
> wrote:
>>
>> On 26/10/16 03:34, Robert Bradshaw wrote:
>> > I'm happy to announce the release of Cython 0.25 which has numerous
>> > new features and bug fixes. It is available at
>> > https://pypi.python.org/pypi/Cython
>> >
>> > We have also moved bug tracking from trac.cython.org to github issues.
>> > However, user support still remains at cython-users@, please refrain
>> > from filing issues until you have confirmed an actual bug.
>>
>> 0.25 does not build on any CentOS 6.x:
>> wget
>>
>> "https://pypi.python.org/packages/f8/2e/5898046f8089205981447c23ebb8fe02cd9d66939cd74338aa4872853d8e/Cython-0.25.tar.gz#md5=1b61433b8410ac382ac9b248b42466fd";
>> tar -zxf Cython-0.25.tar.gz
>> cd Cython-0.25
>> python ./setup.py build
>> (..)
>> building 'Cython.Runtime.refnanny' extension
>> creating
>> build/temp.linux-x86_64-2.6/home/centos/Cython-0.25/Cython/Runtime
>> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall
>> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
>> --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC
>> -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
>> -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic
>> -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c
>> /home/centos/Cython-0.25/Cython/Runtime/refnanny.c -o
>>
>> build/temp.linux-x86_64-2.6/home/centos/Cython-0.25/Cython/Runtime/refnanny.o
>> gcc: /home/centos/Cython-0.25/Cython/Runtime/refnanny.c: No such file or
>> directory
>> gcc: no input files
>> error: command 'gcc' failed with exit status 1
>>
>> If I just run:
>> cythonize /home/centos/Cython-0.25/Cython/Runtime/refnanny.pyx
>> Then the build succeeds.
>>
>> Cheers
>> Antoine
>>
>>
>> >
>> > - Robert
>> >
>> >
>> > Features added
>> > --
>> >
>> > * def/cpdef methods of cdef classes benefit from Cython's internal
>> > function
>> >   implementation, which enables introspection and line profiling for
>> > them.
>> >   Implementation sponsored by Turbostream (www.turbostream-cfd.com).
>> >
>> > * The distutils extension ``Cython.Distutils.build_ext`` has now been
>> > updated
>> >   to use cythonize which properly handles dependencies.  The old
>> > extension can
>> >   still be found in ``Cython.Distutils.old_build_ext`` and is now
>> > deprecated.
>> >
>> > * Calls to Python functions are faster, following the recent "FastCall"
>> >   optimisations that Victor Stinner implemented for CPython 3.6.
>> >   See https://bugs.python.org/issue27128 and related issues.
>> >
>> > * The new METH_FASTCALL calling convention for PyCFunctions is supported
>> >   in CPython 3.6.  See https://bugs.python.org/issue27810
>> >
>> > * C++ classes can now have typedef members. STL containers updated with
>> >   value_type.
>> >
>> > * Support for bazel using a the pyx_library rule in //Tools:rules.bzl.
>> >
>> > * Initial support for using Cython modules in Pyston.  Patch by
>> > Daetalus.
>> >
>> > * Dynamic Python attributes are allowed on cdef classes if an attribute
>> >   ``cdef dict __dict__`` is declared in the class.  Patch by empyrical.
>> >
>> > * Cython implemented C++ classes can make direct calls to base class
>> > methods.
>> >   Patch by empyrical.
>> >
>> > * New directive ``cython.no_gc`` to fully disable GC for a cdef class.
>> >   Patch by Clau

Re: [Cython] Cython 0.25 released

[Robert Bradshaw]

I'm having trouble reproducing this on any of my setups though. Let's
follow up on https://github.com/cython/cython/issues/1499

On Wed, Oct 26, 2016 at 2:11 AM, Antoine Martin 
wrote:

> On 26/10/16 03:34, Robert Bradshaw wrote:
> > I'm happy to announce the release of Cython 0.25 which has numerous
> > new features and bug fixes. It is available at
> > https://pypi.python.org/pypi/Cython
> >
> > We have also moved bug tracking from trac.cython.org to github issues.
> > However, user support still remains at cython-users@, please refrain
> > from filing issues until you have confirmed an actual bug.
>
> 0.25 does not build on any CentOS 6.x:
> wget
> "https://pypi.python.org/packages/f8/2e/5898046f8089205981447c23ebb8fe
> 02cd9d66939cd74338aa4872853d8e/Cython-0.25.tar.gz#md5=
> 1b61433b8410ac382ac9b248b42466fd"
> tar -zxf Cython-0.25.tar.gz
> cd Cython-0.25
> python ./setup.py build
> (..)
> building 'Cython.Runtime.refnanny' extension
> creating build/temp.linux-x86_64-2.6/home/centos/Cython-0.25/
> Cython/Runtime
> gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC
> -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
> -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic
> -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c
> /home/centos/Cython-0.25/Cython/Runtime/refnanny.c -o
> build/temp.linux-x86_64-2.6/home/centos/Cython-0.25/
> Cython/Runtime/refnanny.o
> gcc: /home/centos/Cython-0.25/Cython/Runtime/refnanny.c: No such file or
> directory
> gcc: no input files
> error: command 'gcc' failed with exit status 1
>
> If I just run:
> cythonize /home/centos/Cython-0.25/Cython/Runtime/refnanny.pyx
> Then the build succeeds.
>
> Cheers
> Antoine
>
>
> >
> > - Robert
> >
> >
> > Features added
> > --
> >
> > * def/cpdef methods of cdef classes benefit from Cython's internal
> function
> >   implementation, which enables introspection and line profiling for
> them.
> >   Implementation sponsored by Turbostream (www.turbostream-cfd.com).
> >
> > * The distutils extension ``Cython.Distutils.build_ext`` has now been
> updated
> >   to use cythonize which properly handles dependencies.  The old
> extension can
> >   still be found in ``Cython.Distutils.old_build_ext`` and is now
> deprecated.
> >
> > * Calls to Python functions are faster, following the recent "FastCall"
> >   optimisations that Victor Stinner implemented for CPython 3.6.
> >   See https://bugs.python.org/issue27128 and related issues.
> >
> > * The new METH_FASTCALL calling convention for PyCFunctions is supported
> >   in CPython 3.6.  See https://bugs.python.org/issue27810
> >
> > * C++ classes can now have typedef members. STL containers updated with
> >   value_type.
> >
> > * Support for bazel using a the pyx_library rule in //Tools:rules.bzl.
> >
> > * Initial support for using Cython modules in Pyston.  Patch by Daetalus.
> >
> > * Dynamic Python attributes are allowed on cdef classes if an attribute
> >   ``cdef dict __dict__`` is declared in the class.  Patch by empyrical.
> >
> > * Cython implemented C++ classes can make direct calls to base class
> methods.
> >   Patch by empyrical.
> >
> > * New directive ``cython.no_gc`` to fully disable GC for a cdef class.
> >   Patch by Claudio Freire.
> >
> > * Buffer variables are no longer excluded from ``locals()``.
> >   Patch by da-woods.
> >
> > * Building f-strings is faster, especially when formatting C integers.
> >
> > * for-loop iteration over "std::string".
> >
> > * ``libc/math.pxd`` provides ``e`` and ``pi`` as alias constants to
> simplify
> >   usage as a drop-in replacement for Python's math module.
> >
> > * Speed up cython.inline().
> >
> > * Binary lshift operations with small constant Python integers are
> faster.
> >
> > * Some integer operations on Python long objects are faster in Python
> 2.7.
> >
> > * Support for the C++ ``typeid`` operator.
> >
> > Significant Bugs fixed
> > --
> >
> > * Division of complex numbers avoids overflow by using Smith's method.
> >
> > * Some function signatures in ``libc.math`` and ``numpy.pxd`` were
> incorrect.
> >   Patch by Michael Seifert.
> >
> > Other changes
> > -
> >

Re: [Cython] Cython 0.25 released

[Robert Bradshaw]

Thanks for the bug report. That's why we have alphas and betas.

On Tue, Oct 25, 2016 at 8:56 PM, Yury Selivanov 
wrote:

> Congrats on the release!
>
> It seems though that it's slightly broken: https://github.com/cython/cyth
> on/issues/1496.
>
>
> Yury
>
>
> On 2016-10-25 4:34 PM, Robert Bradshaw wrote:
>
>> I'm happy to announce the release of Cython 0.25 which has numerous
>> new features and bug fixes. It is available at
>> https://pypi.python.org/pypi/Cython
>>
>> We have also moved bug tracking from trac.cython.org to github issues.
>> However, user support still remains at cython-users@, please refrain
>> from filing issues until you have confirmed an actual bug.
>>
>> - Robert
>>
>>
>> Features added
>> --
>>
>> * def/cpdef methods of cdef classes benefit from Cython's internal
>> function
>>implementation, which enables introspection and line profiling for
>> them.
>>Implementation sponsored by Turbostream (www.turbostream-cfd.com).
>>
>> * The distutils extension ``Cython.Distutils.build_ext`` has now been
>> updated
>>to use cythonize which properly handles dependencies.  The old
>> extension can
>>still be found in ``Cython.Distutils.old_build_ext`` and is now
>> deprecated.
>>
>> * Calls to Python functions are faster, following the recent "FastCall"
>>optimisations that Victor Stinner implemented for CPython 3.6.
>>See https://bugs.python.org/issue27128 and related issues.
>>
>> * The new METH_FASTCALL calling convention for PyCFunctions is supported
>>in CPython 3.6.  See https://bugs.python.org/issue27810
>>
>> * C++ classes can now have typedef members. STL containers updated with
>>value_type.
>>
>> * Support for bazel using a the pyx_library rule in //Tools:rules.bzl.
>>
>> * Initial support for using Cython modules in Pyston.  Patch by Daetalus.
>>
>> * Dynamic Python attributes are allowed on cdef classes if an attribute
>>``cdef dict __dict__`` is declared in the class.  Patch by empyrical.
>>
>> * Cython implemented C++ classes can make direct calls to base class
>> methods.
>>Patch by empyrical.
>>
>> * New directive ``cython.no_gc`` to fully disable GC for a cdef class.
>>Patch by Claudio Freire.
>>
>> * Buffer variables are no longer excluded from ``locals()``.
>>Patch by da-woods.
>>
>> * Building f-strings is faster, especially when formatting C integers.
>>
>> * for-loop iteration over "std::string".
>>
>> * ``libc/math.pxd`` provides ``e`` and ``pi`` as alias constants to
>> simplify
>>usage as a drop-in replacement for Python's math module.
>>
>> * Speed up cython.inline().
>>
>> * Binary lshift operations with small constant Python integers are faster.
>>
>> * Some integer operations on Python long objects are faster in Python 2.7.
>>
>> * Support for the C++ ``typeid`` operator.
>>
>> Significant Bugs fixed
>> --
>>
>> * Division of complex numbers avoids overflow by using Smith's method.
>>
>> * Some function signatures in ``libc.math`` and ``numpy.pxd`` were
>> incorrect.
>>Patch by Michael Seifert.
>>
>> Other changes
>> -
>>
>> * The "%%cython" IPython/jupyter magic now defaults to the language level
>> of
>>the current jupyter kernel.  The language level can be set explicitly
>> with
>>"%%cython -2" or "%%cython -3".
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Cython 0.25 released

[Robert Bradshaw]

I'm happy to announce the release of Cython 0.25 which has numerous
new features and bug fixes. It is available at
https://pypi.python.org/pypi/Cython

We have also moved bug tracking from trac.cython.org to github issues.
However, user support still remains at cython-users@, please refrain
from filing issues until you have confirmed an actual bug.

- Robert


Features added
--

* def/cpdef methods of cdef classes benefit from Cython's internal function
  implementation, which enables introspection and line profiling for them.
  Implementation sponsored by Turbostream (www.turbostream-cfd.com).

* The distutils extension ``Cython.Distutils.build_ext`` has now been updated
  to use cythonize which properly handles dependencies.  The old extension can
  still be found in ``Cython.Distutils.old_build_ext`` and is now deprecated.

* Calls to Python functions are faster, following the recent "FastCall"
  optimisations that Victor Stinner implemented for CPython 3.6.
  See https://bugs.python.org/issue27128 and related issues.

* The new METH_FASTCALL calling convention for PyCFunctions is supported
  in CPython 3.6.  See https://bugs.python.org/issue27810

* C++ classes can now have typedef members. STL containers updated with
  value_type.

* Support for bazel using a the pyx_library rule in //Tools:rules.bzl.

* Initial support for using Cython modules in Pyston.  Patch by Daetalus.

* Dynamic Python attributes are allowed on cdef classes if an attribute
  ``cdef dict __dict__`` is declared in the class.  Patch by empyrical.

* Cython implemented C++ classes can make direct calls to base class methods.
  Patch by empyrical.

* New directive ``cython.no_gc`` to fully disable GC for a cdef class.
  Patch by Claudio Freire.

* Buffer variables are no longer excluded from ``locals()``.
  Patch by da-woods.

* Building f-strings is faster, especially when formatting C integers.

* for-loop iteration over "std::string".

* ``libc/math.pxd`` provides ``e`` and ``pi`` as alias constants to simplify
  usage as a drop-in replacement for Python's math module.

* Speed up cython.inline().

* Binary lshift operations with small constant Python integers are faster.

* Some integer operations on Python long objects are faster in Python 2.7.

* Support for the C++ ``typeid`` operator.

Significant Bugs fixed
--

* Division of complex numbers avoids overflow by using Smith's method.

* Some function signatures in ``libc.math`` and ``numpy.pxd`` were incorrect.
  Patch by Michael Seifert.

Other changes
-

* The "%%cython" IPython/jupyter magic now defaults to the language level of
  the current jupyter kernel.  The language level can be set explicitly with
  "%%cython -2" or "%%cython -3".
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.25 beta

[Robert Bradshaw]

One (hopefully last) beta: https://github.com/cython/cython/archive/0.25b2.zip

On Tue, Oct 18, 2016 at 11:16 AM, Yury Selivanov
 wrote:
> Please also take a look at https://github.com/cython/cython/issues/1493
>
>
>
> On 2016-10-10 3:50 PM, Robert Bradshaw wrote:
>>
>> Thanks for all of those to tested the alpha release. I think we've managed
>> to fix all the issues and regressions raised in that thread; here's a beta
>> that's likely to turn into a release candidate.
>>
>> https://github.com/cython/cython/archive/0.25b0.zip
>>
>>
>>
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Regression since 0.25a0 involving std::vector

[Robert Bradshaw]

Thanks for checking. This was fixed at https://github.com/cython/
cython/commit/4bf8b9b260c33fe88de5d743d5accbcbf649d8a6 Try
https://github.com/cython/cython/archive/0.25.x.zip


On Tue, Oct 18, 2016 at 1:17 AM, Jeroen Demeyer 
wrote:

> This is a new regression introduced since 0.25a0, found while testing Sage
> with 0.25b1:
>
>
>
> from libcpp.vector cimport vector
> cdef extern from *:
> cdef cppclass BoostGraph[T]:
> vector[unsigned long] bandwidth_ordering()
> ctypedef BoostGraph[int] BoostVecDiGraph
>
>
>
> Error compiling Cython file:
> 
> ...
> from libcpp.vector cimport vector
> cdef extern from *:
> cdef cppclass BoostGraph[T]:
> vector[unsigned long] bandwidth_ordering()
> ctypedef BoostGraph[int] BoostVecDiGraph
>^
> 
>
> /tmp/boost_graph.pyx:5:20: Compiler crash in AnalyseDeclarationsTransform
>
> File 'ModuleNode.py', line 118, in analyse_declarations:
> ModuleNode(boost_graph.pyx:1:0,
> full_module_name = 'boost_graph')
> File 'Nodes.py', line 425, in analyse_declarations:
> StatListNode(boost_graph.pyx:1:0)
> File 'Nodes.py', line 1567, in analyse_declarations:
> CTypeDefNode(boost_graph.pyx:5:0,
> visibility = u'private')
> File 'Nodes.py', line 1123, in analyse: TemplatedTypeNode(boost_graph.
> pyx:5:20)
>
> Compiler crash traceback from this point on:
>   File 
> "/usr/local/src/sage-config/local/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 1123, in analyse
> self.type = base_type.specialize_here(self.pos, template_types)
>   File "/usr/local/src/sage-config/local/lib/python2.7/site-package
> s/Cython/Compiler/PyrexTypes.py", line 3462, in specialize_here
> return self.specialize(dict(zip(self.templates, template_values)))
>   File "/usr/local/src/sage-config/local/lib/python2.7/site-package
> s/Cython/Compiler/PyrexTypes.py", line 3480, in specialize
> specialized.scope = self.scope.specialize(values, specialized)
>   File 
> "/usr/local/src/sage-config/local/lib/python2.7/site-packages/Cython/Compiler/Symtab.py",
> line 2284, in specialize
> e.type.specialize(values),
>   File "/usr/local/src/sage-config/local/lib/python2.7/site-package
> s/Cython/Compiler/PyrexTypes.py", line 2746, in specialize
> result = CFuncType(self.return_type.specialize(values),
>   File "/usr/local/src/sage-config/local/lib/python2.7/site-package
> s/Cython/Compiler/PyrexTypes.py", line 3489, in specialize
> T = values[self.templates[0]]
> KeyError: 
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.25 beta

[Robert Bradshaw]

One more beta: https://github.com/cython/cython/archive/0.25b1.tar.gz I'm
planning to release before the week is out.

On Mon, Oct 10, 2016 at 12:50 PM, Robert Bradshaw 
wrote:

> Thanks for all of those to tested the alpha release. I think we've managed
> to fix all the issues and regressions raised in that thread; here's a beta
> that's likely to turn into a release candidate.
>
> https://github.com/cython/cython/archive/0.25b0.zip
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.25 beta

[Robert Bradshaw]

We did not manage to fix all bugs from all previous releases with this
release.

On Tue, Oct 11, 2016 at 9:16 AM, Andrew Svetlov 
wrote:

> https://github.com/cython/cython/issues/1461 is still an issue.
>
> On Monday, October 10, 2016 at 10:51:37 PM UTC+3, Robert Bradshaw wrote:
>>
>> Thanks for all of those to tested the alpha release. I think we've
>> managed to fix all the issues and regressions raised in that thread; here's
>> a beta that's likely to turn into a release candidate.
>>
>> https://github.com/cython/cython/archive/0.25b0.zip
>>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Cython 0.25 beta

[Robert Bradshaw]

Thanks for all of those to tested the alpha release. I think we've managed
to fix all the issues and regressions raised in that thread; here's a beta
that's likely to turn into a release candidate.

https://github.com/cython/cython/archive/0.25b0.zip
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] [cython-users] Re: Cython 0.25 alpha0

[Robert Bradshaw]

Thanks for the report; not sure what changed here. Could you file an
issue on github against the 0.25 milestone?

On Sun, Sep 25, 2016 at 12:02 AM, Isuru Fernando  wrote:
> Hi,
>
> With Cython 0.25a0, following causes a compiler crash.
>
> from libcpp.vector cimport vector
> cdef vector[double complex] v
>
>
> Regards,
>
> Isuru Fernando
>
> On Sun, Sep 25, 2016 at 11:35 AM, Matthew Brett 
> wrote:
>>
>> On Sat, Sep 24, 2016 at 10:12 PM, Robert Bradshaw 
>> wrote:
>> > This prerelease version is also available at PyPi
>> >
>> > https://pypi.python.org/pypi/Cython/0.25a0
>> >
>> > installable via
>> >
>> > pip install --pre cython
>>
>> It looks like the new BUILD file is breaking wheel builds on OSX and
>> Windows:
>>
>>
>> https://ci.appveyor.com/project/matthew-brett/cython-wheels/build/1.0.20/job/eu6n2eyc07a33ojl
>> https://travis-ci.org/MacPython/cython-wheels/jobs/162527865#L235
>>
>> Can it be renamed?
>>
>> Cheers,
>>
>> Matthew
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython 0.25 alpha0

[Robert Bradshaw]

This prerelease version is also available at PyPi

https://pypi.python.org/pypi/Cython/0.25a0

installable via

pip install --pre cython


On Sat, Sep 24, 2016 at 9:47 PM, Robert Bradshaw  wrote:
> Please download and test the upcoming release of Cython:
>
> https://github.com/cython/cython/archive/0.25a0.tar.gz
>
> This is not the final release (more pull requests and bug fixes may
> still go in) but a lot has changed since the last release so it'd be
> good to get feedback early.
>
> https://github.com/cython/cython/blob/master/CHANGES.rst
>
> Thanks,
> Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Cython 0.25 alpha0

[Robert Bradshaw]

Please download and test the upcoming release of Cython:

https://github.com/cython/cython/archive/0.25a0.tar.gz

This is not the final release (more pull requests and bug fixes may
still go in) but a lot has changed since the last release so it'd be
good to get feedback early.

https://github.com/cython/cython/blob/master/CHANGES.rst

Thanks,
Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython release

[Robert Bradshaw]

Thanks!

On Tue, Sep 13, 2016 at 11:21 PM, Jeroen Demeyer  wrote:
> On 2016-09-09 10:11, Robert Bradshaw wrote:
>>
>> I'd like to put out another Cython release shortly. Is anyone aware of
>> anything that we should try to get in before then?
>
>
> I just built the SageMath distribution with latest Cython master and there
> were 2 issues (both easily fixed):
>
> - Build issue with pyzmq:
> https://github.com/zeromq/pyzmq/pull/909
>
> - Replaced assignment of Cython.Compiler.Options.directive_defaults by
> passing directives to cythonize()
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython release

[Robert Bradshaw]

On Fri, Sep 9, 2016 at 4:33 AM, Stefan Behnel  wrote:
> Robert Bradshaw schrieb am 09.09.2016 um 10:11:
>> I'd like to put out another Cython release shortly.
>
> Exactly my thoughts.
>
> The next release could use some extra good testing as I reworked some of
> the fused-/cy-function internals last month and might not have found all
> kinds of code that could break along the way. There are so many feature
> combinations these days: decorators, py+pxd, @classmethod vs.
> classmethod(f), def/cdef/cpdef, ...
>
>
>> Is anyone aware of
>> anything that we should try to get in before then?
>
> I've been working on adapting the PEP 525 (async generators) implementation
> recently that is proposed for CPython, but that's not finished, neither in
> CPython nor in Cython, so don't wait for it.
>
> Happy to get 0.25 out soon.

Any thoughts on https://github.com/cython/cython/pull/1456 ?
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython release

[Robert Bradshaw]

On Fri, Sep 9, 2016 at 12:03 PM, Jeroen Demeyer  wrote:
> On 2016-09-09 10:11, Robert Bradshaw wrote:
>>
>> I'd like to put out another Cython release shortly.
>
>
> Do you mean a 0.24.x release or 0.25?
>
> Anyway, this is a serious bug breaking SageMath:
> https://github.com/cython/cython/issues/1433

Hmm... yes.

https://github.com/cython/cython/commit/c3f2d97778838067d9c6cf96fa45ab5a8c9ebcb0

Not sure what caused this to regress (or how it worked before) but
it's all better now.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Cython release

[Robert Bradshaw]

I'd like to put out another Cython release shortly. Is anyone aware of
anything that we should try to get in before then?

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter

[Robert Bradshaw]

On Mon, Aug 22, 2016 at 9:44 PM, Robert Bradshaw  wrote:

> And it would be a step back
> for the nice auto-conversion facilities you care about, e.g.
>
> py_func(c_func(py_value))
>
> would become
>
> c_value = py_value
> cdef some_type c_ret
> inline_cpp "c_ret = c_func(c_value)"
> py_func(c_ret)
>
> It's just choosing a smaller unit (line vs. file) to swap between languages.

And on this note it would also encourage use of the Python/C API (over
swapping "back" to Python) which is often a dangerous step backwards.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter

[Robert Bradshaw]

On Mon, Aug 22, 2016 at 4:48 PM, Jason Newton  wrote:
>
> Bare in mind that was then, and while I'm not Cython developer level I do
> have a better idea of how Cython works now, possibly better than your
> average user.  I've thought about this problem and the state of the overall
> C++ python binding capability alot as I've had to reevaluate it possibly
> more than most, and I've used them more deeply than playcode.

Good to know.

>> > My point was that multifile multi-level wrapper that I mentioned earlier
>> > -
>> > if you're saying that those projects did Cython extensions wrong, then
>> > I'm
>> > incorrect at faulting Cython and should fault the libraries using it.  I
>> > didn't say staying close to python caused $blurb.
>> >
>> > I don't know in a situation as confusing as to all the binding projects
>> > if
>> > this should be taken as validation of philosophy either - I think it is
>> > reasonable to consider the attrition of these projects as a function of
>> > manpower, number of early on project supporters/authors, and if a
>> > project
>> > (like sage) indirectly, through dependency,  kept the project alive.
>> > And
>> > good old fashioned luck.  I noted most of them don't use distutils and
>> > something custom but less capable instead which maybe plays a roll in
>> > how
>> > mature/usable/smalltime they where/are.
>>
>> Certainly the success of a project depends on many external factors,
>> and even raw luck plays a part. But the approach and philosophy taken
>> to attack a problem and guard its API (and the users/contributors that
>> such decisions attract or repel) can't be discounted either,
>> especially when taken over a long timeframe.
>>
>> But if anyone wants to believe that Cython's become popular because of
>> pure luck despite wrongheaded guiding principles or philosophies,
>> it'll be difficult to persuade them otherwise.
>
> You snidely play with extremes, I never said pure luck was the main factor -
> I listed several factors of very high significance.  Further, Cython
> branched off of pyrex - why do you think that project failed (you might
> actually know)?  Did your philosophy carry separate the two or was it
> because you needed things to work (for sage), had a larger set of smart
> developers and improve it faster than pyrex would allow?  I think it's the
> bustling of life that got Cython where it is, I don't believe it's
> philosophy in this case - I think it was lack of feature competitiveness and
> Cython probably worked better (more bugs/cases fixed).

I wouldn't say that Pyrex failed--it gave birth to Cython. At a high
level where I'd say they diverged is that Pyrex was viewed as a tool
to write C extensions, whereas Cython was viewed as a Python compiler.
The latter meant prioritizing things like being an optimizing
compiler, full coverage of the Python API (e.g. generators),
numpy/buffer integration, and Python 3 support were higher priority,
as opposed to being non-goals for Pyrex.

But the key insight of using Python, or at least Python-inspired,
syntax to declare and invoke C constructs lives on. Which is quite
relevant to this thread.

The fact that Greg didn't seem to have the time/interest in corralling
an open source project factored in here as well. If you have anything
to add, Greg, always happy to have you chime in.

>> >> I agree that any efforts to trying to parsing C++ without building on
>> >> an actual compiler are fraught with danger. That's not the case with
>> >> generating C++ code, which is the direction we're going. In
>> >> particular, our goal is to understand C++ enough to invoke it, which
>> >> allows us to be much less pedantic.
>> >
>> > I understand and agree with the logic in stating it's a less complicated
>> > goal but what comparable success stories exist?  I strongly think
>> > "devils in
>> > the details" in correctly making that work and that they will be tough
>> > solvable problems.  And then you're going and promising on unfamiliar
>> > territory.  But what the ultimate takeaway for me is that you won't have
>> > it
>> > ready in any near term.  Do you have the skills and resources to
>> > implement
>> > this in under 2 years?  And then the other question is are you and the
>> > team
>> > reasonably confident you will have it working and usable by then.
>> > Otherwise
>> > you are not being pragmatic.
>> >
>> > On the other hand, if it was reasonably simple as many of your other
>> > points
>> > in future emails point out, I'd really like to know why you hadn't
>> > addressed
>> > them earlier.
>>
>> Other higher priority items for limited resources. And non-type
>> template args are not necessarily that simple given the way things are
>> structured now.
>
> There's always limited resources and priorities but hen it sounds like it's
> going to be a very (ever)? long time until everything is working great
> unless the need is becoming a clear driver or this email thread has
> reassigned priorities.  Do you think you can ske

Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter

[Robert Bradshaw]

On Mon, Aug 22, 2016 at 7:44 AM, Kevin Thornton  wrote:
> I actually just realized another nagging issue with C++/Cython: member
> typedefs are not supported:
>
> cdef vector[T]:
>   ctypedef size_t size_type
>
> These "scoped" typedefs are very widely-used.  We can live without it, and
> we have been, but while we're making a list.. :)

Yes, that's a good one. Worth an issue :).

> On Sat, Aug 20, 2016 at 11:15 PM Robert Bradshaw  wrote:
>>
>> On Fri, Aug 19, 2016 at 9:34 AM, Kevin Thornton  wrote:
>> > Hi Robert,
>> >
>> > Related to this point, I think there are three important features that
>> > Cython would benefit from, as far as C++11 and newer are concerned.  In
>> > order of what I'm guessing to be increased complexity are:
>> >
>> > 1. Non-type template parameters.
>>
>> This is actually probably the hardest :). There is a PR that's stalled
>> out, probably needs some attention by both the author and us; it'd be
>> great to get this in.
>>
>> > 2. "rvalue" references for standalone functions and class member
>> > functions:
>> >
>> > void foo[T]( T && t )
>>
>> From a callers perspective, one can just declare an extern foo as
>> foo(T) in Cython, it doesn't care if the actual call is made by value
>> or by reference--the calling code is exactly the same.
>>
>> Actually implementing it for Cython-defined functions (as the callee)
>> would require additional support, but would probably be pretty easy
>> (implement a new type, with parsing, which generally behaves the same
>> as a reference or value from Python's perspective--the actual heavy
>> lifting happens in the C compiler).
>>
>> Want to file an issue on github for this?
>>
>> > 3. Variadic templates (member and non-member).  More generally,
>> > "parameter
>> > packs":
>> >
>> > http://en.cppreference.com/w/cpp/language/parameter_pack
>>
>> This'd be interesting (especially as to how it relates to changing the
>> arity of functions for concrete instantiations). There's also a
>> question of syntax. Maybe
>>
>> void foo[A, int N, ... Types](A a, Types t):
>> ...
>>
>> It'd likely require #1, e.g. to be able to declare things like
>> tuple_element::type such that std::tuple.get would be correctly
>> typed. Unfortunately seems declaring the (somewhat intuitive) API of
>> std::tuple requires being able to declare a fair amount of its
>> implementation.
>>
>> Still likely feasible though, and I think the notion of variadic
>> templates is natural enough in a Python setting (which has had
>> heterogeneously typed tuples since its inception, though of course not
>> statically typed).
>>
>> > #2 is important for efficiency reasons--it isn't just a convenience
>> > feature.
>> > Without it, Cython cannot support all the member function of
>> > std::vector,
>> > etc.  The implication is that some expensive operations, like moving a
>> > big
>> > object into a vector, require extra copies in Cython while a C++ program
>> > would just move the object into the new location.
>>
>> It's interesting to call this an essential feature, given that C++
>> itself didn't have it until recently :). Not that it can't be a big
>> win, and libraries will more and more probably be designed assuming
>> it.
>>
>> > #3 seems like the hardest, but would enable std::tuple, std::bind, and
>> > other
>> > handy things to be used in a .pyx file.
>> >
>> > Support for "auto" would be nice, but perhaps unrealistic.
>>
>> Just leave the cdef out and you basically get auto, i.e.
>>
>> my_var = expr
>>
>> gives my_var the type of expr. If multiple assignments are made to
>> my_var, it attempts to find a suitable union type that can hold all
>> assignments. The one exception is for integers, where it infers the
>> type to be object iff my_var is used in arithmetic expressions (that
>> it can't deduce would not overflow with the finite-sized C integer
>> type).
>>
>> > I know that PRs have been submitted, and since stalled out, on a couple
>> > of
>> > these features.  I've tried to look at the Cython parser myself, but I'm
>> > not
>> > able to follow it, sadly.
>> >
>> > Thanks for all your hard work,
>> >
>> > K

Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter

[Robert Bradshaw]

On Sun, Aug 21, 2016 at 7:26 AM, Jason Newton  wrote:
>
> On Sun, Aug 21, 2016 at 5:30 AM, Robert Bradshaw  wrote:
>>
>> In my experience Cython has generally been fairly easy to pick up for
>> people who already know Python. And Python often easy to pick up for
>> people who already know C/C++. Of course for many wrappings it often
>> takes non-trivial knowledge of the wrapped library itself too, but
>> typically at the same level as would be required to grok code written
>> against that same library directly from C/C++.
>
> Is your experience drawn from binding moderately complex libraries or play
> code (of complexity like from a tutorial).  Bottom up or top down?  Sorry if
> this sounds asinine to you.

A whole variety: from simple string manipulation libraries to complex
ML and big data processing frameworks. And second hand with complete
amateurs and students to senior software engineers.

> To clarify, I'm coming from the case where I
> didn't read the whole tutorial/docs before being faced with pyx in the
> projects I previously mentioned, while on tight turn around time - I was not
> able to grok in that context.

Understood--I am short on time too. Though your self-admitted lack of
familiarity doesn't bolster your argument that Cython's doing it
wrong, we need this new feature.

> For myself and any other ML user who comes across this thread - can you list
> a few libraries that do things the right way?
>
>> Yes, there's bad code out there in any language (no offense meant
>> towards h5py--I haven't looked at that project myself). Much of it due
>> to cargo-cult perpetuations or archaic (or simply flat-out-wrong)
>> contortions due to historical limitations (e.g. creating a Python
>> module to wrap an _extension module, avoiding all C++ features with
>> extensive C wrappers, ...). (You're familiar with C++, so likely no
>> stranger to this effect.)
>>
>> > These projects complied with Cython's current philosophy
>> > to the degradation of clarity, context, and overall idea of how code was
>> > hooked up.  Perhaps Cython should take the lessons learned from it's
>> > inception, time, and the results of the state of the c-python userbase
>> > to
>> > guide us into a new philosophy.
>>
>> I fail to see how "staying close to Python" caused "degradation of
>> clarity, context, etc." If anything, the lessons learned over time
>> have validated this philosophy. More on this later.
>
>
> My point was that multifile multi-level wrapper that I mentioned earlier -
> if you're saying that those projects did Cython extensions wrong, then I'm
> incorrect at faulting Cython and should fault the libraries using it.  I
> didn't say staying close to python caused $blurb.
>
> I don't know in a situation as confusing as to all the binding projects if
> this should be taken as validation of philosophy either - I think it is
> reasonable to consider the attrition of these projects as a function of
> manpower, number of early on project supporters/authors, and if a project
> (like sage) indirectly, through dependency,  kept the project alive.  And
> good old fashioned luck.  I noted most of them don't use distutils and
> something custom but less capable instead which maybe plays a roll in how
> mature/usable/smalltime they where/are.

Certainly the success of a project depends on many external factors,
and even raw luck plays a part. But the approach and philosophy taken
to attack a problem and guard its API (and the users/contributors that
such decisions attract or repel) can't be discounted either,
especially when taken over a long timeframe.

But if anyone wants to believe that Cython's become popular because of
pure luck despite wrongheaded guiding principles or philosophies,
it'll be difficult to persuade them otherwise.

>> I agree that any efforts to trying to parsing C++ without building on
>> an actual compiler are fraught with danger. That's not the case with
>> generating C++ code, which is the direction we're going. In
>> particular, our goal is to understand C++ enough to invoke it, which
>> allows us to be much less pedantic.
>
> I understand and agree with the logic in stating it's a less complicated
> goal but what comparable success stories exist?  I strongly think "devils in
> the details" in correctly making that work and that they will be tough
> solvable problems.  And then you're going and promising on unfamiliar
> territory.  But what the ultimate takeaway for me is that you won't have it
> ready in any near term.  Do you have the skills and resources to imp

Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter

[Robert Bradshaw]

On Fri, Aug 19, 2016 at 11:19 AM, Jason Newton  wrote:
>
> On Fri, Aug 19, 2016 at 5:36 AM, Robert Bradshaw  wrote:
>>
>> On Thu, Aug 18, 2016 at 12:05 PM, Jason Newton  wrote:
>> > Accidentally posted to an already-opened tab for the cython-users ML
>> > yesterday, moving to here. Following up from a github opened issue here:
>> >
>> > https://github.com/cython/cython/issues/1440
>> >
>> > I was hoping we could give a way to drop straight into C/C++ inside of
>> > Cython pyx files.
>> >
>> > Why?
>> >
>> > -It [helps] avoid needing to declare every class/function in cython, a
>> > somewhat daunting/impossible task that I think everyone hates.  Have you
>> > libraries like Eigen or others that use complex template based
>> > techniques?
>> > How about those with tons of [member] functions to boot or getting C++
>> > inheritance involved.
>>
>> I agree that this is a pain, and better tooling should be developed to
>> (mostly?) eliminate this (e.g. see the recent thread at
>> https://groups.google.com/forum/#!topic/cython-users/c8ChI6jERzY ).
>>
>> Of course having symbols magically appear (e.g. due some #Include, who
>> knows which) has its downsides too, which is why import * is often
>> discouraged in Python too.
>>
>> > -It works around having the Cython compiler know about all of C++'s
>> > nuances
>> > - as an advanced C++ developer these are painful and it is a 2nd class
>> > citizen to Cython's simpler C-support - that's no good.  Just right now
>> > I
>> > was bitten by yet another template argument bug and it's clear C++
>> > template
>> > arguments have been kind of dicy since support appeared.
>>
>> Yes, C++ is an extraordinarily complicated language, and exposing all
>> of that would add significant amounts of complexity to Cython itself,
>> and perhaps more importantly increase the barrier of entry to reading
>> Cython code. One of the guiding principles is that we try to stay as
>> close to Python as possible (if you know Python, you can probably read
>> Cython, and with a minimal amount of C knowledge start writing it) and
>> much of C++ simply isn't Pythonic.
>
> Maybe it's off topic but I debate the guiding principle of Cython -  I was
> not able to comprehend Cython before reading tutorials and this is not my
> first time looking at it, I had a couple runins over the last 5 years with
> it on projects such as h5py easily got lost on what was going on between all
> the wrapper-wrappers (pure py code wrapping py-invokable code) and
> re-declarations.

In my experience Cython has generally been fairly easy to pick up for
people who already know Python. And Python often easy to pick up for
people who already know C/C++. Of course for many wrappings it often
takes non-trivial knowledge of the wrapped library itself too, but
typically at the same level as would be required to grok code written
against that same library directly from C/C++.

Yes, there's bad code out there in any language (no offense meant
towards h5py--I haven't looked at that project myself). Much of it due
to cargo-cult perpetuations or archaic (or simply flat-out-wrong)
contortions due to historical limitations (e.g. creating a Python
module to wrap an _extension module, avoiding all C++ features with
extensive C wrappers, ...). (You're familiar with C++, so likely no
stranger to this effect.)

> These projects complied with Cython's current philosophy
> to the degradation of clarity, context, and overall idea of how code was
> hooked up.  Perhaps Cython should take the lessons learned from it's
> inception, time, and the results of the state of the c-python userbase to
> guide us into a new philosophy.

I fail to see how "staying close to Python" caused "degradation of
clarity, context, etc." If anything, the lessons learned over time
have validated this philosophy. More on this later.

>> Though, as you discovered, there are some basic things like non-type
>> template arguments that we would like to have. If there are other
>> specific C++ constructs that are commonly used but impossible to
>> express in Cython it'd be useful to know.
>
> I haven't had the capability to use Cython sufficiently to learn more of
> them because it currently can't solve my problems.  From prior SWIG et al
> experiences, my outlook is that it is treacherous path to walk and unless
> you brought in llvm/clang into the project for parsing/AST, I'd hold onto
> that outlook.

I agree that any efforts to trying to parsing C++ without building on
an

Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter

[Robert Bradshaw]

On Fri, Aug 19, 2016 at 12:32 PM, Jason Newton  wrote:
>
>
> On Fri, Aug 19, 2016 at 3:19 PM, William Stein  wrote:
>>
>> On Fri, Aug 19, 2016 at 8:19 AM, Jason Newton  wrote:
>> > You must realize that almost any other python driven way to compile
>> > c-code
>> > in the spirit these projects do is deprecated/dead.  Cython has absorbed
>> > all
>> > the reputation and users that didn't go to pure-c/boost.python -
>> > pybind11 is
>> > the new kid on the block there so I'm not including it (I'm of the
>> > opinion
>> > that SWIG users stayed unchanged).  Community belief/QA/designers/google
>> > all
>> > think of Cython first.  Weave has effectively closed up it's doors [...]
>>
>> May I ask why "any other python driven way to compile c-code in the
>> spirit these projects do is deprecated/dead?"I'm curious since
>> when I started Sage (and Cython based on forking Pyrex), it was
>> because none of the other approaches seemed like they would work for
>> the developer base I envisioned growing for Sage.   That was a long
>> time ago, and I'm always pleasantly surprised that Cython has become
>> very popular.  However, I didn't realize the other approaches were
>> deprecated/dead.
>
>
> PyInline's last news update was in 2004 where the author gives a "Hats of[f]
> to PyRex", prior to that only a few blog entries in 2001/2002, I've not come
> across any project using it but maybe that is not sufficient to call it
> deprecated/dead?  Does it work with Python 3?
> http://pyinline.sourceforge.net/
>
> Pyrex no longer has a userbase.  Last ML post was 2014.
>
> Weave only supports python2, got ripped out of Scipy and also directs to
> check out Cython and strongly implies the project is all but
> dead/maintenance mode: https://github.com/scipy/weave
>
> Did I miss any of the python driven ways?

Pybindgen? And of course there's ctypes/cffi. I could have sworn I saw
another project pop up several years ago that was a lot like what
you're suggesting, e.g. a listing of

def foo(a, b):
   [c code using a and b]

I don't recall how the types for a and b were declared/converted, and
I don't think it did much non-trivial stuff yet, but I can't find a
trace of it now (but it's hard to search for laking a name).
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter

[Robert Bradshaw]

On Fri, Aug 19, 2016 at 9:34 AM, Kevin Thornton  wrote:
> Hi Robert,
>
> Related to this point, I think there are three important features that
> Cython would benefit from, as far as C++11 and newer are concerned.  In
> order of what I'm guessing to be increased complexity are:
>
> 1. Non-type template parameters.

This is actually probably the hardest :). There is a PR that's stalled
out, probably needs some attention by both the author and us; it'd be
great to get this in.

> 2. "rvalue" references for standalone functions and class member functions:
>
> void foo[T]( T && t )

>From a callers perspective, one can just declare an extern foo as
foo(T) in Cython, it doesn't care if the actual call is made by value
or by reference--the calling code is exactly the same.

Actually implementing it for Cython-defined functions (as the callee)
would require additional support, but would probably be pretty easy
(implement a new type, with parsing, which generally behaves the same
as a reference or value from Python's perspective--the actual heavy
lifting happens in the C compiler).

Want to file an issue on github for this?

> 3. Variadic templates (member and non-member).  More generally, "parameter
> packs":
>
> http://en.cppreference.com/w/cpp/language/parameter_pack

This'd be interesting (especially as to how it relates to changing the
arity of functions for concrete instantiations). There's also a
question of syntax. Maybe

void foo[A, int N, ... Types](A a, Types t):
...

It'd likely require #1, e.g. to be able to declare things like
tuple_element::type such that std::tuple.get would be correctly
typed. Unfortunately seems declaring the (somewhat intuitive) API of
std::tuple requires being able to declare a fair amount of its
implementation.

Still likely feasible though, and I think the notion of variadic
templates is natural enough in a Python setting (which has had
heterogeneously typed tuples since its inception, though of course not
statically typed).

> #2 is important for efficiency reasons--it isn't just a convenience feature.
> Without it, Cython cannot support all the member function of std::vector,
> etc.  The implication is that some expensive operations, like moving a big
> object into a vector, require extra copies in Cython while a C++ program
> would just move the object into the new location.

It's interesting to call this an essential feature, given that C++
itself didn't have it until recently :). Not that it can't be a big
win, and libraries will more and more probably be designed assuming
it.

> #3 seems like the hardest, but would enable std::tuple, std::bind, and other
> handy things to be used in a .pyx file.
>
> Support for "auto" would be nice, but perhaps unrealistic.

Just leave the cdef out and you basically get auto, i.e.

my_var = expr

gives my_var the type of expr. If multiple assignments are made to
my_var, it attempts to find a suitable union type that can hold all
assignments. The one exception is for integers, where it infers the
type to be object iff my_var is used in arithmetic expressions (that
it can't deduce would not overflow with the finite-sized C integer
type).

> I know that PRs have been submitted, and since stalled out, on a couple of
> these features.  I've tried to look at the Cython parser myself, but I'm not
> able to follow it, sadly.
>
> Thanks for all your hard work,
>
> Kevin
>
> On Fri, Aug 19, 2016 at 2:39 AM Robert Bradshaw  wrote:
>>
>>
>> Though, as you discovered, there are some basic things like non-type
>> template arguments that we would like to have. If there are other
>> specific C++ constructs that are commonly used but impossible to
>> express in Cython it'd be useful to know.
>>
>>
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] RFC: an inline_ function that dumps c/c++ code to the code emitter

[Robert Bradshaw]

On Thu, Aug 18, 2016 at 12:05 PM, Jason Newton  wrote:
> Accidentally posted to an already-opened tab for the cython-users ML
> yesterday, moving to here. Following up from a github opened issue here:
>
> https://github.com/cython/cython/issues/1440
>
> I was hoping we could give a way to drop straight into C/C++ inside of
> Cython pyx files.
>
> Why?
>
> -It [helps] avoid needing to declare every class/function in cython, a
> somewhat daunting/impossible task that I think everyone hates.  Have you
> libraries like Eigen or others that use complex template based techniques?
> How about those with tons of [member] functions to boot or getting C++
> inheritance involved.

I agree that this is a pain, and better tooling should be developed to
(mostly?) eliminate this (e.g. see the recent thread at
https://groups.google.com/forum/#!topic/cython-users/c8ChI6jERzY ).

Of course having symbols magically appear (e.g. due some #Include, who
knows which) has its downsides too, which is why import * is often
discouraged in Python too.

> -It works around having the Cython compiler know about all of C++'s nuances
> - as an advanced C++ developer these are painful and it is a 2nd class
> citizen to Cython's simpler C-support - that's no good.  Just right now I
> was bitten by yet another template argument bug and it's clear C++ template
> arguments have been kind of dicy since support appeared.

Yes, C++ is an extraordinarily complicated language, and exposing all
of that would add significant amounts of complexity to Cython itself,
and perhaps more importantly increase the barrier of entry to reading
Cython code. One of the guiding principles is that we try to stay as
close to Python as possible (if you know Python, you can probably read
Cython, and with a minimal amount of C knowledge start writing it) and
much of C++ simply isn't Pythonic.

Though, as you discovered, there are some basic things like non-type
template arguments that we would like to have. If there are other
specific C++ constructs that are commonly used but impossible to
express in Cython it'd be useful to know.

> -It would allow single source files - I think this is important for runtime
> compiled quasi-JIT/AOC fragments, like OpenCL/PyOpenCL/PyCUDA provide

Not quite following what you're saying here.

> The idea is that Cython glue makes the playing field for extracting data
> easy, but that once it's extracted to a cdef variable for instance, cython
> doesn't need to know what happens.  Maybe in a way sort of like the GCC asm
> extension.  Hopefully simpler variable passing though.

Cython uses "mangled" names (e.g. with a __pyx prefix) to avoid any
possible conflicts. Specifying what/how to mangle could get as ugly as
GCC's asm variable passing. And embedded variable declarations, let
alone control flow statements (especially return, break, ...) could
get really messy. It obscures analysis Cython can do on the code, such
as whether variables are used or what values they may take. Little
code snippets are not always local either, e.g. do they often need to
refer to variables (or headers) referenced elsewhere. And they must
all be mutually compatible.

That's aside from the jarring nature of interleaving Python and C++
code. Would semicolons be required? How would parsers (including IDEs)
handle the C++ snippets? Or would they be placed in opaque
strings/comments (which I'd rather avoid)?

Feels like you want PyInline or weave.inline, but at compile time.

> The alternative to not having this construct is a concrete wall.  I'll have
> to segment up files for C++ for the rest of time until I get function forms
> that Cython can handle. At that point I just say screw it and use boost
> python.  Of course cython does the definitions, data extraction, and
> compilation so much easier than boost.python.  It would be a shame to not
> consider this plight C++ developers have been cornered into and you can't
> say the C++ libraries are broken, it is the strategy Cython is taking that
> cannot work.

Some C++ libraries are not very amenable to being used outside a C++
context. They don't expose an "FFI-friendly" interface and calling
them from other languages is more painful. That doesn't mean they're
broken, just C++ centric.

The strategy that Cython uses is that you use (as close to possible)
Python structures and syntax to write your bindings. This is great for
people who know and understand Python, but for those who would rather
write their code (including bindings) in C++ there's Boost.Python and
others (including writing C extensions by hand).

If your library is full of C++isms, another option is to create a C
wrapper, and expose that to Python (which is what people did before
there was any C++ support). If exposing these as Python functions,
with automatic conversion to/from Python types is the primary value
you're getting from Cython, simply declare these as cpdef functions in
your extern blocks and wrappers will automatically be created. If yo

[Cython] Cython on GitHub

[Robert Bradshaw]

All of of Cython serving infrastructure, save the jenkins buildbot,
has been moved to GitHub. (Documentation is served via
readthedocs.org, and tarballs served from pypi, built from GitHub
head). Each trac ticket has been migrated to a github issue. I have
updated/redirected what links I can.

As for backups, I created
https://github.com/cython/cython-issues/tree/master to store the
non-repository data, which will be periodically updated. Note that the
original trac data (minus session/user info) is there as well.

I will also be locally cloning all the repositories in the cython
organization, and would recommend at least one other person do the
same.

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Migration of tickets from trac to github

[Robert Bradshaw]

On Fri, Jul 29, 2016 at 5:37 AM, Jeroen Demeyer  wrote:
> We should also about a convention about naming testsuite files.
>
> For example, there is
> tests/compile/distutils_libraries_T845.srctree
>
> Should this be changed to
> tests/compile/distutils_libraries_GH1234.srctree
>
> if this becomes GitHub issue #1234?

Sounds like a fine convention to me. I don't think we need to go
renaming all the Txxx ones though.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] long_double_to_float_int failure on s390x (BE) (0.24.1)

[Robert Bradshaw]

Is this a change from 0.24? Do you have any s390x hardware that we
could test on?

On Wed, Aug 10, 2016 at 6:39 AM, Yaroslav Halchenko
 wrote:
> Dear Cython ppl,
>
> Didn't spot any obvious commit/log since 0.24.1 so decided just to report:
> full build log:
> https://buildd.debian.org/status/fetch.php?pkg=cython&arch=s390x&ver=0.24.1-1&stamp=1470118175
>
> 1 test has failed on s390x (only).  Quick resolution would help to
> assure cython's healthy status in debian coming to a freeze point.
>
> ==
> FAIL: long_double_to_float_int (line 195) 
> (int_float_builtins_as_casts_T400.__test__)
> Doctest: int_float_builtins_as_casts_T400.__test__.long_double_to_float_int 
> (line 195)
> --
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
> raise self.failureException(self.format_failure(new.getvalue()))
> AssertionError: Failed doctest test for 
> int_float_builtins_as_casts_T400.__test__.long_double_to_float_int (line 195)
>   File 
> "/«PKGBUILDDIR»/build/work-dir/run/c/int_float_builtins_as_casts_T400/int_float_builtins_as_casts_T400.so",
>  line unknown line number, in long_double_to_float_int (line 195)
>
> --
> File 
> "/«PKGBUILDDIR»/build/work-dir/run/c/int_float_builtins_as_casts_T400/int_float_builtins_as_casts_T400.so",
>  line ?, in 
> int_float_builtins_as_casts_T400.__test__.long_double_to_float_int (line 195)
> Failed example:
> long_double_to_float_int(4.1)
> Expected:
> 4.0
> Got:
> 0.0
> --
> File 
> "/«PKGBUILDDIR»/build/work-dir/run/c/int_float_builtins_as_casts_T400/int_float_builtins_as_casts_T400.so",
>  line ?, in 
> int_float_builtins_as_casts_T400.__test__.long_double_to_float_int (line 195)
> Failed example:
> long_double_to_float_int(-4.1)
> Expected:
> -4.0
> Got:
> 0.0
> --
> File 
> "/«PKGBUILDDIR»/build/work-dir/run/c/int_float_builtins_as_casts_T400/int_float_builtins_as_casts_T400.so",
>  line ?, in 
> int_float_builtins_as_casts_T400.__test__.long_double_to_float_int (line 195)
> Failed example:
> long_double_to_float_int(4)
> Expected:
> 4.0
> Got:
> 0.0
>
>
> --
> Ran 9957 tests in 4237.557s
>
> FAILED (failures=1, skipped=2)
>
>
> --
> Yaroslav O. Halchenko
> Center for Open Neuroscience http://centerforopenneuroscience.org
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython docs are for version 0.15pre

[Robert Bradshaw]

Looks like it was trying to pull from the old docs repo. Thanks for
pointing this out. Fixed.

On Sun, Jul 31, 2016 at 11:38 PM, Jeroen Demeyer  wrote:
> The page
> https://cython.readthedocs.io/en/latest/
> has the documentation for Cython version 0.15pre.
>
> How did that happen???
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Migration of tickets from trac to github

[Robert Bradshaw]

No, this didn't do attachments. It'd be good to get a handle on how
large a loss that would be for Cython (for Sage attachments are key).

I also just discovered
https://gist.github.com/jonmagic/5282384165e0f86ef105 which looks like
it might be a better solution.

On Fri, Jul 29, 2016 at 1:23 AM, Jeroen Demeyer  wrote:
> What about attachments to Trac tickets? This ones mentions a patch, but the
> patch does not appear on GitHub as far as I can tell:
>
> https://github.com/robertwb/issues-import-test/issues/122
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Migration of tickets from trac to github

[Robert Bradshaw]

Yes, several. I was intentionally re-importing the same ranges to see
how things looked.

On Fri, Jul 29, 2016 at 12:49 AM, Jeroen Demeyer  wrote:
> There are some duplicates, like
> https://github.com/robertwb/issues-import-test/issues/130
> and
> https://github.com/robertwb/issues-import-test/issues/137
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Migration of tickets from trac to github

[Robert Bradshaw]

On Fri, Jul 29, 2016 at 12:46 AM, Jeroen Demeyer  wrote:
> On 2016-07-29 06:50, Robert Bradshaw wrote:
>>
>> I'm not quite sure how to serve the mappings
>
> I think it's really important to keep redirects from the old Trac tickets to
> GitHub. There are a lot of links from SageMath to some Cython Trac ticket
> and it would be a shame to lose that.

Yes. I figured out how to do redirects (e.g.
http://trac.cython.org/ticket/32 ). Now to generate the tickets and
get the mappings...

>>> Related to that: converted ticket #73 has a reference to another trac
>>> ticket (#51) in a comment, which now links to the (wrong) github ticket
>>> #51.
>>> https://github.com/robertwb/issues-import-test/issues/73
>>
>>
>> Hmm... I should spell that out.
>
> Just make the link explicit to trac.cython.org...

Yep.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Migration of tickets from trac to github

[Robert Bradshaw]

On Thu, Jul 28, 2016 at 10:15 PM, Yury V. Zaytsev  wrote:
> On Thu, 28 Jul 2016, Robert Bradshaw wrote:
>
>>> When I click on one of the "migrated from" links to "trac.cython.org", I
>>> get redirected back to github, but with the trac ticket number, which means
>>> I get either a 404 or a totally different ticket on github. I guess that can
>>> be fixed once we have a mapping from trac ticket numbers to github tickets.
>>
>>
>> Yep. Had we not had any issues, we could have maybe kept the numbers, but
>> it's too late for that now.
>
>
> Just an idea: one could, of course, disable issues for the main cython
> repository and create a separate repository like `cython-issues`. Not sure
> if it's worth it though...

That's an idea, though then we'd loose things like pull request
linkage and have to migrate the issues that are already there. I think
it's worth having a bounded amount of "odd" history with a "standard"
setup going forward.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Migration of tickets from trac to github

[Robert Bradshaw]

On Thu, Jul 28, 2016 at 9:13 PM, Stefan Behnel  wrote:
> Robert Bradshaw schrieb am 29.07.2016 um 00:33:
>> Please see https://github.com/robertwb/issues-import-test/issues/ as
>> to what this would look like. (This is a partial import, due to github
>> API rate limiting issues.) Unless something looks terribly wrong, I'll
>> start the real migration shortly.
>
> Nice, also the formatting and tagging. And the milestone history also seems
> to have passed through nicely.

Thanks. FYI, I'm using https://github.com/trustmaster/trac2github .
(It's been a looong time since I've messed with php, but the couple of
Python ones were broken or not as feature complete...)

> My comments:
>
> Tags like "C: Code Generation" look like they are referring specifically to
> C-Code, but I guess you meant "C-omponent". Maybe change the abbreviation
> to M-odule or T-opic ?

The C is "component." Probably could simply drop the C altogether.

> Do we really need the empty "modified by" comments, e.g. here:
> https://github.com/robertwb/issues-import-test/issues/110

Hmm... I wonder if this was an ownership change (that was having
trouble exporting).

> This looks a bit overly empty:
> https://github.com/robertwb/issues-import-test/issues/55

Yeah, that was empty before.

> When I click on one of the "migrated from" links to "trac.cython.org", I
> get redirected back to github, but with the trac ticket number, which means
> I get either a 404 or a totally different ticket on github. I guess that
> can be fixed once we have a mapping from trac ticket numbers to github 
> tickets.

Yep. Had we not had any issues, we could have maybe kept the numbers,
but it's too late for that now.

I'm not quite sure how to serve the mappings--I'd rather not run a
service just for that. One idea is a custom 404 page with javascript?
Or I suppose there's a finite number, so we could just a static for
each one in the site in a ticket subdirectory.

> Related to that: converted ticket #73 has a reference to another trac
> ticket (#51) in a comment, which now links to the (wrong) github ticket #51.
> https://github.com/robertwb/issues-import-test/issues/73

Hmm... I should spell that out.

> Also, the formatting is broken in the description of that ticket (#73).
> Might already have been a problem in trac.

The spaces were there, the code markers weren't. Doesn't seem too
common. I think we'll have to fix some one-offs.

> And again #73: there is a link to the old hg repo in a comment. I guess we
> might simply not be able to recover the old hg changeset IDs to provide a
> mapping of those to the corresponding github changeset URLs, right?

Yeah, that's a bit harder. (With an old hg repo, one could probably
recover the mapping using temestamps and/or description+author...)
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

[Cython] Migration of tickets from trac to github

[Robert Bradshaw]

Please see https://github.com/robertwb/issues-import-test/issues/ as
to what this would look like. (This is a partial import, due to github
API rate limiting issues.) Unless something looks terribly wrong, I'll
start the real migration shortly.

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython infrastructure

[Robert Bradshaw]

On Thu, Jul 21, 2016 at 4:29 AM, Dima Pasechnik
 wrote:
> On Thu, Jul 21, 2016 at 10:18 AM, Baptiste Carvello
>  wrote:
>> Le 20/07/2016 19:23, Robert Bradshaw a écrit :
>>
>>> +1
>>>
>>> I'm a big advocate of privacy, and informed consent when choosing to
>>> give any of it away (e.g. allowing linking of activities to build a
>>> (pseudonymous or not) reputation).
>>
>> (philosophical side note: "consent" is not free of coercion, and thus
>> rather irrelevant, when Github is taking over 90% of Open Source projects.)
>>
>>> [...] Personally, I'm actually quite
>>> happy to have my activities on github correlated with my identity.
>>> (Actually, it's a net plus, not a concession.)
>>
>> I understand your point, but I'd like to make a different choice.
>>
>>> Of course you can always set up any number of unrelated pseudonyms on
>>> github, delete cookies, use incognito mode, and even do everything via
>>> tor if you really want.
>>
>> No, I can't (unless I want to play cat and mouse with them, which is no
>> fun). And that is the whole of the problem, as I say in my other message.
>
> Actually, it's not clear why this is a problem. If you do not want to
> play cat and
> mouse (which means removing cookies often, etc), you would create an
> identity that
> only you know, and let this avatar do all the talking and working on github.
> There are plenty of github users out there like this, nobody sees who
> they really are.
> (github knows a working email address for them, that's basically all).
>
>>
>>> However, while "Subscribe to Github" is a perfectly reasonable answer,
>>> and one that would in practice include more people than it would
>>> exclude (compared to our current system, or many alternatives), it's
>>> not like we're going to suddenly refuse all discussions of bugs on the
>>> mailing lists. We're low enough volume to be flexible. A real bug
>>> tracker is simply more useful for tracking issues than an inbox.
>>
>> As long as the mailing list stays, any concrete difficulty can be solved
>> when it arises through a constructive discussion, so nothing is lost!
>>
>> I trust that Cython won't ever do like some other projects, which have
>> suppressed any kind of non-Github contact channel. That would be the
>> real pain.
>
> IMHO this is just spreading FUD.
> Many, many projects with presence on github have, say, google groups
> or/and non-github based bug/issue trackers as primary means of
> communication. Noone ever heard of github undermining these projects
> in any way.

I think he's concerned about projects that decide (deliberately or
not) to discontinue (or ignore) their mailing lists, only
communicating via github (or some other closed system). Or if one had
dozens of bug reports coming in a week, I could understand asking
users to always file reports in the tracker themselves, and keep
discussion there (where it's more easily filtered and keeps the main
dev list a reasonable volume). Yes, this would require a github
account, which is too much for some people.

We're keeping our mailing list(s). We require at least email address
and an online presence (no snail mail) to submit and discuss bugs.
That is too much for some people too.

It sounds like there's no objections to moving the site, everyone can
live with moving the bug tracker, and we're still up in the air on
what to do with the build farm. I'll get on the first two.

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython infrastructure

[Robert Bradshaw]

On Thu, Jul 21, 2016 at 2:44 AM, Isuru Fernando  wrote:
> FYI, travis does support CPython dev versions.
> https://docs.travis-ci.com/user/languages/python#Choosing-Python-versions-to-test-against
>
> Travis supports caching of build dependencies. (I remember it was not free
> on OS X, but it might be now). https://docs.travis-ci.com/user/caching/
>
> Sage is a bit tricky. If there are binaries of Sage develop branch built for
> Ubuntu 12.04 or 14.04 built by Sagebots hosted somewhere, then you can use
> it on Travis. I use a Sage release binary to test a project and haven't had
> any issues with it.

Interesting.

In this case we actually have to (re?)build Sage to test Cython, which
is quite expensive, and testing Sage is even more expensive, so
release binaries might not be quite enough.

> Isuru Fernando
>
> On Thu, Jul 21, 2016 at 2:48 PM, Baptiste Carvello
>  wrote:
>>
>> Le 20/07/2016 19:23, Robert Bradshaw a écrit :
>>
>> > +1
>> >
>> > I'm a big advocate of privacy, and informed consent when choosing to
>> > give any of it away (e.g. allowing linking of activities to build a
>> > (pseudonymous or not) reputation).
>>
>> (philosophical side note: "consent" is not free of coercion, and thus
>> rather irrelevant, when Github is taking over 90% of Open Source
>> projects.)
>>
>> > [...] Personally, I'm actually quite
>> > happy to have my activities on github correlated with my identity.
>> > (Actually, it's a net plus, not a concession.)
>>
>> I understand your point, but I'd like to make a different choice.
>>
>> > Of course you can always set up any number of unrelated pseudonyms on
>> > github, delete cookies, use incognito mode, and even do everything via
>> > tor if you really want.
>>
>> No, I can't (unless I want to play cat and mouse with them, which is no
>> fun). And that is the whole of the problem, as I say in my other message.
>>
>> > However, while "Subscribe to Github" is a perfectly reasonable answer,
>> > and one that would in practice include more people than it would
>> > exclude (compared to our current system, or many alternatives), it's
>> > not like we're going to suddenly refuse all discussions of bugs on the
>> > mailing lists. We're low enough volume to be flexible. A real bug
>> > tracker is simply more useful for tracking issues than an inbox.
>>
>> As long as the mailing list stays, any concrete difficulty can be solved
>> when it arises through a constructive discussion, so nothing is lost!
>>
>> I trust that Cython won't ever do like some other projects, which have
>> suppressed any kind of non-Github contact channel. That would be the
>> real pain.
>>
>> > Does this alleviate your concerns?
>>
>> Not fully, but I can live with it :-)
>>
>> Baptiste
>>
>> ___
>> cython-devel mailing list
>> cython-devel@python.org
>> https://mail.python.org/mailman/listinfo/cython-devel
>
>
>
> ___
> cython-devel mailing list
> cython-devel@python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel

Re: [Cython] Cython infrastructure

[Robert Bradshaw]

On Wed, Jul 20, 2016 at 3:19 AM, Dima Pasechnik
 wrote:
> On Wed, Jul 20, 2016 at 10:55 AM, Baptiste Carvello
>  wrote:
>> Le 20/07/2016 08:01, Robert Bradshaw a écrit :
>>
>>> trac.cython.org
>>> This is probably the most controversial, but I think it makes sense to
>>> migrate to github issues. While clearly not as powerful, featureful,
>>> or customizable as trac, it seems it would fulfill our modest needs. I
>>> am happy to do the migration if no one objects.
>>
>> I suppose the dev guide will be updated with advice on how users without
>> a Github ID should report and interact with issues. It may lead to more
>> issues-related discussion on this mailing list.
>>
>> Cheers,
>> Baptiste
>>
>> For avoidance of doubt: "subscribe to Github" is not an acceptable
>> answer, given that this means consenting to be tracked, not only inside
>> the Cython project, but also across projects.
>
> Github is not spyware, IMHO. It would be good to understand which of these:
>
> https://help.github.com/articles/github-privacy-policy/
>
> you have a problem with.
>
> Yes, you will need to allow cookies in your browser.
> Anything else seems to be irrelevant.

+1

I'm a big advocate of privacy, and informed consent when choosing to
give any of it away (e.g. allowing linking of activities to build a
(pseudonymous or not) reputation). Personally, I'm actually quite
happy to have my activities on github correlated with my identity.
(Actually, it's a net plus, not a concession.)

Of course you can always set up any number of unrelated pseudonyms on
github, delete cookies, use incognito mode, and even do everything via
tor if you really want.

However, while "Subscribe to Github" is a perfectly reasonable answer,
and one that would in practice include more people than it would
exclude (compared to our current system, or many alternatives), it's
not like we're going to suddenly refuse all discussions of bugs on the
mailing lists. We're low enough volume to be flexible. A real bug
tracker is simply more useful for tracking issues than an inbox.

Does this alleviate your concerns?

- Robert
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel