[issue23591] Add Flags and IntFlags

[Ethan Furman]

Ethan Furman added the comment:

The values used in Flags are an implementation detail.  The expected, and 
supported, use is naming the flags:

class Color(Flags):
Red, Green, Blue

even custom names can be set using this method:

>>> class Color(enum.Flags):
...   red, green, blue
...   black = red & green & blue
... 
>>> list(Color)
[, , , ]

>>> class Color(enum.Flags):
...   red, green, blue
...   purple = red | blue
... 
>>> list(Color)
[, , , ]
>>> Color(5)

>>> Color(3)


I'll have to think about ensuring each bit is named -- it's only a (potential) 
problem if the values are specified manually.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Vedran Čačić]

Vedran Čačić added the comment:

I suppose you'll also forbid adding new members to Flags, just like Enum does, 
right? 
(https://docs.python.org/3/library/enum.html#restricted-subclassing-of-enumerations)
 Otherwise cropping -1 at a fixed number of bits might be very counterintuitive.

But still, your "simple algorithm" seems too simple. You might easily produce 
values you won't be able to interpret.

class ThirdBitMustBeSet(Flags):
FIRST = 5
SECOND = 6

>>> ThirdBitMustBeSet.FIRST | ThirdBitMustBeSet.SECOND

produces error when printing, but otherwise works completely fine. A debugging 
nightmare. :-/ I'd at least ensure each bit has a separate name if you're going 
to use that scheme.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Ethan Furman]

Ethan Furman added the comment:

'.0' does not have a name unless the user defines it; similarly, '.-1' does not 
have a name unless the user defines it.

For Flags, negative numbers are supported in order to specify which flags are 
desired, but the final representation will be zero or positive:

>>> class Hah(enum.Flags):
...   this, that, these, those, thuse
... 
>>> Hah(0)

>>> Hah(-1)


The algorithm is simple: start with the biggest Flag and mask off matching bits 
until all bits are are matched.  If any unmatched bits remain an error is 
raised.

If a user does horrible things like your Weird class then any breakage is on 
them.

As it stands, Weird(7) would be , and if A was not defined an 
error would be raised.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Vedran Čačić]

Vedran Čačić added the comment:

I think I have written about all the options in detail in 
http://bugs.python.org/msg272732. If I see it right, your ".0" corresponds to 
what I called ".__NONE__". It is a "right way" to do it, but for it to be 
complete, you also have to add ".__ALL__", representing -1, as its complement. 
If you do both of these, I'm fine with it, and we can discuss further 
problems... :-)

... like inverting representation. If you have non-powers of 2 named, how do 
you know which exact flags a value consists of?

class Weird(Flags):
AB = 3
C = 4
BC = 6
A = 1

what is Weird(7)? Weird.(AB|C) or Weird.(A|BC)? What if you don't have C (or A) 
as a name? Are you going to employ a minimal exact cover algorithm? :-O

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Ethan Furman]

Ethan Furman added the comment:

The zero value is special in that it represents a flag with nothing set.  
Excluding it has its own problems, but if you have an argument for that 
behavior I'm listening.

Any other non-Flag value is not allowed, and will raise an error (IntFlags 
won't share this behavior).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Vedran Čačić]

Vedran Čačić added the comment:

So, in fact, your Flags are simply an overlayed namespace over int (the way to 
give some ints sticky names), and any int is accessible from any Flags, no 
matter whether it has a name or not? I must say that to me it seems radically 
different than (Int)Enum philosophy.

class MyIntEnum(IntEnum):
A = 1

>>> MyIntEnum(0)
ValueError: 0 is not a valid MyIntEnum

So, flags are not enums, nor they share the same principles (identity, 
exclusiveness). Why are they in the enum module at all?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Ethan Furman]

Ethan Furman added the comment:

Serhiy's patch is only for IntFlags, and my patch hasn't yet fully incorporated 
his (many thanks for decompose!)

For IntFlags I do not expect to have very many instances alive at once, 
especially since not-bitwise operators will lose the IntFlag class and become 
plain ints.

Flags are closed.  If the zero value is not specified the repr and str are:

>>> Hah(0)

>>> str(Hah(0))
'Hah.0'

(An RGB class might be:  )

A question I have about IntFlags:

If a third-party lib specifies that certain bits are reserved and should always 
be zero (or at least not changed), do we want to add some easy support for that?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Vedran Čačić]

Vedran Čačić added the comment:

Yes, IntFlags sidesteps a lot of these issues (though not all: inconsistency 
with a lot of principles of IntEnum is still jarring). But I thought we were 
talking about Flags too (it is not in the patch, as far as I see).

But now I see that Flags was kinda abandoned, and we're going forward only with 
IntFlags, right? Maybe the title should be changed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

You still can use identity testing for named instances of IntFlags. But since 
the purpose of IntFlags is replacing int flags, tested values can be int (e.g. 
when read from files as ints). For unknown values you should use either 
equality testing or wrap them in IntFlags.

In your example MyFlags.FIRST & MyFlags.SECOND is MyFlags.NONE. If MyFlags.NONE 
not exists, the result is MyFlags(0). You can apply the patch and experiment 
with it.

--
Added file: http://bugs.python.org/file44114/intflags_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Vedran Čačić]

Vedran Čačić added the comment:

It's true, but it seems that with Enums, we're trying to retrain people to use 
identity testing (see https://docs.python.org/3/library/enum.html#comparisons). 
It would be unfortunate if we had to reuntrain them. :-/

Ethan's proposal (of caching weakrefs) is fine, if he manages to do it 
correctly (I didn't). It's much more complicated than it seems at first, since 
it has to work during the class definition itself, too.

Ok, second problem I encountered: zeros. Imagine

class MyFlags(enum.Flags):
NONE = 0
FIRST = 1 << 0
SECOND = 1 << 1

What is MyFlags.FIRST & MyFlags.SECOND? 0 or MyFlags.NONE? Or even False? :-) I 
would almost be for the third option, if not for the fact that & is used not 
only for "membership" testing, but also (with ~) for clearing flags. Imitating 
Go, we might want to introduce another operator, &~, for clearing flags, but 
that would probably be a too great disruption.

MyFlags.NONE seems cool, but then we have an enum member that is false, another 
thing we have tried to avoid with current Enum design (that's why functional 
API starts values at 1, for example). And what about the case when MyFlags 
doesn't define NONE, only FIRST and SECOND?

If you opt for 0, you open another can of worms: first, return type depends on 
argument values, and Guido hates that. :-) Also, that 0 is still somehow tied 
to MyFlags: it would be a type error to write (MyFlags.FIRST & MyFlags.SECOND) 
| AnotherFlags.SOMETHING, right?

Maybe the correct solution is to _always_ have a special value in every Flags 
class for such cases, but then what is it's ~__NONE__? :-) Fortunately, once 
you add __NONE__ and __ALL__, you get a Boolean closure, and you can 
meaningfully define &, |, ^ and ~ in a type-safe way. But that's probably 
overkill.

And so on... there are a lot of problems, and I've been through them. I'd like 
you to succeed where I didn't, but I have a bad feeling.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I don't think this is a problem. For now os.O_WRONLY is os.O_WRONLY, but 
os.O_WRONLY|os.O_CREAT|os.O_TRUNC is not os.O_WRONLY|os.O_CREAT|os.O_TRUNC. 
There is an exponential number of combinations (actually infinite number if 
count not named bits) and you shouldn't use identity check for flags.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Ethan Furman]

Ethan Furman added the comment:

Currently, the patch has the main values pre-created (so identity works as 
expected), and combinations are created on the fly (like normal class 
instances) -- in which case identity cannot be relied upon.

Once I have the basic work done, I'll go back and add a cache so each unique 
value is only created once;  at some point I'll make those cached values 
weakref'ed so they disappear when no longer used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Vedran Čačić]

Vedran Čačić added the comment:

I tried to implement this once. The biggest problem I encountered is  
inconsistency with Enum identity: Enum's main idea is that its objects are 
pre-created and __new__ just return one of them, so equality is simply identity.

If you try to do this with flags, you quickly hit the exponential blowup and 
with most flags applications, it's utterly impractical to construct all the 
combinations at the start. So you have MyFlags.a|MyFlags.b is not 
MyFlags.a|MyFlags.b, and that is very weird and unexpected (especially if 
MyFlags.a is MyFlags.a, which it is if this case just delegates to 
Enum.__new__).

How does this patch propose to solve this problem?

--
nosy: +veky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Ethan Furman]

Ethan Furman added the comment:

IntFlags will be in before feature freeze.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Due to dynamic typing EnumSet actually is not needed in Python. You can use 
sets, tuples or lists for combining enums in new API. IntFlags is needed for 
old API that already uses ints. I think introducing non-int Flags is 
overengineering, can't imagine somebody will use it in new code.

There is no much time left before feature freeze. If general IntFlags will not 
added, I'm going to open separate issues for adding specialized class for every 
particular case (re, os, etc).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Ethan Furman]

Ethan Furman added the comment:

The idea behind Flags is that they are easily combined enums.  The advantage 
they have over IntFlags is they do not interact with integers, and they cannot 
have non-existing values used.  By keeping the same API as IntFlags we only 
have two APIs instead of three (Enum and Flag instead of Enum, EnumSet, and 
IntFlag), and also make life easier for changing code from one to the other 
(Flag <-> IntFlag).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The name IntFlags is inspired by the Flags attribute in C# (this is the most 
close concept). The "Int" prefix is added for parallel with IntEnum and because 
the class behaves as int in some contexts (supports operators |, &, ^, ~ and 
can be passed to functions that require int). I don't know whether there is a 
need in Flags class that is not a subclass of int.

Other languages provide something like EnumSet - a specialized set of enums. 
But it has different interface.

1) If there are enum members FOO and BAR, their union is {FOO, BAR}, not 
FOO|BAR.
2) The union member itself is not a set.
3) A set can contain only predefined set of values (IntFlag allows non-defined 
bits be set).

If you thing that non-int flags could be useful, you can consider adding 
EnumSet for general enums and IntFlags (or just Flags) for int-like enums. But 
I think that new API can just use a set or a tuple of enums. IntFlags is needed 
for compatibility with old API or C API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23591] Add Flags and IntFlags

[Ethan Furman]

Ethan Furman added the comment:

Flags -> int-backed, but not ints; useful for pure Python flags
IntFlags -> ints (like IntEnum); useful for interfacing with other systems

Are there better names?

--
title: Add IntFlags -> Add Flags and IntFlags

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com