Re: Enums, bitfields and wl_arrays

[Erik De Rijcke]

On Tue, Oct 13, 2015 at 9:27 PM, Auke Booij  wrote:
>
> I really don't understand this discussion. Is the claim that the usage
> of enums in java is problematic, because inserting a new value in an
> existing enum might change the index of later values, thereby creating
> an inconsistency?
>

Correct.


> If so, all I can say that clearly the object you want to associate
> with wayland-style enums is not whatever Java has invented for an
> "enum". The values that are associated to names inside wayland enums
> are very clear-defined, and if a language cannot safely couple names
> back to values, then that is a language that does not understand the
> concept of a variable. If this is a fundamental problem about Java
> enums, you better find a way around it, but I don't see how this is in
> any way a problem on the wayland side: there are enums, and enums
> contain names, and those names have values.
>

If wayland enums would not explicitly declare values, you'd have the same
problem in C. However since wayland enums do explicitly declare their
value, *and* because C allows you to override an enum 'internal value', you
don't have that problem here. Unfortunately, Sun in all it's infinite
wisdom decided that Java enum 'internal values' can not be overridden. Only
new 'properties' can be attached.


> But if this is indeed the issue you are discussing, then this must
> have already been a problem before the introduction of additional XML
> attributes.
>

Correct. My current Java bindings create a Java enum with the wayland enum
value as an additional property. If the order would change, that would be a
potentially breaking change.

Can we agree on the following? Until there is any firm specification
> of open/closed enums, every enum should be considered open, and
> although some legal values might be listed, others might not be, and
> some might only be legal sometimes.


Ok for me but it would be very nice to have the open/closed information ;)


> New values may be added (but not
> changed or removed) to protocol specifications without introducing any
> compatibility issues.
>

This is not clear for me. Compatibility issues for the current C bindings
or for all language bindings? If it's for all language bindings than this
might introduce a whole explosion of derived implicit specifications for
each language. Eg. the enum order in Java. Better would be to specify a
goal "no compatibility issues" and a set of form specifications eg. "the
order of an enum shall not be changed".


> ___
> wayland-devel mailing list
> wayland-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Daniel Stone]

Hi,

On 15 October 2015 at 09:18, Erik De Rijcke  wrote:
> On Tue, Oct 13, 2015 at 9:27 PM, Auke Booij  wrote:
>> New values may be added (but not
>> changed or removed) to protocol specifications without introducing any
>> compatibility issues.
>
> This is not clear for me. Compatibility issues for the current C bindings or
> for all language bindings? If it's for all language bindings than this might
> introduce a whole explosion of derived implicit specifications for each
> language. Eg. the enum order in Java. Better would be to specify a goal "no
> compatibility issues" and a set of form specifications eg. "the order of an
> enum shall not be changed".

Yes, agreed. That's the only tractable way: the definition of a closed
enum requires strictly linear append-only additions. Next topic. :)

Cheers,
Daniel
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Auke Booij]

On 15 October 2015 at 09:18, Erik De Rijcke  wrote:
> If wayland enums would not explicitly declare values, you'd have the same
> problem in C. However since wayland enums do explicitly declare their value,
> *and* because C allows you to override an enum 'internal value', you don't
> have that problem here. Unfortunately, Sun in all it's infinite wisdom
> decided that Java enum 'internal values' can not be overridden. Only new
> 'properties' can be attached.
>
>>
>> But if this is indeed the issue you are discussing, then this must
>> have already been a problem before the introduction of additional XML
>> attributes.
>
>
> Correct. My current Java bindings create a Java enum with the wayland enum
> value as an additional property. If the order would change, that would be a
> potentially breaking change.
>
>> Can we agree on the following? Until there is any firm specification
>> of open/closed enums, every enum should be considered open, and
>> although some legal values might be listed, others might not be, and
>> some might only be legal sometimes.
>
>
> Ok for me but it would be very nice to have the open/closed information ;)
>
>>
>> New values may be added (but not
>> changed or removed) to protocol specifications without introducing any
>> compatibility issues.
>
>
> This is not clear for me. Compatibility issues for the current C bindings or
> for all language bindings? If it's for all language bindings than this might
> introduce a whole explosion of derived implicit specifications for each
> language. Eg. the enum order in Java. Better would be to specify a goal "no
> compatibility issues" and a set of form specifications eg. "the order of an
> enum shall not be changed".

This is exactly the kind of specification that I do not intend to
make, since it is (ostensibly) caused solely by a broken
implementation of enums in Java. It's not a problem in C, and not a
problem in any language that understands the concept of a variable.

On 13 October 2015 at 23:15, Bryce Harrington  wrote:
> On Tue, Oct 13, 2015 at 08:27:58PM +0100, Auke Booij wrote:
>> But if this is indeed the issue you are discussing, then this must
>> have already been a problem before the introduction of additional XML
>> attributes.
>>
>> Can we agree on the following? Until there is any firm specification
>> of open/closed enums, every enum should be considered open, and
>> although some legal values might be listed, others might not be, and
>> some might only be legal sometimes. New values may be added (but not
>> changed or removed) to protocol specifications without introducing any
>> compatibility issues.
>
> So, worst case if the binding language is inflexible here, then it'd
> need to maintain a mapping for the values to resequence them?

Yes, that is my point exactly.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

Hi,

On Wed, Oct 14, 2015 at 10:25 AM, Pekka Paalanen  wrote:
> On Tue, 13 Oct 2015 20:27:58 +0100
> Auke Booij  wrote:
>
>> On 13 October 2015 at 16:19, Solerman Kaplon  wrote:
>> > Em 13-10-2015 11:35, Nils Chr. Brause escreveu:
>> >>
>> >> In C++ the order doesn't matter either, since each entry has a defined
>> >> value. I wonder why this is different in Java?
>
>> I really don't understand this discussion. Is the claim that the usage
>> of enums in java is problematic, because inserting a new value in an
>> existing enum might change the index of later values, thereby creating
>> an inconsistency?
>
> Yes, this is how I understood the problem from the irc discussion,
> indeed.
>
>> If so, all I can say that clearly the object you want to associate
>> with wayland-style enums is not whatever Java has invented for an
>> "enum". The values that are associated to names inside wayland enums
>> are very clear-defined, and if a language cannot safely couple names
>> back to values, then that is a language that does not understand the
>> concept of a variable. If this is a fundamental problem about Java
>> enums, you better find a way around it, but I don't see how this is in
>> any way a problem on the wayland side: there are enums, and enums
>> contain names, and those names have values.
>>
>> But if this is indeed the issue you are discussing, then this must
>> have already been a problem before the introduction of additional XML
>> attributes.
>>
>> Can we agree on the following? Until there is any firm specification
>> of open/closed enums, every enum should be considered open, and
>> although some legal values might be listed, others might not be, and
>> some might only be legal sometimes. New values may be added (but not
>> changed or removed) to protocol specifications without introducing any
>> compatibility issues.
>
> FWIW, I would be happy with that.

I would be happy with that, too. Also, this would bring us a step forward. :)

> Solerman, you forgot to reply-to-all. I added back the CCs.
>
>
> Thanks,
> pq

Cheers,
Nils
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

On Tue, 13 Oct 2015 20:27:58 +0100
Auke Booij  wrote:

> On 13 October 2015 at 16:19, Solerman Kaplon  wrote:
> > Em 13-10-2015 11:35, Nils Chr. Brause escreveu:
> >>
> >> In C++ the order doesn't matter either, since each entry has a defined
> >> value. I wonder why this is different in Java?

> I really don't understand this discussion. Is the claim that the usage
> of enums in java is problematic, because inserting a new value in an
> existing enum might change the index of later values, thereby creating
> an inconsistency?

Yes, this is how I understood the problem from the irc discussion,
indeed.

> If so, all I can say that clearly the object you want to associate
> with wayland-style enums is not whatever Java has invented for an
> "enum". The values that are associated to names inside wayland enums
> are very clear-defined, and if a language cannot safely couple names
> back to values, then that is a language that does not understand the
> concept of a variable. If this is a fundamental problem about Java
> enums, you better find a way around it, but I don't see how this is in
> any way a problem on the wayland side: there are enums, and enums
> contain names, and those names have values.
> 
> But if this is indeed the issue you are discussing, then this must
> have already been a problem before the introduction of additional XML
> attributes.
> 
> Can we agree on the following? Until there is any firm specification
> of open/closed enums, every enum should be considered open, and
> although some legal values might be listed, others might not be, and
> some might only be legal sometimes. New values may be added (but not
> changed or removed) to protocol specifications without introducing any
> compatibility issues.

FWIW, I would be happy with that.

Solerman, you forgot to reply-to-all. I added back the CCs.


Thanks,
pq


pgpLYZkTEJb3d.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

On Tue, 13 Oct 2015 10:19:43 -0700
Bill Spitzak  wrote:

> On 10/13/2015 08:18 AM, Erik De Rijcke wrote:
> 
> > Because Java is not C/C++. In Java, enum values are unique by instance
> > alone. It does not have user 'defined' value(s) that make it unique. It
> > *can* have user defined properties (like any object) in addition to
> >   implicit compiler generated enum type specific properties. One of
> > those properties is it's "ordinal". Change the order of the enums and
> > the ordinal property will change, which can cause undefined behavior.
> > Now you need to wonder no more. :)
> 
> The language binding should sort the enum by numerical value and use 
> that to determine the ordinal number. Ie it is based on the numerical 
> value, not on the location in the xml file.

This still has the very same issue if you need to add a new value that
falls between existing values in the sort order.

You dropped all CC's, btw.


Thanks,
pq


pgpT25qHbgm3t.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Erik De Rijcke]

On Tue, Oct 13, 2015 at 4:35 PM, Nils Chr. Brause 
wrote:

> Hi,
>
> On Mon, Oct 12, 2015 at 11:08 AM, Pekka Paalanen 
> wrote:
> > On Mon, 12 Oct 2015 10:41:22 +0200
> > Erik De Rijcke  wrote:
> >
> >> Adding enum members is backward compatible for Java. If you compile
> against
> >> an enum with 2 members, and later on a new member is added, you can
> simply
> >> use the new version of the enum.
> >>
> >> Important however is that the order of old members do not change when
> new
> >> members are added.
> >
> > This is important. I don't think anyone has ever mentioned anything
> > like this before.
> >
> > We certainly have not expected the order to matter inside an 
> > tag, since we require the value to be explicitly given. Not when you
> > are coming from C, anyway.
>
> In C++ the order doesn't matter either, since each entry has a defined
> value.
> I wonder why this is different in Java?
>
Because Java is not C/C++. In Java, enum values are unique by instance
alone. It does not have user 'defined' value(s) that make it unique. It
*can* have user defined properties (like any object) in addition to
 implicit compiler generated enum type specific properties. One of those
properties is it's "ordinal". Change the order of the enums and the ordinal
property will change, which can cause undefined behavior. Now you need to
wonder no more. :)
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Solerman Kaplon]

Em 13-10-2015 11:35, Nils Chr. Brause escreveu:
In C++ the order doesn't matter either, since each entry has a defined value. 
I wonder why this is different in Java?


Java Enums doesn't have "value". It just a class instance. But since it's a 
class one can add regular fields to it, like this (writing of the top of my head):


public static enum Orientation {
  Portrait(1), Landscape(2);

  final int value;

  Orientation(int value) {
this.value = value;
  }

  public int getValue() {
return this.value;
  }
}

Solerman
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

Hi,

On Mon, Oct 12, 2015 at 11:08 AM, Pekka Paalanen  wrote:
> On Mon, 12 Oct 2015 10:41:22 +0200
> Erik De Rijcke  wrote:
>
>> Adding enum members is backward compatible for Java. If you compile against
>> an enum with 2 members, and later on a new member is added, you can simply
>> use the new version of the enum.
>>
>> Important however is that the order of old members do not change when new
>> members are added.
>
> This is important. I don't think anyone has ever mentioned anything
> like this before.
>
> We certainly have not expected the order to matter inside an 
> tag, since we require the value to be explicitly given. Not when you
> are coming from C, anyway.

In C++ the order doesn't matter either, since each entry has a defined value.
I wonder why this is different in Java?

> Apparently we would need to start enforcing the rule:
>
> "Only ever add new enum value to the end of a closed enum."
>
> Yeah, now we also need the concept of open and closed enums, because
> for something like used in xdg-shell for window states, the whole enum
> concept doesn't work e.g. in Java (I hear).

A hint whether an enum is open or closed would probably be useful for
languages like Java. The binding developers need to come up with a solution
for their particular binding if they want to use type safe enumerations.

> Should we be adding all these to the Wayland XML language or not?

As far as I'm concerned, everything that can make the life of a language
binding developer easier and doesn't change the C API should be included
in the Wayland XML file.

> Thanks,
> pq

Cheers,
Nils
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

On Tue, Oct 13, 2015 at 5:18 PM, Erik De Rijcke  wrote:
>
>
> On Tue, Oct 13, 2015 at 4:35 PM, Nils Chr. Brause 
> wrote:
>>
>> Hi,
>>
>> On Mon, Oct 12, 2015 at 11:08 AM, Pekka Paalanen 
>> wrote:
>> > On Mon, 12 Oct 2015 10:41:22 +0200
>> > Erik De Rijcke  wrote:
>> >
>> >> Adding enum members is backward compatible for Java. If you compile
>> >> against
>> >> an enum with 2 members, and later on a new member is added, you can
>> >> simply
>> >> use the new version of the enum.
>> >>
>> >> Important however is that the order of old members do not change when
>> >> new
>> >> members are added.
>> >
>> > This is important. I don't think anyone has ever mentioned anything
>> > like this before.
>> >
>> > We certainly have not expected the order to matter inside an 
>> > tag, since we require the value to be explicitly given. Not when you
>> > are coming from C, anyway.
>>
>> In C++ the order doesn't matter either, since each entry has a defined
>> value.
>> I wonder why this is different in Java?
>
> Because Java is not C/C++. In Java, enum values are unique by instance
> alone. It does not have user 'defined' value(s) that make it unique. It
> *can* have user defined properties (like any object) in addition to
> implicit compiler generated enum type specific properties. One of those
> properties is it's "ordinal". Change the order of the enums and the ordinal
> property will change, which can cause undefined behavior. Now you need to
> wonder no more. :)

I'm not a Java developer, I just want to understand and help :)
I suppose you couldn't just ignore the ordinal property and solely rely on some
user defined propery, which holds the value defined in the XML file?

>
>
> ___
> wayland-devel mailing list
> wayland-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Bill Spitzak]

On 10/13/2015 08:18 AM, Erik De Rijcke wrote:


Because Java is not C/C++. In Java, enum values are unique by instance
alone. It does not have user 'defined' value(s) that make it unique. It
*can* have user defined properties (like any object) in addition to
  implicit compiler generated enum type specific properties. One of
those properties is it's "ordinal". Change the order of the enums and
the ordinal property will change, which can cause undefined behavior.
Now you need to wonder no more. :)


The language binding should sort the enum by numerical value and use 
that to determine the ordinal number. Ie it is based on the numerical 
value, not on the location in the xml file.


___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Auke Booij]

On 13 October 2015 at 16:19, Solerman Kaplon  wrote:
> Em 13-10-2015 11:35, Nils Chr. Brause escreveu:
>>
>> In C++ the order doesn't matter either, since each entry has a defined
>> value. I wonder why this is different in Java?
>
>
> Java Enums doesn't have "value". It just a class instance. But since it's a
> class one can add regular fields to it, like this (writing of the top of my
> head):
>
> public static enum Orientation {
>   Portrait(1), Landscape(2);
>
>   final int value;
>
>   Orientation(int value) {
> this.value = value;
>   }
>
>   public int getValue() {
> return this.value;
>   }
> }
>
> Solerman


I really don't understand this discussion. Is the claim that the usage
of enums in java is problematic, because inserting a new value in an
existing enum might change the index of later values, thereby creating
an inconsistency?

If so, all I can say that clearly the object you want to associate
with wayland-style enums is not whatever Java has invented for an
"enum". The values that are associated to names inside wayland enums
are very clear-defined, and if a language cannot safely couple names
back to values, then that is a language that does not understand the
concept of a variable. If this is a fundamental problem about Java
enums, you better find a way around it, but I don't see how this is in
any way a problem on the wayland side: there are enums, and enums
contain names, and those names have values.

But if this is indeed the issue you are discussing, then this must
have already been a problem before the introduction of additional XML
attributes.

Can we agree on the following? Until there is any firm specification
of open/closed enums, every enum should be considered open, and
although some legal values might be listed, others might not be, and
some might only be legal sometimes. New values may be added (but not
changed or removed) to protocol specifications without introducing any
compatibility issues.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Bryce Harrington]

On Tue, Oct 13, 2015 at 08:27:58PM +0100, Auke Booij wrote:
> On 13 October 2015 at 16:19, Solerman Kaplon  wrote:
> > Em 13-10-2015 11:35, Nils Chr. Brause escreveu:
> >>
> >> In C++ the order doesn't matter either, since each entry has a defined
> >> value. I wonder why this is different in Java?
> >
> >
> > Java Enums doesn't have "value". It just a class instance. But since it's a
> > class one can add regular fields to it, like this (writing of the top of my
> > head):
> >
> > public static enum Orientation {
> >   Portrait(1), Landscape(2);
> >
> >   final int value;
> >
> >   Orientation(int value) {
> > this.value = value;
> >   }
> >
> >   public int getValue() {
> > return this.value;
> >   }
> > }
> >
> > Solerman
> 
> 
> I really don't understand this discussion. Is the claim that the usage
> of enums in java is problematic, because inserting a new value in an
> existing enum might change the index of later values, thereby creating
> an inconsistency?
> 
> If so, all I can say that clearly the object you want to associate
> with wayland-style enums is not whatever Java has invented for an
> "enum". The values that are associated to names inside wayland enums
> are very clear-defined, and if a language cannot safely couple names
> back to values, then that is a language that does not understand the
> concept of a variable. If this is a fundamental problem about Java
> enums, you better find a way around it, but I don't see how this is in
> any way a problem on the wayland side: there are enums, and enums
> contain names, and those names have values.

I got the impression based on the prior discussion that there were
already several ideas for working around this in java.  So perhaps it
can be just an internal implementation detail of that binding.

> But if this is indeed the issue you are discussing, then this must
> have already been a problem before the introduction of additional XML
> attributes.
> 
> Can we agree on the following? Until there is any firm specification
> of open/closed enums, every enum should be considered open, and
> although some legal values might be listed, others might not be, and
> some might only be legal sometimes. New values may be added (but not
> changed or removed) to protocol specifications without introducing any
> compatibility issues.

So, worst case if the binding language is inflexible here, then it'd
need to maintain a mapping for the values to resequence them?

Bryce


___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

On Mon, 12 Oct 2015 10:41:22 +0200
Erik De Rijcke  wrote:

> Adding enum members is backward compatible for Java. If you compile against
> an enum with 2 members, and later on a new member is added, you can simply
> use the new version of the enum.
> 
> Important however is that the order of old members do not change when new
> members are added.

This is important. I don't think anyone has ever mentioned anything
like this before.

We certainly have not expected the order to matter inside an 
tag, since we require the value to be explicitly given. Not when you
are coming from C, anyway.

Apparently we would need to start enforcing the rule:

"Only ever add new enum value to the end of a closed enum."

Yeah, now we also need the concept of open and closed enums, because
for something like used in xdg-shell for window states, the whole enum
concept doesn't work e.g. in Java (I hear).

Should we be adding all these to the Wayland XML language or not?


Thanks,
pq


pgpHONLY0Z_zc.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

Hi,

On Thu, Oct 8, 2015 at 10:13 PM, Daniel Stone  wrote:
> Hi,
>
> On 5 October 2015 at 21:31, Victor Berger  wrote:
>> Le 2015-10-05 21:53, Nils Chr. Brause a écrit :
>>> Since a language binding should support unknown values anyway, I don't
>>> quite
>>> see the benefit of an 'open' flag.
>>
>> It depends. In some case, it could be valid to support unknown values by
>> simply ignoring them.
>>
>> Consider a program written against a certain version of the protocol. Then,
>> it is used on a compositor with a newer version of the protocol, where one
>> of the enums have one more possible value.
>>
>> What can the client do if it receives this value ? It has no possible way to
>> give it a proper meaning, so it can either ignore it, or trigger a fatal
>> error. Are there other options ?
>>
>> In this case, for a "closed" enum, the binding can decide to ignore any
>> value received on the wire that is not described in the XML, as anyway, the
>> client would not know what to do with it (and if they knew, they should be
>> using a more recent version of the binding).
>
> This should never happen. The compositor is responsible for tracking
> which version the client has bound, and not sending it values it
> doesn't know what to do with.
>
> I've heard mention that xdg-shell allows this, but I also can't see it
> anywhere in the spec. Jasper?
>
>> This can be useful in some languages to allow such a thing. For example in
>> Rust, handling an "open" enum would require some overhead in terms of
>> usability of the API, to gracefully handle unknown values. While simply
>> ignoring them would lead to a much simpler design. Unless it is not a good
>> idea to ignore unkown values, in which case I'd gladly read an explanation
>> of why it is and which problems I missed.
>>
>> Without this flag, I have to choices: either consider enums as uints/ints
>> and a set of constants to compare against (like C), of put this overhead on
>> every single enum. To be honest, none of these possibilities feel
>> satisfying. But They are still possible outcomes, hence the bindings _can_
>> live without this indication.
>
> My main concern was what I raised on IRC and Pekka in email:
>> > There is a paradox: if the new attributes affect codegen for some
>> > languages in a way that will break the build (which is exactly the
>> > reason why they are wanted: to enforce type-safety), we cannot add
>> > these attributes to the existing protocols, most importantly
>> > wayland.xml, because it would "break existing code". You might escape
>> > that paradox once, by claiming that there is no "existing code", after
>> > all the generators for these languages will be new. But this escape
>> > hatch will only work once. When these generators get into use, we can
>> > no longer add the attributes to stable protocols without breaking
>> > things. Not even if the generators had a "legacy mode" that ignores the
>> > new attributes, because they would then regress on arguments that
>> > already had the new attributes set.
>
> The exact scenario being:
>   - protocol v1 ships an enum with two possible values
>   - protocol v2 adds another possible value for that enum
>   - a client built against v1 will only ever speak v1 to the
> compositor, but at runtime the library may support v2
>   - a client built against v2 may also speak to a v1 compositor
>
> For Rust, the answer here appears to be that the packages will all be
> tightly bounded in versioning between build and runtime, so this
> scenario is irrelevant.
>
> For other bindings though, what happens here? Is ProtocolEnum
> type-compatible, such that a client built against v1 bindings can pass
> those values into a library which also supports v2? Does every bump
> give you a new type, such that ProtocolEnumV1, ProtocolEnumV2, etc,
> are all different types? If so, do you generate new call signatures
> for every request/event which takes that enum?

I can only speak for the C++ bindings here. I just added a a new entry
to the wl_seat::capability bitfield, recompiled the library and ran a client
without rebuilding it and it worked just fine. Therefore it doesn't seem to
be an issue here. :)

> This seems like it could get hairy. But if the answer from
> Java/Haskell/etc is that the types are compatible, or there's some
> other way to work around this such that it is possible to add new enum
> values without breaking all existing users of rich bindings, then
> great!
>
>> I hope this was a clear explanation of why I think these values can be
>> useful. Without then, I cannot really make use of the new attribute linking
>> an argument to its enum, apart from writing it in the documentation and
>> using (u)ints everywhere.
>
> It has been very helpful, thanks.
>
> Seems like, with this question resolved, we've finally arrived at a
> conclusion here.
>
> Cheers,
> Daniel
___
wayland-devel mailing list

Re: Enums, bitfields and wl_arrays

[Daniel Stone]

Hi,

On 5 October 2015 at 21:31, Victor Berger  wrote:
> Le 2015-10-05 21:53, Nils Chr. Brause a écrit :
>> Since a language binding should support unknown values anyway, I don't
>> quite
>> see the benefit of an 'open' flag.
>
> It depends. In some case, it could be valid to support unknown values by
> simply ignoring them.
>
> Consider a program written against a certain version of the protocol. Then,
> it is used on a compositor with a newer version of the protocol, where one
> of the enums have one more possible value.
>
> What can the client do if it receives this value ? It has no possible way to
> give it a proper meaning, so it can either ignore it, or trigger a fatal
> error. Are there other options ?
>
> In this case, for a "closed" enum, the binding can decide to ignore any
> value received on the wire that is not described in the XML, as anyway, the
> client would not know what to do with it (and if they knew, they should be
> using a more recent version of the binding).

This should never happen. The compositor is responsible for tracking
which version the client has bound, and not sending it values it
doesn't know what to do with.

I've heard mention that xdg-shell allows this, but I also can't see it
anywhere in the spec. Jasper?

> This can be useful in some languages to allow such a thing. For example in
> Rust, handling an "open" enum would require some overhead in terms of
> usability of the API, to gracefully handle unknown values. While simply
> ignoring them would lead to a much simpler design. Unless it is not a good
> idea to ignore unkown values, in which case I'd gladly read an explanation
> of why it is and which problems I missed.
>
> Without this flag, I have to choices: either consider enums as uints/ints
> and a set of constants to compare against (like C), of put this overhead on
> every single enum. To be honest, none of these possibilities feel
> satisfying. But They are still possible outcomes, hence the bindings _can_
> live without this indication.

My main concern was what I raised on IRC and Pekka in email:
> > There is a paradox: if the new attributes affect codegen for some
> > languages in a way that will break the build (which is exactly the
> > reason why they are wanted: to enforce type-safety), we cannot add
> > these attributes to the existing protocols, most importantly
> > wayland.xml, because it would "break existing code". You might escape
> > that paradox once, by claiming that there is no "existing code", after
> > all the generators for these languages will be new. But this escape
> > hatch will only work once. When these generators get into use, we can
> > no longer add the attributes to stable protocols without breaking
> > things. Not even if the generators had a "legacy mode" that ignores the
> > new attributes, because they would then regress on arguments that
> > already had the new attributes set.

The exact scenario being:
  - protocol v1 ships an enum with two possible values
  - protocol v2 adds another possible value for that enum
  - a client built against v1 will only ever speak v1 to the
compositor, but at runtime the library may support v2
  - a client built against v2 may also speak to a v1 compositor

For Rust, the answer here appears to be that the packages will all be
tightly bounded in versioning between build and runtime, so this
scenario is irrelevant.

For other bindings though, what happens here? Is ProtocolEnum
type-compatible, such that a client built against v1 bindings can pass
those values into a library which also supports v2? Does every bump
give you a new type, such that ProtocolEnumV1, ProtocolEnumV2, etc,
are all different types? If so, do you generate new call signatures
for every request/event which takes that enum?

This seems like it could get hairy. But if the answer from
Java/Haskell/etc is that the types are compatible, or there's some
other way to work around this such that it is possible to add new enum
values without breaking all existing users of rich bindings, then
great!

> I hope this was a clear explanation of why I think these values can be
> useful. Without then, I cannot really make use of the new attribute linking
> an argument to its enum, apart from writing it in the documentation and
> using (u)ints everywhere.

It has been very helpful, thanks.

Seems like, with this question resolved, we've finally arrived at a
conclusion here.

Cheers,
Daniel
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Daniel Stone]

Hi,

On 8 October 2015 at 21:13, Daniel Stone  wrote:
> On 5 October 2015 at 21:31, Victor Berger  wrote:
>> In this case, for a "closed" enum, the binding can decide to ignore any
>> value received on the wire that is not described in the XML, as anyway, the
>> client would not know what to do with it (and if they knew, they should be
>> using a more recent version of the binding).
>
> This should never happen. The compositor is responsible for tracking
> which version the client has bound, and not sending it values it
> doesn't know what to do with.
>
> I've heard mention that xdg-shell allows this, but I also can't see it
> anywhere in the spec. Jasper?

Whoops ...

9:18 PM  daniels, sorry, what were we talking about with enums?
9:20 PM  Jasper: people are concerned that xdg-shell has
'open' enums, in that a compositor can send a configure event with
enum values unknown to the client
9:20 PM  daniels, that's correct
9:20 PM  Jasper: bollocks, really?
9:20 PM  daniels,
http://cgit.freedesktop.org/wayland/weston/tree/protocol/xdg-shell.xml#n321
9:23 PM  Jasper: ah, i see - that does make quite a bit of sense
9:23 PM  Jasper: but i do wonder if we should bound that by
what the client actually supports, e.g. don't send gnome-specific
values unless the client has bound gtk_shell
9:23 PM  daniels, the state mechanism is designed to solve a
very specific race condition, so we don't want people inventing their
own protocols to do the same.
9:24 PM  Jasper: oh, indeed
9:24 PM  daniels, I would be fine with that, but I'm not sure
it helps Rust clients.
9:24 PM  Jasper: indeed, more of a general musing
9:24 PM  Jasper: thanks for pointing that out; not sure how i missed it

How does that play with Java/Haskell/etc? Would that just degenerate
into a plain undecorated uint, or ... ?

Cheers,
Daniel
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Bill Spitzak]

On Mon, Oct 5, 2015 at 6:03 AM, Pekka Paalanen  wrote:


> For me, that depends only on what requirements a "bitfield" has. Does
> it have to be uint? Do the values have to be POT? That's all up to you,
> as it has no effect on C bindings nor generated documentation other
> than wording.
>
>
I believe it is ok for the langauge binding to be incapable of producing
negative values for an enum argument, even if it is declared as an int
instead of a uint. I also feel that if, after careful examination, it is
determined that no working program would ever try to send or interpret a
negative value, that the use of int instead of uint be defined as an error
in the xml file, and fixed without changing the protocol version.

Non-power-of-2 names are a requirement, and are trivial to support.

  enum {
FLIP_VERTICAL = 1,
FLIP_HORIZONTAL = 2,
ROTATE_180 = 3 // so programmer does not have to realize the 2 flips
look like a rotation
  };

If a language really can't deal with them, then it can skip those values. I
find it hard to believe there is any such language! It can however make it
impossible to bit-test against a non-power-of-2 value.

For both enums and bitfields it is also useful to give more than one name
to the same value. For instance a "modifier" name used when building a
value, verses a more obvious name for the value that happens to equal that
bit. It can also be used to fix typos or replace unusual syntax with more
standard syntax, without breaking the interface. If a language cannot deal
with this it can use only the first name for a given value.

The condition "one uses bitwise operations" is up to interpretation and
> so is not a clear restriction. Do you want to actually pose any
> restrictions, e.g. on the possible named values?
>

There is a "minimum" set of bitwise operations that must be supported. I
believe these are:

* Ability to "or" any two values of the enum and produce a valid value of
the enum.

* Ability to test any value of the enum against a compile-time constant
enum value that is a power of 2, to see if a particular bit is on.

That does not mean a language binding *can't* implement other operations on
the bitfield. The language binding is free to implement anything it thinks
makes sense (turning bits off, flipping them, testing against non power of
2 including all/any variations, testing against variables, etc).


> Or would you have restrictions on the values if the enum is not defined
> as a bitfield? E.g. would you deny bitwise operations on non-bitfields?
>

No. A language binding is free to implement any operations it wants on the
enums. It can support binary operations on non-bitfields. The main reason
for the bitfield indicator is because supporting bitfields is often
difficult and results in cryptic error messages, and thus language bindings
would like to know when they don't have to bother doing this. The fact that
it can produce some compile-time errors for misuse in some language
bindings is an added bonus.

Similarly it seems like an "open" indicator that can be added to enums
(including bitfields) would help, as it sounds like some languages make it
difficult or cryptic to allow casting of arbitrary numbers to the enum
value. The "open" flag would be an indication that the language binding
needs to support this. The lack of it does *not* mean that a language
binding must prevent casting of arbitrary numbers to the enum.

All this syntax *never* says "you can't do something" to a language
binding. It is more of an indicator that something is allowed. For instance
it sounds like none of this syntax is going to stop the C binding from
allowing any constant or any raw number from being passed to as an enum
value. That is ok because the syntax never says "you can't do that".

Yes. Making dotted optional has the possibility to reduce renaming
> changes when bumping an experimental protocol as per the recent
> suggestions, or when stabilizing it.
>
> Keeping it optional has some pros, and no cons as I can see.
>

That seems right, and it matches the current patches. I think a version
that does not support cross-interface enums can be done first (with the
proposed patches) and then this syntax (supporting dots in the enum names)
added as another patch.

I feel like it is acceptable to limit the enums to being in the same xml
file (plus any includes it does) for now. Does anybody know of any
counter-examples (for instance desktop shells using enums from wl_shell).
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Victor Berger]

Le 2015-10-05 21:53, Nils Chr. Brause a écrit :

Bill Spitzak wrote:

Similarly it seems like an "open" indicator that can be added to enums
(including bitfields) would help, as it sounds like some languages 
make it
difficult or cryptic to allow casting of arbitrary numbers to the enum 
value.
The "open" flag > would be an indication that the language binding 
needs to
support this. The lack of it does *not* mean that a language binding 
must

prevent casting of arbitrary numbers to the enum.


Since a language binding should support unknown values anyway, I don't 
quite

see the benefit of an 'open' flag.


It depends. In some case, it could be valid to support unknown values by 
simply ignoring them.


Consider a program written against a certain version of the protocol. 
Then, it is used on a compositor with a newer version of the protocol, 
where one of the enums have one more possible value.


What can the client do if it receives this value ? It has no possible 
way to give it a proper meaning, so it can either ignore it, or trigger 
a fatal error. Are there other options ?


In this case, for a "closed" enum, the binding can decide to ignore any 
value received on the wire that is not described in the XML, as anyway, 
the client would not know what to do with it (and if they knew, they 
should be using a more recent version of the binding).


This can be useful in some languages to allow such a thing. For example 
in Rust, handling an "open" enum would require some overhead in terms of 
usability of the API, to gracefully handle unknown values. While simply 
ignoring them would lead to a much simpler design. Unless it is not a 
good idea to ignore unkown values, in which case I'd gladly read an 
explanation of why it is and which problems I missed.


Without this flag, I have to choices: either consider enums as 
uints/ints and a set of constants to compare against (like C), of put 
this overhead on every single enum. To be honest, none of these 
possibilities feel satisfying. But They are still possible outcomes, 
hence the bindings _can_ live without this indication.


There is a similar concern with the enum/bitfield duality: in Rust, enum 
type are just a set of possible values, and are not meant for bitwise 
operations. On the other hand there is way to define bitfiealds types, 
with all the fancy methods you like to have to manipulate bitfields. So 
having the information of which enum is a bitfield would allow me to 
make the generated API much more comfortable to use.


I hope this was a clear explanation of why I think these values can be 
useful. Without then, I cannot really make use of the new attribute 
linking an argument to its enum, apart from writing it in the 
documentation and using (u)ints everywhere.


___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

I see. You're right. :)

On 5 Oct 2015 22:31, "Victor Berger"  wrote:
>
> Le 2015-10-05 21:53, Nils Chr. Brause a écrit :
>>
>> Bill Spitzak wrote:
>>>
>>> Similarly it seems like an "open" indicator that can be added to enums
>>> (including bitfields) would help, as it sounds like some languages make
it
>>> difficult or cryptic to allow casting of arbitrary numbers to the enum
value.
>>> The "open" flag > would be an indication that the language binding
needs to
>>> support this. The lack of it does *not* mean that a language binding
must
>>> prevent casting of arbitrary numbers to the enum.
>>
>>
>> Since a language binding should support unknown values anyway, I don't
quite
>> see the benefit of an 'open' flag.
>
>
> It depends. In some case, it could be valid to support unknown values by
simply ignoring them.
>
> Consider a program written against a certain version of the protocol.
Then, it is used on a compositor with a newer version of the protocol,
where one of the enums have one more possible value.
>
> What can the client do if it receives this value ? It has no possible way
to give it a proper meaning, so it can either ignore it, or trigger a fatal
error. Are there other options ?
>
> In this case, for a "closed" enum, the binding can decide to ignore any
value received on the wire that is not described in the XML, as anyway, the
client would not know what to do with it (and if they knew, they should be
using a more recent version of the binding).
>
> This can be useful in some languages to allow such a thing. For example
in Rust, handling an "open" enum would require some overhead in terms of
usability of the API, to gracefully handle unknown values. While simply
ignoring them would lead to a much simpler design. Unless it is not a good
idea to ignore unkown values, in which case I'd gladly read an explanation
of why it is and which problems I missed.
>
> Without this flag, I have to choices: either consider enums as uints/ints
and a set of constants to compare against (like C), of put this overhead on
every single enum. To be honest, none of these possibilities feel
satisfying. But They are still possible outcomes, hence the bindings _can_
live without this indication.
>
> There is a similar concern with the enum/bitfield duality: in Rust, enum
type are just a set of possible values, and are not meant for bitwise
operations. On the other hand there is way to define bitfiealds types, with
all the fancy methods you like to have to manipulate bitfields. So having
the information of which enum is a bitfield would allow me to make the
generated API much more comfortable to use.
>
> I hope this was a clear explanation of why I think these values can be
useful. Without then, I cannot really make use of the new attribute linking
an argument to its enum, apart from writing it in the documentation and
using (u)ints everywhere.
>
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

Hi,

Auke Booij wrote:
> I think that we have an agreement in principle, just not in terms (and
> this also goes back to my comment to Victor Berger). What I meant to
> express is that any bindings that violate this rule are on the risk of
> the bindings writers, rather than the wayland and wayland protocol
> developers.
>
> Would you agree with the following? Otherwise please suggest a
> phrasing you would agree with.
>
> [start]
> The purpose of the enum and bitfield attributes is to document what
> arguments refer to which enums, and to document which numeric enum
> values are primarily accessed using bitwise operations.
> Additionally, the enum and bitfield attributes may be used by other
> code, such as bindings to other languages, for example to enhance type
> safety of code. However, such usage is only supported if the following
> property is satisfied: code written prior to the specification of
> these attributes still works after their specification. In other
> words, specifying an attribute for an argument, that previously did
> not have an enum or bitfield attribute, should not break API. Code
> that does not satisfy this rule is not guaranteed to obey backwards
> compatibility.
> [end]

Personally, I'm fine with that. :)

Pekka Paalanen wrote:
> For me, that depends only on what requirements a "bitfield" has. Does
> it have to be uint? Do the values have to be POT? That's all up to you,
> as it has no effect on C bindings nor generated documentation other
> than wording.
>
> If you agree on the restrictions and requirements, then we can see what
> it applies to.
>
> The condition "one uses bitwise operations" is up to interpretation and
> so is not a clear restriction. Do you want to actually pose any
> restrictions, e.g. on the possible named values?
>
> Or would you have restrictions on the values if the enum is not defined
> as a bitfield? E.g. would you deny bitwise operations on non-bitfields?

A bitfield doesn't have to be an unsigned integer and it doesn't have to be a
POT either. It can be anything that holds a series of bits, as a bitfield
doesn't have a numerical meaning.
In my opinion it's also not relevant whether an enumeration is signed or
unsigned or a POT or NPOT. Just think of it as a mapping from a particual
bit-pattern (which it is on the wire) to a meaning.

(Maybe this view comes from the years of working with VHDL, where everything
is just a bit pattern and it's up to the user to put meaning into it. C is very
close to that, though, as you can cast whatever you want)

Pekka Paalanen wrote:
> What is a bitfield and how does it differ from non-bitfield enum?

In a bitfield, every single bit (except the unused ones) has a distinct
meaning (see my format example). In an enum, that is not the case.

Bill Spitzak wrote:
> I also feel that if, after careful examination, it is determined that no
> working program would ever try to send or interpret a negative value, that
> the use of int instead of uint be defined as an error in the xml file, and
> fixed without changing the protocol version.

See above, the signedness of an enumeration doesn't really matter. ;)

Bill Spitzak wrote:
> Similarly it seems like an "open" indicator that can be added to enums
> (including bitfields) would help, as it sounds like some languages make it
> difficult or cryptic to allow casting of arbitrary numbers to the enum value.
> The "open" flag > would be an indication that the language binding needs to
> support this. The lack of it does *not* mean that a language binding must
> prevent casting of arbitrary numbers to the enum.

Since a language binding should support unknown values anyway, I don't quite
see the benefit of an 'open' flag.

Bill Spitzak wrote:
> I feel like it is acceptable to limit the enums to being in the same xml file
> (plus any includes it does) for now. Does anybody know of any
> counter-examples (for instance desktop shells using enums from wl_shell).

Do you mean, you don't want to mark enums if they are defined in another XML
file? I would prefer having marked every enum properly from the beginning.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

Re-adding the CCs: Jeroen, Jason, Boyan.

Jeroen is the one who started the April 2015 thread on this.

On Sun, 4 Oct 2015 22:15:12 +0100
Auke Booij  wrote:

> This email contains a clear suggestions that I'd like everyone to
> read, not just Nils and Erik. But Nils and Erik brought up a few
> points that I'd like to have settled, since they have been brought up
> quite frequently now.

As Erik wrote on On Sat, 3 Oct 2015 21:05:24 +0200: "... a wayland
'open' enum is not really an enum but a set of pre-defined constants."

This is true and we are trying to retrofit additional semantics to the
definition.

> On 3 October 2015 at 11:14, Nils Chr. Brause  wrote:
> > Pekka Paalanen wrote:
> >> Later that thread also brought up two incompatible definitions of a
> >> "bitfield":
> >> - a type where bit-wise logic operations make sense
> >> - a type where all listed values are powers of two (POT), and bit-wise
> >>   logic operations make sense
> >>
> >> As a practical example, is wl_output::transform a bitfield or an enum?
> >> - it is used as an int, not uint
> >> - it lists all possible values
> >> - the listed values are not all POT
> >> - bit-wise operations make sense (check for flipped)
> >>
> >> Arguably it should have been an uint, but I don't think we can change
> >> it now.
> >
> > I have a rather pragmatic view at this: If it looks and feels like a
> > bitfield, it is a bitfield.
> > And when looking an the definition, it for all the wold looks like a
> > bitfield to me.
> > Bit 0 means 90°, bit 1 means 180° and bit 3 means flipped.
> > The signedness doesn't matter for a bitfield, since it is just a
> > collection of bits without a numerical meaning.
> 
> So perhaps we should say:
> 
> "An enum can be specified to be a bitfield if the primary interface to
> its numeric value deals with bitwise operations."
> 
> Is that agreeable?

For me, that depends only on what requirements a "bitfield" has. Does
it have to be uint? Do the values have to be POT? That's all up to you,
as it has no effect on C bindings nor generated documentation other
than wording.

If you agree on the restrictions and requirements, then we can see what
it applies to.

The condition "one uses bitwise operations" is up to interpretation and
so is not a clear restriction. Do you want to actually pose any
restrictions, e.g. on the possible named values?

Or would you have restrictions on the values if the enum is not defined
as a bitfield? E.g. would you deny bitwise operations on non-bitfields?

I'd also be wary of using definitions where one is not a superset of
the other. In that case there would not be a "safe default" to pick for
a protocol XML writer, and it might be impossible to fix it after the
interface has become stable.

For example, I would assume that definitions like "only bitfield values
can use bitwise ops, and only non-bitfield values can be NPOT" will
lead to artificial problems.

What is a bitfield and how does it differ from non-bitfield enum?

> > Pekka Paalanen wrote:
> >> yeah, adding the interface name makes perfect sense. It could be
> >> optional if wanted. No dot would mean the enum is in the same
> >> interface, a dot would signify a specific interface. I don't think
> >> there is any use for an anonymous global namespace like Bill mentioned.
> >
> > Sounds good to me. Although I think it would be easier to implement,
> > if we'd just use the dot notation everywhere. This would result in
> > less code and (statistically) less bugs.
> 
> It is the "dotless" version that is already supported by my patches.
> The dotted one requires new code. So apart from a single if statement,
> this is not really more work. But I guess this is getting into
> bikeshedding terrain.

Yes. Making dotted optional has the possiblity to reduce renaming
changes when bumping an experimental protocol as per the recent
suggestions, or when stabilizing it.

Keeping it optional has some pros, and no cons as I can see.

> > Auke Booij wrote:
> >> The enum and bitfield attributes are in principle for documentation
> >> purposes only.
> >
> > No. The whole point of the enum an bitfield attributes is to help
> > non-C languages to generate type-safe bindings.
> >
> > Auke Booij wrote:
> >> The enum and bitfield attributes may also be used by
> >> bindings, but only in such a way that code written prior to the
> >> specification of these attributes still works after their
> >> specification.
> >
> > I think every language binding should ultimately decide for
> > themselves, how they are using the new attributes. There is no way you
> > can dictate anyone how to use them.
> 
> I think that we have an agreement in principle, just not in terms (and
> this also goes back to my comment to Victor Berger). What I meant to
> express is that any bindings that violate this rule are on the risk of
> the bindings writers, rather than the wayland and wayland protocol
> developers.

Yes, this nit-picking on 

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

Hi,

On Sat, Oct 3, 2015 at 9:05 PM, Erik De Rijcke  wrote:
> - Defining an uint (bitfield) or int (single enum) in the xml for an enum
> argument is the wrong approach. It prescribes, not describes. In some
> languages eg Java, there is no such thing as an uint. What is important, is
> to know how many enum values you are allowed to pass in an enum argument.
> How this translates low level is an implementation detail (wire format). We
> don't define opcodes either. The wire format in then end, is implicitly
> defined by the C generator based on the argument type in the xml (for an
> enum arg, some cases use an array of shorts, some cases use a int
> bitfield.). It would be better to define that relationship.

While I totally agree here, I highly doubt that this will be changed.

> General concerns as a language binding dev:
>
> - The definition of an enum, at least in Java (and I assume in a lot of
> other languages too), is a fixed set of values that are know at compile
> time. This set is unmodifiable at runtime. This means no new values of an
> enum type can be (safely) added/defined at runtime. It seems to me this
> definition is ultimately incompatible with wayland enums for several
> reasons.
>
> First reason is the case of an 'open' enum, as seen in xdg-shell state. From
> my understanding, a client or server should be able to act on any 'open'
> enum value it receives over the wire, even if this value was not known at
> compile time. As such a generated binding can not simply map all unknown
> wire values to the same "UNKNOWN" language specific enum type as the raw
> value would be lost in such case. It also makes it impossible to send out
> any undefined raw value. An 'open' enum is thus impossible to implement
> using a (language specific) enum as type. The core reason being that a
> wayland 'open' enum is not really an enum but a set of pre-defined
> constants.
>
> The second reason is the case of the 'closed' enum, as seen in all other
> enum definitions. A 'closed' wayland enum does allow to be directly mapped
> to a language specific enum type. However there is still the case of enum
> values known only by the other side (=different protocol versions). Again
> the solution(s) for this are the same as the first reason. Map to an
> 'UNKNOWN" value and have your raw value lost, or pass on the raw value but
> loose your language enum type usage.
>
> Both concerns seem to have no relevance to wayland per se, and should simply
> be solved by the language binding. I believe this not to be the case as
> there is the open question (for me) on what should be considered as
> acceptable behaviour when dealing with wayland enums in language bindings.
> If it is acceptable to have your raw value lost for a 'closed' enum, but not
> for an open value, then the language binding must be able to deal with that
> appropriately. However for that it most know in some way when an enum is
> open, and when it is closed (either by implicit ruling, eg it's an array
> thus open, or explicitly through an xml attribute).

In C++ it is quite easy to deal with open enumerations and unknown
enumeration values. The 'enum class' in C++11 can be converted to and
from its underlying type (integer), so that the user still can emit
and receive values unknown to the bindings. Other languages would
probably need to come up with a similar mechanism.

Cheers,
Nils
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Auke Booij]

This email contains a clear suggestions that I'd like everyone to
read, not just Nils and Erik. But Nils and Erik brought up a few
points that I'd like to have settled, since they have been brought up
quite frequently now.

On 3 October 2015 at 11:14, Nils Chr. Brause  wrote:
> Pekka Paalanen wrote:
>> Later that thread also brought up two incompatible definitions of a
>> "bitfield":
>> - a type where bit-wise logic operations make sense
>> - a type where all listed values are powers of two (POT), and bit-wise
>>   logic operations make sense
>>
>> As a practical example, is wl_output::transform a bitfield or an enum?
>> - it is used as an int, not uint
>> - it lists all possible values
>> - the listed values are not all POT
>> - bit-wise operations make sense (check for flipped)
>>
>> Arguably it should have been an uint, but I don't think we can change
>> it now.
>
> I have a rather pragmatic view at this: If it looks and feels like a
> bitfield, it is a bitfield.
> And when looking an the definition, it for all the wold looks like a
> bitfield to me.
> Bit 0 means 90°, bit 1 means 180° and bit 3 means flipped.
> The signedness doesn't matter for a bitfield, since it is just a
> collection of bits without a numerical meaning.

So perhaps we should say:

"An enum can be specified to be a bitfield if the primary interface to
its numeric value deals with bitwise operations."

Is that agreeable?

> Pekka Paalanen wrote:
>> yeah, adding the interface name makes perfect sense. It could be
>> optional if wanted. No dot would mean the enum is in the same
>> interface, a dot would signify a specific interface. I don't think
>> there is any use for an anonymous global namespace like Bill mentioned.
>
> Sounds good to me. Although I think it would be easier to implement,
> if we'd just use the dot notation everywhere. This would result in
> less code and (statistically) less bugs.

It is the "dotless" version that is already supported by my patches.
The dotted one requires new code. So apart from a single if statement,
this is not really more work. But I guess this is getting into
bikeshedding terrain.

> Auke Booij wrote:
>> The enum and bitfield attributes are in principle for documentation
>> purposes only.
>
> No. The whole point of the enum an bitfield attributes is to help
> non-C languages to generate type-safe bindings.
>
> Auke Booij wrote:
>> The enum and bitfield attributes may also be used by
>> bindings, but only in such a way that code written prior to the
>> specification of these attributes still works after their
>> specification.
>
> I think every language binding should ultimately decide for
> themselves, how they are using the new attributes. There is no way you
> can dictate anyone how to use them.

I think that we have an agreement in principle, just not in terms (and
this also goes back to my comment to Victor Berger). What I meant to
express is that any bindings that violate this rule are on the risk of
the bindings writers, rather than the wayland and wayland protocol
developers.

Would you agree with the following? Otherwise please suggest a
phrasing you would agree with.

[start]
The purpose of the enum and bitfield attributes is to document what
arguments refer to which enums, and to document which numeric enum
values are primarily accessed using bitwise operations.
Additionally, the enum and bitfield attributes may be used by other
code, such as bindings to other languages, for example to enhance type
safety of code. However, such usage is only supported if the following
property is satisfied: code written prior to the specification of
these attributes still works after their specification. In other
words, specifying an attribute for an argument, that previously did
not have an enum or bitfield attribute, should not break API. Code
that does not satisfy this rule is not guaranteed to obey backwards
compatibility.
[end]

> Auke Booij wrote:
>> I agree with your concern. However, note that "for documentation
>> purposes" does not mean that it is just a documentation string: we
>> document a very clear and precise fact, for the purpose of
>> understanding the API (ie documentation). My previous patches indeed
>> do some checking wrt enum name cross-matching. This would have to be
>> extended for enum names that refer to a different interface.
>
> You shouldn't write "for documentation purposes only" then, as it is
> clearly not. It is mainly for the type-safety of non-C
> language-bindings.

Please do comment on the above, as I understand your point, but also
want to be pragmatic here.

On 3 October 2015 at 20:05, Erik De Rijcke  wrote:
> - Defining an uint (bitfield) or int (single enum) in the xml for an enum
> argument is the wrong approach. It prescribes, not describes. In some
> languages eg Java, there is no such thing as an uint. What is important, is
> to know how many enum values you are allowed to pass in an enum argument.
> How 

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

Hi,

Pekka Paalanen wrote:
> Adding these attributes to XML must not change the signatures generated
> by wayland-scanner.
>
> Why: these are also used in C++, which does not implicitly covert
> between enums and other stuff, which would break the build.
> http://lists.freedesktop.org/archives/wayland-devel/2014-September/017057.html

There are two different types of enumerations in C++11. The C-style
'enum', which behaves in C++ just like it did in C and the modern
'enum class', which makes each enum a distinct type. The
wayland-scanner generates C code and therefore C-style enums. In C++
they are still nothing else than integers (just like in C). Therefore
they don't change any signatures.

Pekka Paalanen wrote:
> Later that thread also brought up two incompatible definitions of a
> "bitfield":
> - a type where bit-wise logic operations make sense
> - a type where all listed values are powers of two (POT), and bit-wise
>   logic operations make sense
>
> As a practical example, is wl_output::transform a bitfield or an enum?
> - it is used as an int, not uint
> - it lists all possible values
> - the listed values are not all POT
> - bit-wise operations make sense (check for flipped)
>
> Arguably it should have been an uint, but I don't think we can change
> it now.

I have a rather pragmatic view at this: If it looks and feels like a
bitfield, it is a bitfield.
And when looking an the definition, it for all the wold looks like a
bitfield to me.
Bit 0 means 90°, bit 1 means 180° and bit 3 means flipped.
The signedness doesn't matter for a bitfield, since it is just a
collection of bits without a numerical meaning.

Pekka Paalanen wrote:
> There is a paradox: if the new attributes affect codegen for some
> languages in a way that will break the build (which is exactly the
> reason why they are wanted: to enforce type-safety), we cannot add
> these attributes to the existing protocols, most importantly
> wayland.xml, because it would "break existing code".

If the correct types have already been used before, the changes
shouldn't actually break any code. If they do, that means that the
code was invalid in the first place (e.g. enum X used where enum Y is
required orr ORing together non-bitfield values).

Pekka Paalanen wrote:
> Should we make any stability guarantees towards non-C bindings
> generators that exceed what we need for C?

It should at least be guaranteed, the the enum and bitfield attributes
are correct. And that can ultimatly only done by a human. The scanner
might be able to detect if a specified enum actually exists, but only
a human can tell if it is the correct one.

Pekka Paalanen wrote:
> yeah, adding the interface name makes perfect sense. It could be
> optional if wanted. No dot would mean the enum is in the same
> interface, a dot would signify a specific interface. I don't think
> there is any use for an anonymous global namespace like Bill mentioned.

Sounds good to me. Although I think it would be easier to implement,
if we'd just use the dot notation everywhere. This would result in
less code and (statistically) less bugs.

Auke Booij wrote:
> The enum and bitfield attributes are in principle for documentation
> purposes only.

No. The whole point of the enum an bitfield attributes is to help
non-C languages to generate type-safe bindings.

Auke Booij wrote:
> The enum and bitfield attributes may also be used by
> bindings, but only in such a way that code written prior to the
> specification of these attributes still works after their
> specification.

I think every language binding should ultimately decide for
themselves, how they are using the new attributes. There is no way you
can dictate anyone how to use them.

Pekka Paalanen wrote:
> We could have a "strict mode" to wayland-scanner, where you need to
> feed all relevant additional XML files also, and the check would be
> cross-file then, erroring on all unknown links. Then we'd just change
> our builds to use this mode.

This would be very useful to to detect misspellings in interface names.

Bryce Harrington wrote:
> Sounds like might be something worth adding to distcheck.

Absolutely!

Auke Booij wrote:
> I agree with your concern. However, note that "for documentation
> purposes" does not mean that it is just a documentation string: we
> document a very clear and precise fact, for the purpose of
> understanding the API (ie documentation). My previous patches indeed
> do some checking wrt enum name cross-matching. This would have to be
> extended for enum names that refer to a different interface.

You shouldn't write "for documentation purposes only" then, as it is
clearly not. It is mainly for the type-safety of non-C
language-bindings.

Cheers,
Nils
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Erik De Rijcke]

The enum discussion seems to become a lengthy one and it's becoming hard to
see the forest for the trees.

I'll give my input as a Java bindings dev. Please correct any bad
assumptions or observations I've made.

>From what I see in Auke's patches:

- Defining an uint (bitfield) or int (single enum) in the xml for an enum
argument is the wrong approach. It prescribes, not describes. In some
languages eg Java, there is no such thing as an uint. What is important, is
to know how many enum values you are allowed to pass in an enum argument.
How this translates low level is an implementation detail (wire format). We
don't define opcodes either. The wire format in then end, is implicitly
defined by the C generator based on the argument type in the xml (for an
enum arg, some cases use an array of shorts, some cases use a int
bitfield.). It would be better to define that relationship.

General concerns as a language binding dev:

- The definition of an enum, at least in Java (and I assume in a lot of
other languages too), is a fixed set of values that are know at compile
time. This set is unmodifiable at runtime. This means no new values of an
enum type can be (safely) added/defined at runtime. It seems to me this
definition is ultimately incompatible with wayland enums for several
reasons.

First reason is the case of an 'open' enum, as seen in xdg-shell state.
>From my understanding, a client or server should be able to act on any
'open' enum value it receives over the wire, even if this value was not
known at compile time. As such a generated binding can not simply map all
unknown wire values to the same "UNKNOWN" language specific enum type as
the raw value would be lost in such case. It also makes it impossible to
send out any undefined raw value. An 'open' enum is thus impossible to
implement using a (language specific) enum as type. The core reason being
that a wayland 'open' enum is not really an enum but a set of pre-defined
constants.

The second reason is the case of the 'closed' enum, as seen in all other
enum definitions. A 'closed' wayland enum does allow to be directly mapped
to a language specific enum type. However there is still the case of enum
values known only by the other side (=different protocol versions). Again
the solution(s) for this are the same as the first reason. Map to an
'UNKNOWN" value and have your raw value lost, or pass on the raw value but
loose your language enum type usage.

Both concerns seem to have no relevance to wayland per se, and should
simply be solved by the language binding. I believe this not to be the case
as there is the open question (for me) on what should be considered as
acceptable behaviour when dealing with wayland enums in language bindings.
If it is acceptable to have your raw value lost for a 'closed' enum, but
not for an open value, then the language binding must be able to deal with
that appropriately. However for that it most know in some way when an enum
is open, and when it is closed (either by implicit ruling, eg it's an array
thus open, or explicitly through an xml attribute).

amen

On Thu, Oct 1, 2015 at 7:49 PM, Bryce Harrington 
wrote:

> The topic of adding better enum/bitfield support to the protocol has
> come up a few[0] times[1] in the past, and again more recently[2].  We
> also have several proposed patches in patchwork, but I'm not sure they
> reflect consensus and none have Reviewed-by's on them yet [3,4,5,6,7].
>
> From what I gather of the discussions, no one is really against this
> feature conceptually, and impementationally the discussions appear to
> have moved further afield.  It feels like we're real close to having
> something that could be landed, but it's not 100% clear to me what
> exactly to land.  Since it's a protocol types change I would prefer to
> make sure it has a strong consensus before landing it.
>
> I know that several people have proposed patches on this - Bill, Nils
> and Auke at least.  Since there's a definite need for this, and since
> agreement appears to be not far off, I would like to get this landed
> this release.  And ideally I'd like to get this landed early in the
> release so we give plenty of time for testing.
>
> Since Auke's patchset proposalis the most recent, let's take that one as
> the candidate for landing.  Gentlemen, I'd like to ask you to review
> these three patches [5,6,7] and either give your Reviewed-by's or flag
> specific improvements needed.  If you have a more conceptual
> disagreement, and don't think the patchset is landable as implemented,
> please raise that issue asap too.
>
> Bryce
>
> 0:
> http://lists.freedesktop.org/archives/wayland-devel/2015-April/021438.html
> 1:
> http://lists.freedesktop.org/archives/wayland-devel/2015-June/023008.html
> 2:
> http://lists.freedesktop.org/archives/wayland-devel/2015-September/024249.html
>
> 3: http://patchwork.freedesktop.org/patch/47726/
> 4: http://patchwork.freedesktop.org/patch/47727/
> 5: 

Re: Enums, bitfields and wl_arrays

[Victor Berger]

Le 2015-10-02 15:16, Pekka Paalanen a écrit :

On Fri, 2 Oct 2015 13:50:42 +0100
Auke Booij  wrote:


[start]
The enum and bitfield attributes are in principle for documentation
purposes only. The enum and bitfield attributes may also be used by
bindings, but only in such a way that code written prior to the
specification of these attributes still works after their
specification. In other words, specifying an attribute for an
argument, that previously did not have it, should not break API.
[end]


I like this very much. Let's see if anyone disagrees.

Do you intend to allow also changing rather than only adding these new
attributes in the wording above?


While I don't disagree, I have a small concern:

Do we agree that this involves at some point writing a specification of 
the format of the XML files?


Because otherwise, if the XML format remains defined by the 
implementation of the C scanner, and that these attributes are 
explicitly defined as for documentation only and ignored by the C 
scanner, this means the XML writers would be allowed to write any 
garbage they want as a value for these fields.


As a consequence, bindings writers cannot give any value to these 
fields, and they become basically useless. I mean, it seems logical that 
the value of the "enum" field should be something like "enum_name" or 
"interface_name.enum_name", or whatever format will be chosen. But these 
fields have practically no value if we cannot expect this format to be 
respected (documentation-only fields can also simply be written in the 
"description" field, if they are not used anywhere).


--
Victor

___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Victor Berger]

Le 2015-10-02 14:12, Auke Booij a écrit :

However, I'm not sure who you are trying to protect here. Everyone
agrees that the new attributes should not change anything for C/C++,
and in the current patches, they don't. And the other bindings writers
understand the compatibility issues regarding this change, and, if I
may speak for the majority of them, care much less about compatibility
issues than about being able to provide proper APIs for their users.

In haskell, and many other modern languages, an easily fixable compile
issue (we keep calling it a compatibility issue, which I think is a
misnomer, although I don't know what category it should fall under) is
*vastly* preferred over a potential bug. Arguably, that's the entire
point of the language. Modern languages attempt to cross-check as many
properties as possible, and the enum attribute would contribute
greatly to this, even if that means breaking API every so often.

The wayland protocol currently does not specify the enum attribute,
and I see no way how to write an API whose entire purpose is to
*break* when you erroneously mix up enum attribute data, without
breaking API as this data is added. More importantly, this problem
does not matter because it is not a problem but a *solution*.


I mostly agree with that, from my Rust point of view.

While a huge part of my bindings can be generated by the scanner, these 
is still a need of some hand-written glue, to cleanly connect Rust into 
the wayland-client lib via the protocol. And given the current package 
model of Rust, I expect each protocol file to be interfaced as a rust 
package including a given version of the xml file and using the scanner 
to generate it at compile time. With all these packages (the scanner 
being one of them) properly versioned, nobody would be take by surprise 
by and update of the protocol file.


So, my main argument for Rust is that the mental overhead of binding 
generation would not be handled by the downstream consumer, but by the 
bindings maintainers. If some protocol requires an old version of the 
scanner, so be it, it just needs to set its dependencies accordingly and 
everything will be fine.


These were my two cents about Rust, I cannot talk for other languages.

Victor

___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

On Fri, 2 Oct 2015 13:50:42 +0100
Auke Booij  wrote:

> Maybe a solution can be found in more languages, and we can make the
> compatibility, that e.g. Pekka is looking for, a requirement for
> bindings, rather than make compatibility a requirement for the
> protocol writers. So something along these lines would be in the
> specification:
> 
> [start]
> The enum and bitfield attributes are in principle for documentation
> purposes only. The enum and bitfield attributes may also be used by
> bindings, but only in such a way that code written prior to the
> specification of these attributes still works after their
> specification. In other words, specifying an attribute for an
> argument, that previously did not have it, should not break API.
> [end]
> 
> Obviously C is not rich enough to do this in an elegant way. (Maybe
> it's possible in C++ with some template magic?) But it definitely
> solves the "backwards compatibility" debate, since, while anyone is
> free to ignore this bit in the specification, solutions might
> sometimes be possible, and it guards the entire C world from issues in
> the non-C bindings.

Hi Auke,

I like this very much. Let's see if anyone disagrees.

Do you intend to allow also changing rather than only adding these new
attributes in the wording above?

I read it as no, and I suppose that's good. We'd have that anyway. The
"do not use this or you get to keep all pieces" wording I used was
perhaps too liberal.


Thanks,
pq


pgpE58KqXWES1.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

On Fri, 02 Oct 2015 15:49:15 +0200
Victor Berger  wrote:

> Le 2015-10-02 15:16, Pekka Paalanen a écrit :
> > On Fri, 2 Oct 2015 13:50:42 +0100
> > Auke Booij  wrote:
> >> 
> >> [start]
> >> The enum and bitfield attributes are in principle for documentation
> >> purposes only. The enum and bitfield attributes may also be used by
> >> bindings, but only in such a way that code written prior to the
> >> specification of these attributes still works after their
> >> specification. In other words, specifying an attribute for an
> >> argument, that previously did not have it, should not break API.
> >> [end]
> > 
> > I like this very much. Let's see if anyone disagrees.
> > 
> > Do you intend to allow also changing rather than only adding these new
> > attributes in the wording above?
> 
> While I don't disagree, I have a small concern:
> 
> Do we agree that this involves at some point writing a specification of 
> the format of the XML files?

We have to write the new bits now. Writing the whole spec is not a
blocker for this, IMO.

> Because otherwise, if the XML format remains defined by the 
> implementation of the C scanner, and that these attributes are 
> explicitly defined as for documentation only and ignored by the C 
> scanner, this means the XML writers would be allowed to write any 
> garbage they want as a value for these fields.

I'd expect we'd have wayland-scanner check the linkage, that is, that
the enum exists unless it's from an interface not defined in this XML
file.

So, garbage with an interface name prefix would get through.

We could have a "strict mode" to wayland-scanner, where you need to
feed all relevant additional XML files also, and the check would be
cross-file then, erroring on all unknown links. Then we'd just change
our builds to use this mode.

> As a consequence, bindings writers cannot give any value to these 
> fields, and they become basically useless. I mean, it seems logical that 
> the value of the "enum" field should be something like "enum_name" or 
> "interface_name.enum_name", or whatever format will be chosen. But these 
> fields have practically no value if we cannot expect this format to be 
> respected (documentation-only fields can also simply be written in the 
> "description" field, if they are not used anywhere).

Haha. :-)

I think this was in my early objections to the whole idea of adding
these attributes in the first place. You'd only know you screwed up by
generating docs and seeing the reference links don't work.

Having the whole XML language specced out wouldn't fix this, because it
would still need a human to review the XML.


Thanks,
pq


pgp8KYpSCtQN7.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Auke Booij]

On 2 October 2015 at 13:12, Auke Booij  wrote:
> The wayland protocol currently does not specify the enum attribute,
> and I see no way how to write an API whose entire purpose is to
> *break* when you erroneously mix up enum attribute data, without
> breaking API as this data is added.

Actually, for my particular scenario, I just thought of a very elegant
solution that would introduce no new compatibility issues. With this
solution, *more* code would compile *with* the right enum attributes
than *without*. So we would have the best of both worlds: the type
safety that modern languages want, without new compatibility issues to
keep track of. (If anyone is interested in the technicalities behind
this, contact me. But it is a solution that is rather specific to
Haskell. Essentially, it would work by, rather than passing enum
values as ints until we have a proper data type for them, we would
pass each field as a different data type, until we know that two enum
fields should have been the same type. In haskell, this can be done in
a backwards-compatible way.)

Maybe a solution can be found in more languages, and we can make the
compatibility, that e.g. Pekka is looking for, a requirement for
bindings, rather than make compatibility a requirement for the
protocol writers. So something along these lines would be in the
specification:

[start]
The enum and bitfield attributes are in principle for documentation
purposes only. The enum and bitfield attributes may also be used by
bindings, but only in such a way that code written prior to the
specification of these attributes still works after their
specification. In other words, specifying an attribute for an
argument, that previously did not have it, should not break API.
[end]

Obviously C is not rich enough to do this in an elegant way. (Maybe
it's possible in C++ with some template magic?) But it definitely
solves the "backwards compatibility" debate, since, while anyone is
free to ignore this bit in the specification, solutions might
sometimes be possible, and it guards the entire C world from issues in
the non-C bindings.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

On Fri, 2 Oct 2015 12:53:13 +0100
Auke Booij  wrote:

> On 1 October 2015 at 20:00, Nils Chr. Brause  wrote:

> > I would prefer, if the enum attributes would also name the interface,
> > where the enum can be found, e.g.:
> > 
> > If two enums in different interfaces happen to have the same name (if
> > that's possible?), this would lead to ambiguities otherwise. Also a
> > scanner wouldn't have to look up the interface name that way.
> 
> While in principle I think this is a great idea, this will need a few
> specifications, which is why I decided not to add those in just yet.
> Are cross-XML references allowed in this sense? In that case, the
> scanner cannot verify their correctness, since only the current XML
> file is available to it. Additionally, moving a certain interface from
> xdg_shell to the core wayland protocol would now mean potentially
> having to weaken the type safety of an interface, or having to copy
> the enum over.

Hi,

yeah, adding the interface name makes perfect sense. It could be
optional if wanted. No dot would mean the enum is in the same
interface, a dot would signify a specific interface. I don't think
there is any use for an anonymous global namespace like Bill mentioned.

As for link validity check, I could live with checking it only if the
mentioned or implied interface is defined in the same XML file. This
should cover most cases, and the rest would manifest as compile errors
on non-C language bindings.

It's really all we can do anyway. XML files are perfectly allowed to
reference interfaces defined elsewhere, even in XML files not available
at the time.

A generator could have an extended check mode, where you can feed it a
bunch of XML files for reference, while generating the code just for
one, but I don't think it would be a good idea to absolutely require
that on the XML language spec level, because it would again break old
build systems.

Btw. when xdg-shell or any other new extension gets promoted to Wayland
as stable, it will not be appended into wayland.xml. It will simply be
yet another XML file to be installed by libwayland. Furthermore,
libwayland will not install pre-generated C-bindings for it. All
projects using the extension need to run wayland-scanner or any other
generator during their build.


Thanks,
pq


pgpWPb_c66KOe.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

On Fri, 02 Oct 2015 14:43:57 +0200
Victor Berger  wrote:

> Le 2015-10-02 14:12, Auke Booij a écrit :
> > However, I'm not sure who you are trying to protect here. Everyone
> > agrees that the new attributes should not change anything for C/C++,
> > and in the current patches, they don't. And the other bindings writers
> > understand the compatibility issues regarding this change, and, if I
> > may speak for the majority of them, care much less about compatibility
> > issues than about being able to provide proper APIs for their users.

I'm trying to protect *you* from people who write the XML files. But if
you say you do not need this kind of protection, and no other non-C
language bindings writer needs either, then we can simply proceed in
the way you wrote your patch set:

These new attributes are defined to be for documentation purposes only,
and we explicitly say that you should *not* use this information in
generating APIs. If you do so anyway, you have no right to complain if
your or a third person's code breaks.

This affects you and all your language's users. If they ever complain to
Wayland upstream about breaking things, we'll just laugh at their face
and point to the spec that says your generator is buggy to begin
with. ;-)

Well maybe not so rudely.

I want to be sure that you fully understand the implications here,
before we commit to this rule.

And it's not just you, it's everyone wanting to write a bindings
generator, e.g. for Java and C++.

> > In haskell, and many other modern languages, an easily fixable compile
> > issue (we keep calling it a compatibility issue, which I think is a
> > misnomer, although I don't know what category it should fall under) is
> > *vastly* preferred over a potential bug. Arguably, that's the entire
> > point of the language. Modern languages attempt to cross-check as many
> > properties as possible, and the enum attribute would contribute
> > greatly to this, even if that means breaking API every so often.
> > 
> > The wayland protocol currently does not specify the enum attribute,
> > and I see no way how to write an API whose entire purpose is to
> > *break* when you erroneously mix up enum attribute data, without
> > breaking API as this data is added. More importantly, this problem
> > does not matter because it is not a problem but a *solution*.
> 
> I mostly agree with that, from my Rust point of view.
> 
> While a huge part of my bindings can be generated by the scanner, these 
> is still a need of some hand-written glue, to cleanly connect Rust into 
> the wayland-client lib via the protocol. And given the current package 
> model of Rust, I expect each protocol file to be interfaced as a rust 
> package including a given version of the xml file and using the scanner 
> to generate it at compile time. With all these packages (the scanner 
> being one of them) properly versioned, nobody would be take by surprise 
> by and update of the protocol file.
> 
> So, my main argument for Rust is that the mental overhead of binding 
> generation would not be handled by the downstream consumer, but by the 
> bindings maintainers. If some protocol requires an old version of the 
> scanner, so be it, it just needs to set its dependencies accordingly and 
> everything will be fine.

This is a very good point. I understand you are covered, then.

I see Auke just posted another interesting email.

So we have thumbs up from Haskell and Rust for not needing any
additional XML stability guaranteed than what C does.

Did we have Java and C++ people around? Other languages?


Thanks,
pq


pgpHjib_l6XGn.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Auke Booij]

On 1 October 2015 at 20:00, Nils Chr. Brause  wrote:
>> Since Auke's patchset proposalis the most recent, let's take that one as
>> the candidate for landing.  Gentlemen, I'd like to ask you to review
>> these three patches [5,6,7] and either give your Reviewed-by's or flag
>> specific improvements needed.  If you have a more conceptual
>> disagreement, and don't think the patchset is landable as implemented,
>> please raise that issue asap too.
>
> There are some enum attributes missing, namely:
> - wl_shm_pool::create_buffer::format (it's wl_shm::format)
> - wl_shell_surface::set_fullscreen::method (it's
> wl_shell_surface::fullscreen_method)
> - wl_surface::set_buffer_transform::transform (it's wl_output::transform)

wl_shell_surface::set_fullscreen::method is a mistake by me. You are
right, that should have been in there. The reason I left out the other
two is because of what you write here:

> I would prefer, if the enum attributes would also name the interface,
> where the enum can be found, e.g.:
> 
> If two enums in different interfaces happen to have the same name (if
> that's possible?), this would lead to ambiguities otherwise. Also a
> scanner wouldn't have to look up the interface name that way.

While in principle I think this is a great idea, this will need a few
specifications, which is why I decided not to add those in just yet.
Are cross-XML references allowed in this sense? In that case, the
scanner cannot verify their correctness, since only the current XML
file is available to it. Additionally, moving a certain interface from
xdg_shell to the core wayland protocol would now mean potentially
having to weaken the type safety of an interface, or having to copy
the enum over.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays in the .xml file

[Pekka Paalanen]

On Fri, 25 Sep 2015 12:25:51 +0200
"Nils Chr. Brause"  wrote:

> Hi,
> 
> On Fri, Sep 25, 2015 at 10:52 AM, Victor Berger
>  wrote:
> >
> > What I meant here is that the format and contents of the XML files is
> > currently defined by the implementation of the C scanner, which is a less
> > than optimal situation to discuss evolutions of this format.
> >
> > There will most likely be a need to write a proper document describing the
> > format of the XML files, as well as the semantic meanings of each field and
> > attribute it contains.
> 
> Isn't the DTD file the specification for the XML file format?

No, it's not. It's an unmaintained relic waiting to be deleted. I
suppose it has remained as an informational reference.

It was once written for running the XML files through xmllint because
wayland-scanner rather crashed than complained about bad XML. The
xmllint phase was since removed on the basis that wayland-scanner is
our definition - if we need more errors, we should enhance
wayland-scanner.

See:
http://cgit.freedesktop.org/wayland/wayland/commit/?id=05f95c85c8cad07bee233f1d4e205a12538365e1
http://cgit.freedesktop.org/wayland/wayland/commit/?id=6292fe2af6a45decb7fd39090e74dd87bc4e22b2

The DTD has never specified the semantics, like Victor points out.


Thanks,
pq


pgpZusTHd90NH.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Auke Booij]

Thanks for bringing this up again, Bryce.

On 1 October 2015 at 18:59, Jasper St. Pierre  wrote:
> We have a few constraints. First off, not all enums are closed. Some
> are intentionally open, like xdg_shell.state. So we definitely need a
> distinction between a closed enum and an open enum. I'm not familiar
> enough with Rust to be able to apply something to that.

Yes, this is a point that was raised before (IIRC especially by
Pekka). This is why, in my patches, I document these attributes as
being for documentation purposes only, although obviously they might
be abused by some bindings. Yes, a closed enum could be encoded into
more strongly typed languages (in which case a compilable program only
specifies what it does on the "legal" values, and simply crashes or
errors out on the unknown ones - this would be justified by the broken
promise that the enum was closed). But I agree that this is a
guarantee you cannot always make, and will open up a debate far bigger
than the actual problems these patches set out to solve.

> There's also a compatibility issue that has been brought up, but I
> never understood that one. Somebody else would be able to talk about
> that better.

Can you quote what you are referring to here?
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Pekka Paalanen]

Hi all,

let's add Jeroen, Jason, Boyan to CC.

The fundamental requirement here is that we must not break existing
code or users. All arguments I have can be traced back to this simple
statement.

First, I would rather keep the wl_array discussion in a separate thread
and not mix it up here. What we do for wl_array does not affect enums
or bitfields, but the other way around maybe.


On Thu, 1 Oct 2015 10:49:02 -0700
Bryce Harrington  wrote:

> The topic of adding better enum/bitfield support to the protocol has
> come up a few[0] times[1] in the past, and again more recently[2].  We
> also have several proposed patches in patchwork, but I'm not sure they
> reflect consensus and none have Reviewed-by's on them yet [3,4,5,6,7].

This might have been first email about the topic:
http://lists.freedesktop.org/archives/wayland-devel/2014-August/016988.html
http://lists.freedesktop.org/archives/wayland-devel/2014-September/thread.html#16994

I suppose since then we have agreed at least on one thing:

Adding these attributes to XML must not change the signatures generated
by wayland-scanner.

Why: these are also used in C++, which does not implicitly covert
between enums and other stuff, which would break the build.
http://lists.freedesktop.org/archives/wayland-devel/2014-September/017057.html

> From what I gather of the discussions, no one is really against this
> feature conceptually, and impementationally the discussions appear to
> have moved further afield.  It feels like we're real close to having
> something that could be landed, but it's not 100% clear to me what
> exactly to land.  Since it's a protocol types change I would prefer to
> make sure it has a strong consensus before landing it.

In September 2014 iteration of the topic we (I, Jason, Boyan) basically
rejected the idea:
http://lists.freedesktop.org/archives/wayland-devel/2014-September/017457.html

Later that thread also brought up two incompatible definitions of a
"bitfield":
- a type where bit-wise logic operations make sense
- a type where all listed values are powers of two (POT), and bit-wise
  logic operations make sense

As a practical example, is wl_output::transform a bitfield or an enum?
- it is used as an int, not uint
- it lists all possible values
- the listed values are not all POT
- bit-wise operations make sense (check for flipped)

Arguably it should have been an uint, but I don't think we can change
it now.

> I know that several people have proposed patches on this - Bill, Nils
> and Auke at least.  Since there's a definite need for this, and since
> agreement appears to be not far off, I would like to get this landed
> this release.  And ideally I'd like to get this landed early in the
> release so we give plenty of time for testing.

The things that have been lacking in my opinion are specifications,
especially answering the questions listed here by Victor after our irc
discussion:
http://lists.freedesktop.org/archives/wayland-devel/2015-September/024382.html

The "do not break existing code" rule applies also proactively, not
just at the time we decide to start using the new attributes, so it is
important to define what things are under the "stable protocol"
definition and cannot change after publishing. IOW, what are we carving
in stone.

There is a paradox: if the new attributes affect codegen for some
languages in a way that will break the build (which is exactly the
reason why they are wanted: to enforce type-safety), we cannot add
these attributes to the existing protocols, most importantly
wayland.xml, because it would "break existing code". You might escape
that paradox once, by claiming that there is no "existing code", after
all the generators for these languages will be new. But this escape
hatch will only work once. When these generators get into use, we can
no longer add the attributes to stable protocols without breaking
things. Not even if the generators had a "legacy mode" that ignores the
new attributes, because they would then regress on arguments that
already had the new attributes set.

The important thing to realize is that once an XML file starts using
these new attributes, the attributes can no longer be added later into
more arguments. We need to get it right on the first go, but hey,
that's how extending already stabilized protocols goes.

A further complication here is that currently, the definition of the
XML language is literally "what wayland-scanner accepts", bugs and
oversights in wayland-scanner notwithstanding. This makes it tricky to
add XML language features that are not really used in the C bindings.

Or, would it be ok to dismiss all non-C languages as second-class
citizens, and say that we can change whatever as long as it does not
break the wire format, wayland-scanner or the bindings generated by it?
Somehow I don't think that would be an acceptable position.

This is actually a fundamental question:

Should we make any stability guarantees towards non-C 

Re: Enums, bitfields and wl_arrays in the .xml file

[Pekka Paalanen]

On Fri, 25 Sep 2015 12:43:55 +0200
Victor Berger  wrote:

> On 2015-09-25 12:25, Nils Chr. Brause wrote :
> > Hi,
> > 
> > On Fri, Sep 25, 2015 at 10:52 AM, Victor Berger
> >  wrote:

> >> The questions about how breaking evolutions will be handled need to be
> >> specified as well: how should an old scanner behave when it encounters 
> >> a
> >> more recent protocol file, containing fields or attributes it does not
> >> recognize ? Ignore them ? Fail and declare being "not compatible with 
> >> this
> >> protocol format, please upgrade" ?
> > 
> > The last one makes the most sense to me. Someone who develops a
> > language binding should always keep her/his scanner up to date with the
> > most recent XML file format.
> 
> Agreed...

This fails if someone is stuck with an old version of libwayland and
the scanner, e.g. due to policy or proprietary software reasons, and
wants to use a recent third party XML. They are not good reasons...

But since I believe that wayland-scanner already ignores unknown
attributes, there is no problem. We cannot retroactively change old
scanners anyway.


> > [...]
> > Also, from the discussion last year it emerged that the scanner should 
> > also be
> > modified to at least check for the validity of the new attribues.
> > You can find the work that I had done so far here:
> > https://github.com/NilsBrause/Wayland/commits/xml
> 
> ... but from the discussion I had with pq on IRC, it does not seem like 
> having the C-scanner start rejecting "old" protocol files is a welcomes 
> outcome, from they point of view.

Yes. Let's not break existing users, be that old XML files or old
scanners.

> However, maybe checking the file, and emitting warnings if it is in an 
> old format could be an option ?
> 
> The other bindings could explicitly refuse old formats, I don't think 
> it's a deal breaker either.

Did you just swap the scenario here? How can you even detect old XML
files? The proposals so far do not include anything for that, and I
don't think it should even be an issue.

Adding an XML language version might be possible, but is it necessary?


Thanks,
pq


pgpXU1pLF85fj.pgp
Description: OpenPGP digital signature
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Auke Booij]

On 2 October 2015 at 12:31, Pekka Paalanen  wrote:
>> I know that several people have proposed patches on this - Bill, Nils
>> and Auke at least.  Since there's a definite need for this, and since
>> agreement appears to be not far off, I would like to get this landed
>> this release.  And ideally I'd like to get this landed early in the
>> release so we give plenty of time for testing.
>
> The things that have been lacking in my opinion are specifications,
> especially answering the questions listed here by Victor after our irc
> discussion:
> http://lists.freedesktop.org/archives/wayland-devel/2015-September/024382.html
>
> The "do not break existing code" rule applies also proactively, not
> just at the time we decide to start using the new attributes, so it is
> important to define what things are under the "stable protocol"
> definition and cannot change after publishing. IOW, what are we carving
> in stone.
>
> There is a paradox: if the new attributes affect codegen for some
> languages in a way that will break the build (which is exactly the
> reason why they are wanted: to enforce type-safety), we cannot add
> these attributes to the existing protocols, most importantly
> wayland.xml, because it would "break existing code". You might escape
> that paradox once, by claiming that there is no "existing code", after
> all the generators for these languages will be new. But this escape
> hatch will only work once. When these generators get into use, we can
> no longer add the attributes to stable protocols without breaking
> things. Not even if the generators had a "legacy mode" that ignores the
> new attributes, because they would then regress on arguments that
> already had the new attributes set.
>
> The important thing to realize is that once an XML file starts using
> these new attributes, the attributes can no longer be added later into
> more arguments. We need to get it right on the first go, but hey,
> that's how extending already stabilized protocols goes.
>
> A further complication here is that currently, the definition of the
> XML language is literally "what wayland-scanner accepts", bugs and
> oversights in wayland-scanner notwithstanding. This makes it tricky to
> add XML language features that are not really used in the C bindings.
>
> Or, would it be ok to dismiss all non-C languages as second-class
> citizens, and say that we can change whatever as long as it does not
> break the wire format, wayland-scanner or the bindings generated by it?
> Somehow I don't think that would be an acceptable position.
>
> This is actually a fundamental question:
>
> Should we make any stability guarantees towards non-C bindings
> generators that exceed what we need for C?
>
> If no, then there's nothing to solve, but like I said, that would be
> pretty harsh and inconsiderate.
>
> Since most XML file authors are likely going to test only with
> wayland-scanner, wayland-scanner should detect as many problems as
> possible. This is why I have asked for generator patches to at least use
> the new attributes somehow. Otherwise we would be breaking the non-C
> generators all the time.

I agree (if that's what you're saying) that this point about
compatibility is the main debate towards the new attributes.

However, I'm not sure who you are trying to protect here. Everyone
agrees that the new attributes should not change anything for C/C++,
and in the current patches, they don't. And the other bindings writers
understand the compatibility issues regarding this change, and, if I
may speak for the majority of them, care much less about compatibility
issues than about being able to provide proper APIs for their users.

In haskell, and many other modern languages, an easily fixable compile
issue (we keep calling it a compatibility issue, which I think is a
misnomer, although I don't know what category it should fall under) is
*vastly* preferred over a potential bug. Arguably, that's the entire
point of the language. Modern languages attempt to cross-check as many
properties as possible, and the enum attribute would contribute
greatly to this, even if that means breaking API every so often.

The wayland protocol currently does not specify the enum attribute,
and I see no way how to write an API whose entire purpose is to
*break* when you erroneously mix up enum attribute data, without
breaking API as this data is added. More importantly, this problem
does not matter because it is not a problem but a *solution*.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Auke Booij]

On 2 October 2015 at 14:49, Victor Berger  wrote:
> Le 2015-10-02 15:16, Pekka Paalanen a écrit :
>>
>> On Fri, 2 Oct 2015 13:50:42 +0100
>> Auke Booij  wrote:
>>>
>>>
>>> [start]
>>> The enum and bitfield attributes are in principle for documentation
>>> purposes only. The enum and bitfield attributes may also be used by
>>> bindings, but only in such a way that code written prior to the
>>> specification of these attributes still works after their
>>> specification. In other words, specifying an attribute for an
>>> argument, that previously did not have it, should not break API.
>>> [end]
>>
>>
>> I like this very much. Let's see if anyone disagrees.
>>
>> Do you intend to allow also changing rather than only adding these new
>> attributes in the wording above?
>
>
> While I don't disagree, I have a small concern:
>
> Do we agree that this involves at some point writing a specification of the
> format of the XML files?
>
> Because otherwise, if the XML format remains defined by the implementation
> of the C scanner, and that these attributes are explicitly defined as for
> documentation only and ignored by the C scanner, this means the XML writers
> would be allowed to write any garbage they want as a value for these fields.
>
> As a consequence, bindings writers cannot give any value to these fields,
> and they become basically useless. I mean, it seems logical that the value
> of the "enum" field should be something like "enum_name" or
> "interface_name.enum_name", or whatever format will be chosen. But these
> fields have practically no value if we cannot expect this format to be
> respected (documentation-only fields can also simply be written in the
> "description" field, if they are not used anywhere).
>
> --
> Victor
>

I agree with your concern. However, note that "for documentation
purposes" does not mean that it is just a documentation string: we
document a very clear and precise fact, for the purpose of
understanding the API (ie documentation). My previous patches indeed
do some checking wrt enum name cross-matching. This would have to be
extended for enum names that refer to a different interface.

Pekka's proposal to include a "strict" mode will help here, for
cross-XML-interface-checking. Even though this would be optional, it
means we'll have to let go of the idea that protocol files are mostly
independent (if there ever was any such belief).
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Bryce Harrington]

On Fri, Oct 02, 2015 at 03:48:33PM +0300, Pekka Paalanen wrote:
> On Fri, 2 Oct 2015 12:53:13 +0100
> Auke Booij  wrote:
> 
> > On 1 October 2015 at 20:00, Nils Chr. Brause  
> > wrote:
> 
> > > I would prefer, if the enum attributes would also name the interface,
> > > where the enum can be found, e.g.:
> > > 
> > > If two enums in different interfaces happen to have the same name (if
> > > that's possible?), this would lead to ambiguities otherwise. Also a
> > > scanner wouldn't have to look up the interface name that way.
> > 
> > While in principle I think this is a great idea, this will need a few
> > specifications, which is why I decided not to add those in just yet.
> > Are cross-XML references allowed in this sense? In that case, the
> > scanner cannot verify their correctness, since only the current XML
> > file is available to it. Additionally, moving a certain interface from
> > xdg_shell to the core wayland protocol would now mean potentially
> > having to weaken the type safety of an interface, or having to copy
> > the enum over.
> 
> Hi,
> 
> yeah, adding the interface name makes perfect sense. It could be
> optional if wanted. No dot would mean the enum is in the same
> interface, a dot would signify a specific interface. I don't think
> there is any use for an anonymous global namespace like Bill mentioned.
> 
> As for link validity check, I could live with checking it only if the
> mentioned or implied interface is defined in the same XML file. This
> should cover most cases, and the rest would manifest as compile errors
> on non-C language bindings.
> 
> It's really all we can do anyway. XML files are perfectly allowed to
> reference interfaces defined elsewhere, even in XML files not available
> at the time.
> 
> A generator could have an extended check mode, where you can feed it a
> bunch of XML files for reference, while generating the code just for
> one, but I don't think it would be a good idea to absolutely require
> that on the XML language spec level, because it would again break old
> build systems.

Sounds like might be something worth adding to distcheck.
 
> Btw. when xdg-shell or any other new extension gets promoted to Wayland
> as stable, it will not be appended into wayland.xml. It will simply be
> yet another XML file to be installed by libwayland. Furthermore,
> libwayland will not install pre-generated C-bindings for it. All
> projects using the extension need to run wayland-scanner or any other
> generator during their build.

Do you expect that might pose problems for anyone?

Bryce
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Jasper St. Pierre]

We have a few constraints. First off, not all enums are closed. Some
are intentionally open, like xdg_shell.state. So we definitely need a
distinction between a closed enum and an open enum. I'm not familiar
enough with Rust to be able to apply something to that.

Second, I think we need to make a big effort to map out how the XML
converts to a wire format. For the most part, it's obvious, except for
the n -> sun hack we apply when we don't have an interface name. We
should probably specify that somewhere.

There's also a compatibility issue that has been brought up, but I
never understood that one. Somebody else would be able to talk about
that better.

On Thu, Oct 1, 2015 at 10:49 AM, Bryce Harrington  wrote:
> The topic of adding better enum/bitfield support to the protocol has
> come up a few[0] times[1] in the past, and again more recently[2].  We
> also have several proposed patches in patchwork, but I'm not sure they
> reflect consensus and none have Reviewed-by's on them yet [3,4,5,6,7].
>
> From what I gather of the discussions, no one is really against this
> feature conceptually, and impementationally the discussions appear to
> have moved further afield.  It feels like we're real close to having
> something that could be landed, but it's not 100% clear to me what
> exactly to land.  Since it's a protocol types change I would prefer to
> make sure it has a strong consensus before landing it.
>
> I know that several people have proposed patches on this - Bill, Nils
> and Auke at least.  Since there's a definite need for this, and since
> agreement appears to be not far off, I would like to get this landed
> this release.  And ideally I'd like to get this landed early in the
> release so we give plenty of time for testing.
>
> Since Auke's patchset proposalis the most recent, let's take that one as
> the candidate for landing.  Gentlemen, I'd like to ask you to review
> these three patches [5,6,7] and either give your Reviewed-by's or flag
> specific improvements needed.  If you have a more conceptual
> disagreement, and don't think the patchset is landable as implemented,
> please raise that issue asap too.
>
> Bryce
>
> 0: http://lists.freedesktop.org/archives/wayland-devel/2015-April/021438.html
> 1: http://lists.freedesktop.org/archives/wayland-devel/2015-June/023008.html
> 2: 
> http://lists.freedesktop.org/archives/wayland-devel/2015-September/024249.html
>
> 3: http://patchwork.freedesktop.org/patch/47726/
> 4: http://patchwork.freedesktop.org/patch/47727/
> 5: http://patchwork.freedesktop.org/patch/53018/
> 6: http://patchwork.freedesktop.org/patch/53019/
> 7: http://patchwork.freedesktop.org/patch/53020/
> ___
> wayland-devel mailing list
> wayland-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel



-- 
  Jasper
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Bill Spitzak]

I'm not sure why one of the large patches seems to be attributed to me, it
is actually Auke's.

My only contribution was a small patch (number 4) to use the enum/bitfield
information in the generated docs. My patch is slightly different that
Auke's proposal and he liked it better, I was just holding on to it until
the enum patches were applied.

I think the necessary enums were added to the xdg_shell and some other
protocol files but I don't see those patches listed.


On Thu, Oct 1, 2015 at 10:59 AM, Jasper St. Pierre 
wrote:

> We have a few constraints. First off, not all enums are closed. Some
> are intentionally open, like xdg_shell.state. So we definitely need a
> distinction between a closed enum and an open enum. I'm not familiar
> enough with Rust to be able to apply something to that.
>
> Second, I think we need to make a big effort to map out how the XML
> converts to a wire format. For the most part, it's obvious, except for
> the n -> sun hack we apply when we don't have an interface name. We
> should probably specify that somewhere.
>
> There's also a compatibility issue that has been brought up, but I
> never understood that one. Somebody else would be able to talk about
> that better.
>
> On Thu, Oct 1, 2015 at 10:49 AM, Bryce Harrington 
> wrote:
> > The topic of adding better enum/bitfield support to the protocol has
> > come up a few[0] times[1] in the past, and again more recently[2].  We
> > also have several proposed patches in patchwork, but I'm not sure they
> > reflect consensus and none have Reviewed-by's on them yet [3,4,5,6,7].
> >
> > From what I gather of the discussions, no one is really against this
> > feature conceptually, and impementationally the discussions appear to
> > have moved further afield.  It feels like we're real close to having
> > something that could be landed, but it's not 100% clear to me what
> > exactly to land.  Since it's a protocol types change I would prefer to
> > make sure it has a strong consensus before landing it.
> >
> > I know that several people have proposed patches on this - Bill, Nils
> > and Auke at least.  Since there's a definite need for this, and since
> > agreement appears to be not far off, I would like to get this landed
> > this release.  And ideally I'd like to get this landed early in the
> > release so we give plenty of time for testing.
> >
> > Since Auke's patchset proposalis the most recent, let's take that one as
> > the candidate for landing.  Gentlemen, I'd like to ask you to review
> > these three patches [5,6,7] and either give your Reviewed-by's or flag
> > specific improvements needed.  If you have a more conceptual
> > disagreement, and don't think the patchset is landable as implemented,
> > please raise that issue asap too.
> >
> > Bryce
> >
> > 0:
> http://lists.freedesktop.org/archives/wayland-devel/2015-April/021438.html
> > 1:
> http://lists.freedesktop.org/archives/wayland-devel/2015-June/023008.html
> > 2:
> http://lists.freedesktop.org/archives/wayland-devel/2015-September/024249.html
> >
> > 3: http://patchwork.freedesktop.org/patch/47726/
> > 4: http://patchwork.freedesktop.org/patch/47727/
> > 5: http://patchwork.freedesktop.org/patch/53018/
> > 6: http://patchwork.freedesktop.org/patch/53019/
> > 7: http://patchwork.freedesktop.org/patch/53020/
> > ___
> > wayland-devel mailing list
> > wayland-devel@lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/wayland-devel
>
>
>
> --
>   Jasper
>
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Nils Chr. Brause]

Hi,

On Thu, Oct 1, 2015 at 7:49 PM, Bryce Harrington  wrote:
> The topic of adding better enum/bitfield support to the protocol has
> come up a few[0] times[1] in the past, and again more recently[2].  We
> also have several proposed patches in patchwork, but I'm not sure they
> reflect consensus and none have Reviewed-by's on them yet [3,4,5,6,7].

I think this topic was first brought up in September 2014, so it is
more than a year now.

> From what I gather of the discussions, no one is really against this
> feature conceptually, and impementationally the discussions appear to
> have moved further afield.  It feels like we're real close to having
> something that could be landed, but it's not 100% clear to me what
> exactly to land.  Since it's a protocol types change I would prefer to
> make sure it has a strong consensus before landing it.
>
> I know that several people have proposed patches on this - Bill, Nils
> and Auke at least.  Since there's a definite need for this, and since
> agreement appears to be not far off, I would like to get this landed
> this release.  And ideally I'd like to get this landed early in the
> release so we give plenty of time for testing.

That would be great.

> Since Auke's patchset proposalis the most recent, let's take that one as
> the candidate for landing.  Gentlemen, I'd like to ask you to review
> these three patches [5,6,7] and either give your Reviewed-by's or flag
> specific improvements needed.  If you have a more conceptual
> disagreement, and don't think the patchset is landable as implemented,
> please raise that issue asap too.

There are some enum attributes missing, namely:
- wl_shm_pool::create_buffer::format (it's wl_shm::format)
- wl_shell_surface::set_fullscreen::method (it's
wl_shell_surface::fullscreen_method)
- wl_surface::set_buffer_transform::transform (it's wl_output::transform)

I would prefer, if the enum attributes would also name the interface,
where the enum can be found, e.g.:

If two enums in different interfaces happen to have the same name (if
that's possible?), this would lead to ambiguities otherwise. Also a
scanner wouldn't have to look up the interface name that way.

The scanner code looks good to me and seems to work.

The documentation looks good to me, too.

>
> Bryce
>
> 0: http://lists.freedesktop.org/archives/wayland-devel/2015-April/021438.html
> 1: http://lists.freedesktop.org/archives/wayland-devel/2015-June/023008.html
> 2: 
> http://lists.freedesktop.org/archives/wayland-devel/2015-September/024249.html
>
> 3: http://patchwork.freedesktop.org/patch/47726/
> 4: http://patchwork.freedesktop.org/patch/47727/
> 5: http://patchwork.freedesktop.org/patch/53018/
> 6: http://patchwork.freedesktop.org/patch/53019/
> 7: http://patchwork.freedesktop.org/patch/53020/
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays

[Bill Spitzak]

On Thu, Oct 1, 2015 at 12:00 PM, Nils Chr. Brause 
wrote:

>
> There are some enum attributes missing, namely:
> - wl_shm_pool::create_buffer::format (it's wl_shm::format)
> - wl_shell_surface::set_fullscreen::method (it's
> wl_shell_surface::fullscreen_method)
> - wl_surface::set_buffer_transform::transform (it's wl_output::transform)
>
> I would prefer, if the enum attributes would also name the interface,
> where the enum can be found, e.g.:
> 
> If two enums in different interfaces happen to have the same name (if
> that's possible?), this would lead to ambiguities otherwise. Also a
> scanner wouldn't have to look up the interface name that way.
>

Oh rats, it does look like you are right, and the current patch just
skipped all the cross-object enums. There will have to be changes to the
xml translators to handle these.

I'm not sure how hard it is to make xml translators change behaviour
depending on whether there is a period in the string. If so then the
current protocol file patches could continue to work, by making "no period"
mean "in the current object". But it might be easier to just add the object
name to all enum attributes, like you propose. That could also leave open
the possibility that "no period" means some future "global" namespace for
the enum.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays in the .xml file

[Victor Berger]

Hi,

First of all, thanks for your replies.

On 2015-09-24 20:35, Bill Spitzak wrote :


Um, that is entirely the point of this change! The current xml does
not provide enough information so that a language binding can know
that some arrangements are illegal. Therefore current language
bindings don't produce errors for invalid code. With this extra
information, the compiler now knows that the arrangement is invalid,
and can produce an error. If a new error is a "breaking change" then
this patch MUST be a breaking change, or it is not working.


Yes I agree with you from the perspective of my Rust bindings. However, 
from discussions on IRC, there is a strong position towards 
backwards-compatibility.
From what I understood, it seems like XML files that are currently 
accepted as valid by the C scanner will continue to be accepted by it.


Thus, we risk reaching a point where different scanners accept different 
XML formats. And for now, the specification of the XML files is defined 
by the implementation of the C scanner, which is likely to be the most 
tolerant scanner.



What do you mean by "variant"? My guess is this does not mean "a new
enum value that did not exist before", but instead the ability to send
arbitrary numbers. But that seems to be duplicated by your next
statement:


Actually, I was exactly meaning "a new enum value that did not exist 
before". Rust "enums" are actually tagged unions, and if you match over 
an instance of these "enums", the compiler will force you to match all 
possible values. Hence, introducing a new possible value for a Rust enum 
is a breaking change of the API (in terms of the generated code).


But indeed, given there is a need to support arbitrary values for enums 
in some protocols, this problem can be solved at the same time as the 
next:



- some enums, like xdg_shell::xdg_surface::states are by definition
not complete.


Therefore the language binding must provide a method to send an
arbitrary number. However this can require extra syntax so that it is
clear that the programmer intended it. In C++ this is done by casting
that number to the enumeration value, ie you can't put 42 in the
argument list, but you can put EnumerationType(42). Your description
of how you would handle this in Rust sounds the same.


In Rust it would require some more overhead (casting an arbitrary number 
as an enum is unsafe and undefined behavior in Rust, even if the enum is 
internally represented by the same type as the given integer), but it is 
essentially the same idea, yes.


Actually, my concern was more about *receiving* and arbitrary number 
than *sending* it. The bindings must be able to gracefully handle a 
value returned by the compositor that does not fit into the enums 
described by the protocol. It is however mostly the same problem, and 
can be fixed in the same way.



- and most likely other similar issues with "array" types or so



These questions need to be resolved, and a proper specification of
what should contain the .xml files will certainly need to be written
to properly handle all this.


The proposed additions provide EXACTLY what is needed by many language
bindings. Yes maybe there willl be new things added later. That is not
a reason to stop the current proposal.


What I meant here is that the format and contents of the XML files is 
currently defined by the implementation of the C scanner, which is a 
less than optimal situation to discuss evolutions of this format.


There will most likely be a need to write a proper document describing 
the format of the XML files, as well as the semantic meanings of each 
field and attribute it contains.
The questions about how breaking evolutions will be handled need to be 
specified as well: how should an old scanner behave when it encounters a 
more recent protocol file, containing fields or attributes it does not 
recognize ? Ignore them ? Fail and declare being "not compatible with 
this protocol format, please upgrade" ?



It does sound a bit like it would be useful for Rust to know whether
it needs to support arbitrary values ("variant"s?), because an enum
that does not support them might be a lot simpler (in fact this is
probably the main reason for the bitfield indicator, in C++ and Python
and probably many languages a type that supports bit operations is
much more complex and ugly than an enum). So maybe a new field should
be added to enum/bitfields to indicate that arbitrary values must be
supported. However that is a NEW field, it does not change the current
proposal at all, and there is no reason to hold off useful and
important changes until this new field is decided on.


I first thought knowing when to support arbitrary values would be an 
important point, but actually, as I described a few paragraphs earlier 
in this email, it is mostly the same issue as future-proofing the enums 
to handle newer values the compositor might send.


As I said described earlier in this mail, adding a new 

Re: Enums, bitfields and wl_arrays in the .xml file

[Victor Berger]

On 2015-09-25 12:25, Nils Chr. Brause wrote :

Hi,

On Fri, Sep 25, 2015 at 10:52 AM, Victor Berger
 wrote:


What I meant here is that the format and contents of the XML files is
currently defined by the implementation of the C scanner, which is a 
less

than optimal situation to discuss evolutions of this format.

There will most likely be a need to write a proper document describing 
the
format of the XML files, as well as the semantic meanings of each 
field and

attribute it contains.


Isn't the DTD file the specification for the XML file format?


Not exactly, it only defines the valid structure of the file, but not 
how a scanner should interpret the values contained in the fields.


Mainly, a specification of how a scanner should behave given an XML 
input. In my opinion, it should be informative enough to write a new 
scanner
for any language _without_ needing to reverse-engineer the C scanner 
(which I had to do to write my Rust bindings).


Ideally, the protocol itself could be separated from the 
wayland-{client,server} libraries, but in practice it is not possible to 
re-implement the protocol down from the
wire format (which is a shame, but required for EGL support with mesa). 
So I think it makes sense to describe it altogether.



The questions about how breaking evolutions will be handled need to be
specified as well: how should an old scanner behave when it encounters 
a

more recent protocol file, containing fields or attributes it does not
recognize ? Ignore them ? Fail and declare being "not compatible with 
this

protocol format, please upgrade" ?


The last one makes the most sense to me. Someone who develops a
language binding should always keep her/his scanner up to date with the
most recent XML file format.


Agreed...


[...]
Also, from the discussion last year it emerged that the scanner should 
also be

modified to at least check for the validity of the new attribues.
You can find the work that I had done so far here:
https://github.com/NilsBrause/Wayland/commits/xml


... but from the discussion I had with pq on IRC, it does not seem like 
having the C-scanner start rejecting "old" protocol files is a welcomes 
outcome, from they point of view.


However, maybe checking the file, and emitting warnings if it is in an 
old format could be an option ?


The other bindings could explicitly refuse old formats, I don't think 
it's a deal breaker either.



Victor


___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays in the .xml file

[Nils Chr. Brause]

Hi,

On Fri, Sep 25, 2015 at 10:52 AM, Victor Berger
 wrote:
>
> What I meant here is that the format and contents of the XML files is
> currently defined by the implementation of the C scanner, which is a less
> than optimal situation to discuss evolutions of this format.
>
> There will most likely be a need to write a proper document describing the
> format of the XML files, as well as the semantic meanings of each field and
> attribute it contains.

Isn't the DTD file the specification for the XML file format?

> The questions about how breaking evolutions will be handled need to be
> specified as well: how should an old scanner behave when it encounters a
> more recent protocol file, containing fields or attributes it does not
> recognize ? Ignore them ? Fail and declare being "not compatible with this
> protocol format, please upgrade" ?

The last one makes the most sense to me. Someone who develops a
language binding should always keep her/his scanner up to date with the
most recent XML file format.

> I don't think other bindings than the C-scanner already have so many uses
> that they can't afford a breaking change (correct me if I'm wrong), so this
> is most likely the ideal time to properly specify the protocol file format
> in a way that contains all information required for a proper type-safety as
> needed by more high-level languages than C.
> [...]
> Future changes to the XML format would likely not be as painless, when
> various languages bindings will have many downstream users. Thus it is
> important to get a good version of the XML specification now. As there are
> bindings projects for various languages (at least C++, Java, Rust and
> Haskell that I am aware of), we can probably together reach a state handling
> (almost ?) all needs of type-safety.

I agree. The earlier these important changes are being made, the easier it is.

> I am aware this is a more consequent work than simply patching the xml
> files on the go as the need is encountered, but providing a good and complete
> proposition will most likely be better received by the wayland devs and the
> downstream users than just "breaking things as we need it".

Also, from the discussion last year it emerged that the scanner should also be
modified to at least check for the validity of the new attribues.
You can find the work that I had done so far here:
https://github.com/NilsBrause/Wayland/commits/xml

Cheers,
Nils
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays in the .xml file

[Victor Berger]

Hi,

After some discussions on IRC, it appears this raises several concerns 
about back-compatibility.


The main points being:

- if a protocol file previously did not use these extra attributes, and 
choses to add them, depending on the language using them it can be a 
breaking change (as it would change the prototype of the requests from 
u(int) to the enum type for example)
- if a variant is added to an already existing enum, it can be a 
breaking change for some languages (Rust is one of them, as the compiler 
enforces you to cover all possible variants when matching on an enum).
- some enums, like xdg_shell::xdg_surface::states are by definition not 
complete.

- and most likely other similar issues with "array" types or so

These questions need to be resolved, and a proper specification of what 
should contain the .xml files will certainly need to be written to 
properly handle all this.


To other bindings writers: what are you opinions and constraints on 
these back-compatibility hazards ?


In my case (Rust):
- the question adding the an "enum" attribute to a field that didn't use 
it is a breaking change that needs thought.

- adding a variant to a bitfield is not a problem
- adding a variant to a classic enum is not trivial a priori, but I have 
possibilities to future-proof this with small overhead, and it would 
handle incomplete enums as well. (it involves using a wrapping struct 
that optionnaly tries to convert the raw value to the enum)
- as well, adding the specification of the type contained in an array to 
an arg that didn't have it would be a breaking change, and as such needs 
thought.



Victor

Le 2015-09-18 20:55, Bill Spitzak a écrit :

On Fri, Sep 18, 2015 at 5:37 AM, Nils Chr. Brause
 wrote:


Hi,

There are even earlier discussions about including 'bitfield' and
'enum' fields into the XML protocol file, e.g:


http://lists.freedesktop.org/archives/wayland-devel/2014-September/017071.html

[1]
But none of them led to any actual changes.

I still would very much like to see the 'bitfield' and 'enum'
fields
to be included in the XML protocol file. This would greatly
simplify
the creation of Wayland bindings for most high level programming
languages and thus increase the popularity of Wayland amongst
programmers who don't wish to use toolkits.
Without these information about bitfields and enumerations,
language
bindings would have to maintain their own version of the XML
protocol
file, like I am doing with my C++ bindings here:
https://github.com/NilsBrause/waylandpp [2]


Yes please get this in. I really expected this to be merged long ago.

This is almost a requirement for bindings for any language other than
C. And it greatly improves the automatically generated documentation
of the C api, too!

And it is absolutely 100% compatible with everything that exists right
now (except code reading the xml files, which might have to be
modified to ignore the new tags). It does not change the size or bit
layout of any data in the stream protocol and does not change the C
api, and does not prevent the sending or construction of any message
or event that can be sent right now. In particular it does not prevent
bit patterns that are not in the enumeration from being sent by the C
api.



Links:
--
[1]
http://lists.freedesktop.org/archives/wayland-devel/2014-September/017071.html
[2] https://github.com/NilsBrause/waylandpp

___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel


___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays in the .xml file

[Nils Chr. Brause]

Hi

These are all  very interesting points.

I understand that some language bindings might break existing code, if
they want to use the 'enum' and 'bitfield' attributes. But they could
simply bump the major version of their bindings, in that case. In my
opinion this is a low price to pay for higher qualitiy bindings.

In the case of my C++ bindings, things are fortunately a bit easier:

- Since I am already using the 'enum' and 'bitfield' attributes,
nothing needs to be changed here.
- Adding entries to enumerations or bitfields doesn't break existing code.
- Being an incomplete enumeration is only a problem, if the user
wishes to use a value that is not part of the protocol XML used to
generate the bindings. (One could still forcefully convert an integer
to an enum  class and vice-versa through static_cast<>(), though.)
- Adding a content type to an array might need a bit of thought.
Currently there seems to be only one use of the array type in the
official Wayland protocol XML, which is wl_keyboard::enter. I define
its wl_array as a std::vector. If it turns out to be something
else, this would indeed break existing code. But since I have not yet
declared my bindings as stable, this isn't a problem for me.

Cheers,
Nils


On Thu, Sep 24, 2015 at 1:00 PM, Victor Berger
 wrote:
> Hi,
>
> After some discussions on IRC, it appears this raises several concerns about
> back-compatibility.
>
> The main points being:
>
> - if a protocol file previously did not use these extra attributes, and
> choses to add them, depending on the language using them it can be a
> breaking change (as it would change the prototype of the requests from
> u(int) to the enum type for example)
> - if a variant is added to an already existing enum, it can be a breaking
> change for some languages (Rust is one of them, as the compiler enforces you
> to cover all possible variants when matching on an enum).
> - some enums, like xdg_shell::xdg_surface::states are by definition not
> complete.
> - and most likely other similar issues with "array" types or so
>
> These questions need to be resolved, and a proper specification of what
> should contain the .xml files will certainly need to be written to properly
> handle all this.
>
> To other bindings writers: what are you opinions and constraints on these
> back-compatibility hazards ?
>
> In my case (Rust):
> - the question adding the an "enum" attribute to a field that didn't use it
> is a breaking change that needs thought.
> - adding a variant to a bitfield is not a problem
> - adding a variant to a classic enum is not trivial a priori, but I have
> possibilities to future-proof this with small overhead, and it would handle
> incomplete enums as well. (it involves using a wrapping struct that
> optionnaly tries to convert the raw value to the enum)
> - as well, adding the specification of the type contained in an array to an
> arg that didn't have it would be a breaking change, and as such needs
> thought.
>
> 
> Victor
>
> Le 2015-09-18 20:55, Bill Spitzak a écrit :
>>
>> On Fri, Sep 18, 2015 at 5:37 AM, Nils Chr. Brause
>>  wrote:
>>
>>> Hi,
>>>
>>> There are even earlier discussions about including 'bitfield' and
>>> 'enum' fields into the XML protocol file, e.g:
>>>
>>
>> http://lists.freedesktop.org/archives/wayland-devel/2014-September/017071.html
>>>
>>> [1]
>>> But none of them led to any actual changes.
>>>
>>> I still would very much like to see the 'bitfield' and 'enum'
>>> fields
>>> to be included in the XML protocol file. This would greatly
>>> simplify
>>> the creation of Wayland bindings for most high level programming
>>> languages and thus increase the popularity of Wayland amongst
>>> programmers who don't wish to use toolkits.
>>> Without these information about bitfields and enumerations,
>>> language
>>> bindings would have to maintain their own version of the XML
>>> protocol
>>> file, like I am doing with my C++ bindings here:
>>> https://github.com/NilsBrause/waylandpp [2]
>>
>>
>> Yes please get this in. I really expected this to be merged long ago.
>>
>> This is almost a requirement for bindings for any language other than
>> C. And it greatly improves the automatically generated documentation
>> of the C api, too!
>>
>> And it is absolutely 100% compatible with everything that exists right
>> now (except code reading the xml files, which might have to be
>> modified to ignore the new tags). It does not change the size or bit
>> layout of any data in the stream protocol and does not change the C
>> api, and does not prevent the sending or construction of any message
>> or event that can be sent right now. In particular it does not prevent
>> bit patterns that are not in the enumeration from being sent by the C
>> api.
>>
>>
>>
>> Links:
>> --
>> [1]
>>
>> http://lists.freedesktop.org/archives/wayland-devel/2014-September/017071.html
>> [2] https://github.com/NilsBrause/waylandpp
>>
>> 

Re: Enums, bitfields and wl_arrays in the .xml file

[Bill Spitzak]

On Thu, Sep 24, 2015 at 4:00 AM, Victor Berger <
victor.ber...@polytechnique.org> wrote:

> Hi,
>
> After some discussions on IRC, it appears this raises several concerns
> about back-compatibility.
>
> The main points being:
>
> - if a protocol file previously did not use these extra attributes, and
> choses to add them, depending on the language using them it can be a
> breaking change (as it would change the prototype of the requests from
> u(int) to the enum type for example)
>

Um, that is entirely the point of this change! The current xml does not
provide enough information so that a language binding can know that some
arrangements are illegal. Therefore current language bindings don't produce
errors for invalid code. With this extra information, the compiler now
knows that the arrangement is invalid, and can produce an error. If a new
error is a "breaking change" then this patch MUST be a breaking change, or
it is not working.


> - if a variant is added to an already existing enum, it can be a breaking
> change for some languages (Rust is one of them, as the compiler enforces
> you to cover all possible variants when matching on an enum).
>

What do you mean by "variant"? My guess is this does not mean "a new enum
value that did not exist before", but instead the ability to send arbitrary
numbers. But that seems to be duplicated by your next statement:


> - some enums, like xdg_shell::xdg_surface::states are by definition not
> complete.
>

Therefore the language binding must provide a method to send an arbitrary
number. However this can require extra syntax so that it is clear that the
programmer intended it. In C++ this is done by casting that number to the
enumeration value, ie you can't put 42 in the argument list, but you can
put EnumerationType(42). Your description of how you would handle this in
Rust sounds the same.

- and most likely other similar issues with "array" types or so
>
These questions need to be resolved, and a proper specification of what
> should contain the .xml files will certainly need to be written to properly
> handle all this.
>

The proposed additions provide EXACTLY what is needed by many language
bindings. Yes maybe there willl be new things added later. That is not a
reason to stop the current proposal.

It does sound a bit like it would be useful for Rust to know whether it
needs to support arbitrary values ("variant"s?), because an enum that does
not support them might be a lot simpler (in fact this is probably the main
reason for the bitfield indicator, in C++ and Python and probably many
languages a type that supports bit operations is much more complex and ugly
than an enum). So maybe a new field should be added to enum/bitfields to
indicate that arbitrary values must be supported. However that is a NEW
field, it does not change the current proposal at all, and there is no
reason to hold off useful and important changes until this new field is
decided on.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays in the .xml file

[Bill Spitzak]

On Fri, Sep 18, 2015 at 5:37 AM, Nils Chr. Brause 
wrote:

> Hi,
>
> There are even earlier discussions about including 'bitfield' and
> 'enum' fields into the XML protocol file, e.g:
>
> http://lists.freedesktop.org/archives/wayland-devel/2014-September/017071.html
> But none of them led to any actual changes.
>
> I still would very much like to see the 'bitfield' and 'enum' fields
> to be included in the XML protocol file. This would greatly simplify
> the creation of Wayland bindings for most high level programming
> languages and thus increase the popularity of Wayland amongst
> programmers who don't wish to use toolkits.
> Without these information about bitfields and enumerations, language
> bindings would have to maintain their own version of the XML protocol
> file, like I am doing with my C++ bindings here:
> https://github.com/NilsBrause/waylandpp


Yes please get this in. I really expected this to be merged long ago.

This is almost a requirement for bindings for any language other than C.
And it greatly improves the automatically generated documentation of the C
api, too!

And it is absolutely 100% compatible with everything that exists right now
(except code reading the xml files, which might have to be modified to
ignore the new tags). It does not change the size or bit layout of any data
in the stream protocol and does not change the C api, and does not prevent
the sending or construction of any message or event that can be sent right
now. In particular it does not prevent bit patterns that are not in the
enumeration from being sent by the C api.
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel

Re: Enums, bitfields and wl_arrays in the .xml file

[Nils Chr. Brause]

Hi,

There are even earlier discussions about including 'bitfield' and
'enum' fields into the XML protocol file, e.g:
http://lists.freedesktop.org/archives/wayland-devel/2014-September/017071.html
But none of them led to any actual changes.

I still would very much like to see the 'bitfield' and 'enum' fields
to be included in the XML protocol file. This would greatly simplify
the creation of Wayland bindings for most high level programming
languages and thus increase the popularity of Wayland amongst
programmers who don't wish to use toolkits.
Without these information about bitfields and enumerations, language
bindings would have to maintain their own version of the XML protocol
file, like I am doing with my C++ bindings here:
https://github.com/NilsBrause/waylandpp

Adding content information to the 'array' type is also a good idea in
my humble opinion.

Cheers,
Nils


On Thu, Sep 17, 2015 at 5:00 PM, Victor Berger  wrote:
> Hi,
>
> I would like to revive a previous discussion that apparently died a few
> months ago on this mailing-list: the question of adding information
> specifying when a (u)int argument in a message is supposed to be a value of
> an enum, and which is the associated enum, as well a specifying when an enum
> should be treated as a bitfield.
>
> (see
> http://lists.freedesktop.org/archives/wayland-devel/2015-June/023008.html )
>
> I'm in the process of writing a generator for wayland bindings for the Rust
> language, and such information would be a huge help to provide a safer
> interface.
>
> On the same vein, I'm also having some issues with the wl_array type. To my
> understanding, it is itself a plain byte array. But can actually contain an
> arbitrary type. For example the array provided by the wl_keyboard::enter
> event is actually an array of keycodes, and as such of ints or uints (not
> sure about that). However this is never specified in the .xml protocol file.
>
> Wouldn't it be appropriate to add this information in the file ? Maybe
> something like:
>
>  content="uint"/>
>
> I believe such an addition would be as backward-compatible as the previously
> suggested change about enums and bitfields.
>
> Thanks,
> Victor Berger
>
> ___
> wayland-devel mailing list
> wayland-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/wayland-devel
___
wayland-devel mailing list
wayland-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/wayland-devel