Re: [llvmlinux] [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-05 Thread Bryce Lelbach
On 2012.12.04 23.24, Sebastian Andrzej Siewior wrote:
> > VLAIS is not something they are willing to accept (for various
> > reasons). There are other patches to LLVM that are still working
> Is this not described in C99 6.7.2.1p16?

No, this is not the case - flexible array members are a very different beast,
which are in-and-of-themselves of debatable merit. C99 6.7.2.1p8 explicitly
states that VLAIS are illegal. The same language is present in C11 (same
section):

A member of a structure or union may have any object type other than a 
variably
modified type.

In C99 6.7.5.1p3, the definition of a variably modified type is given:

.. [snip] ... If the nested sequence of declarators in a full declarator
contains a variable length array type, the type specified by the full
declarator is said to be variably modified.

It would require incredible, earth-moving acts for VLAIS to become a part of the
ISO/IEC standard for the C programming language.

P.S. As a general note, while I do agree that it is preferable to rewrite code
using VLAIS instead of utilizing a macro, I would ask that some reconsideration
be given, unless there are Linux kernel developers who are willing and able
to implement the necessary rewrites. The LLVMLinux project does not wish to
decrease the code quality of the Linux kernel, and certainly we do not intend to
cause any GCC breakage.

However, I think that rewrites to all the relevant VLAIS code would be quite
time consuming for the LLVMLinux team, and the proposed macro does not appear
to me to present any particular evil, other than by nature of being a macro.

If faced between the choice of either accepting this VLAIS patch, or not being
able to compile the Linux kernel with standard-compliant, largely GCC-compatible
compilers such as Clang and icc, I would think that accepting the VLAIS patch
would be preferable.

(note also that I have only come in on the end of this thread, and I may have
missed part of the context).

-- 
Bryce Adelstein-Lelbach aka wash
STE||AR Group, Center for Computation and Technology, LSU
--
860-808-7497 - Cell
225-578-6182 - Work (no voicemail)
--
stellar.cct.lsu.edu
boost-spirit.com
llvm.linuxfoundation.org
--


signature.asc
Description: Digital signature


Re: [llvmlinux] [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-05 Thread Bryce Lelbach
On 2012.12.04 23.24, Sebastian Andrzej Siewior wrote:
  VLAIS is not something they are willing to accept (for various
  reasons). There are other patches to LLVM that are still working
 Is this not described in C99 6.7.2.1p16?

No, this is not the case - flexible array members are a very different beast,
which are in-and-of-themselves of debatable merit. C99 6.7.2.1p8 explicitly
states that VLAIS are illegal. The same language is present in C11 (same
section):

A member of a structure or union may have any object type other than a 
variably
modified type.

In C99 6.7.5.1p3, the definition of a variably modified type is given:

.. [snip] ... If the nested sequence of declarators in a full declarator
contains a variable length array type, the type specified by the full
declarator is said to be variably modified.

It would require incredible, earth-moving acts for VLAIS to become a part of the
ISO/IEC standard for the C programming language.

P.S. As a general note, while I do agree that it is preferable to rewrite code
using VLAIS instead of utilizing a macro, I would ask that some reconsideration
be given, unless there are Linux kernel developers who are willing and able
to implement the necessary rewrites. The LLVMLinux project does not wish to
decrease the code quality of the Linux kernel, and certainly we do not intend to
cause any GCC breakage.

However, I think that rewrites to all the relevant VLAIS code would be quite
time consuming for the LLVMLinux team, and the proposed macro does not appear
to me to present any particular evil, other than by nature of being a macro.

If faced between the choice of either accepting this VLAIS patch, or not being
able to compile the Linux kernel with standard-compliant, largely GCC-compatible
compilers such as Clang and icc, I would think that accepting the VLAIS patch
would be preferable.

(note also that I have only come in on the end of this thread, and I may have
missed part of the context).

-- 
Bryce Adelstein-Lelbach aka wash
STE||AR Group, Center for Computation and Technology, LSU
--
860-808-7497 - Cell
225-578-6182 - Work (no voicemail)
--
stellar.cct.lsu.edu
boost-spirit.com
llvm.linuxfoundation.org
--


signature.asc
Description: Digital signature


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-04 Thread Behan Webster

On 12-12-04 11:24 PM, Sebastian Andrzej Siewior wrote:

On Mon, Dec 03, 2012 at 07:57:33PM +0100, Behan Webster wrote:

However, in order to approximate what gcc is doing in code, obviously
some math is required. The thought was that macros would hide the
worst of it, trying not to obfuscate what was actually being done.

Why hide? The thing that is done here is setting up pointers and keep this
struct as a container. It is never used again, just freed. Therefore I would
suggest to remove and do something different.

Fair enough. That works too.


I've been profiling some sample code around this implementation
comparing it between compilers, and it approximates the code size and
speed of using VLAIS in gcc (especially with -O2). It actually
performs better than the previously proposed macros.

I'm not concerned about speed here. This is an one time setup.

Which is all good in the case of the gadget code.

However VLAIS is used in other places too where code size and speed are 
very important. The proposal was for something which might be used for 
more than just the gadget code. VLAIS is used in only a handful of 
places in the kernel. They just happen to be in important parts of the 
kernel which was why the approach was to change as little as necessary.



But certainly if anyone has a solution which works for everyone, then
I'm more than happy to adopt it. The LLVM community has made quite a
few changes in order to help get Linux working with clang. However,

That is nice to hear. Besides gcc there is the icc.
Lots of people have brought up icc. Although in the same breath most 
also say nobody is using it anymore. However, I have no idea whether 
that is true. The difference is that clang is widely available in most 
distros; icc is not.



VLAIS is not something they are willing to accept (for various
reasons). There are other patches to LLVM that are still working

Is this not described in C99 6.7.2.1p16?
You're thinking of "Flexible Array Members". Essentially the last 
element in a structure can be the equivalent of a zero length array; 
something which both gcc and clang support today.


From C99 6.7.2.1p16:
16 As a special case, the last element of a structure with more than 
one named member may
have an incomplete array type; this is called a flexible array member. 
In most situations,
the flexible array member is ignored. In particular, the size of the 
structure is as if the
flexible array member were omitted except that it may have more 
trailing padding than
the omission would imply. Howev er, when a . (or ->) operator has a 
left operand that is
(a pointer to) a structure with a flexible array member and the right 
operand names that
member, it behaves as if that member were replaced with the longest 
array (with the same
element type) that would not make the structure larger than the object 
being accessed; the
offset of the array shall remain that of the flexible array member, 
even if this would differ
from that of the replacement array. If this array would have no 
elements, it behaves as if
it had one element but the behavior is undefined if any attempt is made 
to access that

element or to generate a pointer one past it.


However if we're considering the C99 standard, it also explicitly 
disallows the use of VLAs in structs (VLAIS).


From C99 6.7.2.1 p2:
2 A structure or union shall not contain a member with incomplete or 
function type (hence,
a structure shall not contain an instance of itself, but may contain a 
pointer to an instance
of itself), except that the last member of a structure with more than 
one named member
may have incomplete array type; such a structure (and any union 
containing, possibly
recursively, a member that is such a structure) shall not be a member 
of a structure or an

element of an array.

From C99 6.7.5.2 p4:

4 If the size is not present, the array type is an incomplete type. 
Essentially all arrays in a structure must not be variable length except 
the last member of the array which can be a zero length array (which can 
be used to approximate a VLA at the end). It also says that a struct 
which uses a flexible array member can't be used inside another struct.



their way upstream that are required to be able to compile Linux as
well.

I hope the other are "simple" to get in :)
I hope so too. But like the Linux kernel community, the LLVM community 
have standards and procedures for getting code accepted which take time.


Thanks,

Behan

--
Behan Webster
beh...@converseincode.com

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-04 Thread Sebastian Andrzej Siewior
On Mon, Dec 03, 2012 at 07:57:33PM +0100, Behan Webster wrote:
> However, in order to approximate what gcc is doing in code, obviously
> some math is required. The thought was that macros would hide the
> worst of it, trying not to obfuscate what was actually being done.

Why hide? The thing that is done here is setting up pointers and keep this
struct as a container. It is never used again, just freed. Therefore I would
suggest to remove and do something different.

> One of the project members came up with this alternative. How about
> something like this? Less math, though more string pasting. When
> compiled, the unused variables get optimized away. Otherwise the
> memory packing is identical to using VLAIS in gcc.
> 
> #define vla_struct(structname) size_t structname##__##next = 0
> #define vla_struct_size(structname) structname##__##next
> 
> #define vla_item(structname, type, name, n) \
> type * structname##_##name; \
> size_t structname##_##name##__##pad = \
> (structname##__##next & (__alignof__(type)-1)); \
> size_t structname##_##name##__##offset = \
> structname##__##next + structname##_##name##__##pad; \
> size_t structname##_##name##__##sz = n * sizeof(type); \
> structname##__##next = structname##__##next + \
> structname##_##name##__##pad +
> structname##_##name##__##sz;
> 
> #define vla_ptr(ptr,structname,name) structname##_##name = \
> (typeof(structname##_##name))[structname##_##name##__##offset]
> 
> 
> Then you can do something like this that looks vaguely struct-like:
> 
> vla_struct(foo);
> vla_item(foo, char,  vara, 1);
> vla_item(foo, short, varb, 10);
> vla_item(foo, int,   varc, 5);
> vla_item(foo, long,  vard, 3);
> size_t total = vla_struct_size(foo);
> char buffer[total];
> 
> vla_ptr(buffer, foo, varc);
> foo_varc = 1;

I prefer to try to rewritte the code in the gadget in a different manner
before using macro magic. I guess most people around here think that extensive
usage of macros equals giving a gun to a chimpanzee. It may work for a while
and may even look cute in the eye of the gun sponsor. However once it fires…

> I've been profiling some sample code around this implementation
> comparing it between compilers, and it approximates the code size and
> speed of using VLAIS in gcc (especially with -O2). It actually
> performs better than the previously proposed macros.

I'm not concerned about speed here. This is an one time setup.

> But certainly if anyone has a solution which works for everyone, then
> I'm more than happy to adopt it. The LLVM community has made quite a
> few changes in order to help get Linux working with clang. However,

That is nice to hear. Besides gcc there is the icc.

> VLAIS is not something they are willing to accept (for various
> reasons). There are other patches to LLVM that are still working
Is this not described in C99 6.7.2.1p16?

> their way upstream that are required to be able to compile Linux as
> well.

I hope the other are "simple" to get in :)

> 
> Behan
> 

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-04 Thread Sebastian Andrzej Siewior
On Mon, Dec 03, 2012 at 07:57:33PM +0100, Behan Webster wrote:
 However, in order to approximate what gcc is doing in code, obviously
 some math is required. The thought was that macros would hide the
 worst of it, trying not to obfuscate what was actually being done.

Why hide? The thing that is done here is setting up pointers and keep this
struct as a container. It is never used again, just freed. Therefore I would
suggest to remove and do something different.

 One of the project members came up with this alternative. How about
 something like this? Less math, though more string pasting. When
 compiled, the unused variables get optimized away. Otherwise the
 memory packing is identical to using VLAIS in gcc.
 
 #define vla_struct(structname) size_t structname##__##next = 0
 #define vla_struct_size(structname) structname##__##next
 
 #define vla_item(structname, type, name, n) \
 type * structname##_##name; \
 size_t structname##_##name##__##pad = \
 (structname##__##next  (__alignof__(type)-1)); \
 size_t structname##_##name##__##offset = \
 structname##__##next + structname##_##name##__##pad; \
 size_t structname##_##name##__##sz = n * sizeof(type); \
 structname##__##next = structname##__##next + \
 structname##_##name##__##pad +
 structname##_##name##__##sz;
 
 #define vla_ptr(ptr,structname,name) structname##_##name = \
 (typeof(structname##_##name))ptr[structname##_##name##__##offset]
 
 
 Then you can do something like this that looks vaguely struct-like:
 
 vla_struct(foo);
 vla_item(foo, char,  vara, 1);
 vla_item(foo, short, varb, 10);
 vla_item(foo, int,   varc, 5);
 vla_item(foo, long,  vard, 3);
 size_t total = vla_struct_size(foo);
 char buffer[total];
 
 vla_ptr(buffer, foo, varc);
 foo_varc = 1;

I prefer to try to rewritte the code in the gadget in a different manner
before using macro magic. I guess most people around here think that extensive
usage of macros equals giving a gun to a chimpanzee. It may work for a while
and may even look cute in the eye of the gun sponsor. However once it fires…

 I've been profiling some sample code around this implementation
 comparing it between compilers, and it approximates the code size and
 speed of using VLAIS in gcc (especially with -O2). It actually
 performs better than the previously proposed macros.

I'm not concerned about speed here. This is an one time setup.

 But certainly if anyone has a solution which works for everyone, then
 I'm more than happy to adopt it. The LLVM community has made quite a
 few changes in order to help get Linux working with clang. However,

That is nice to hear. Besides gcc there is the icc.

 VLAIS is not something they are willing to accept (for various
 reasons). There are other patches to LLVM that are still working
Is this not described in C99 6.7.2.1p16?

 their way upstream that are required to be able to compile Linux as
 well.

I hope the other are simple to get in :)

 
 Behan
 

Sebastian
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-04 Thread Behan Webster

On 12-12-04 11:24 PM, Sebastian Andrzej Siewior wrote:

On Mon, Dec 03, 2012 at 07:57:33PM +0100, Behan Webster wrote:

However, in order to approximate what gcc is doing in code, obviously
some math is required. The thought was that macros would hide the
worst of it, trying not to obfuscate what was actually being done.

Why hide? The thing that is done here is setting up pointers and keep this
struct as a container. It is never used again, just freed. Therefore I would
suggest to remove and do something different.

Fair enough. That works too.


I've been profiling some sample code around this implementation
comparing it between compilers, and it approximates the code size and
speed of using VLAIS in gcc (especially with -O2). It actually
performs better than the previously proposed macros.

I'm not concerned about speed here. This is an one time setup.

Which is all good in the case of the gadget code.

However VLAIS is used in other places too where code size and speed are 
very important. The proposal was for something which might be used for 
more than just the gadget code. VLAIS is used in only a handful of 
places in the kernel. They just happen to be in important parts of the 
kernel which was why the approach was to change as little as necessary.



But certainly if anyone has a solution which works for everyone, then
I'm more than happy to adopt it. The LLVM community has made quite a
few changes in order to help get Linux working with clang. However,

That is nice to hear. Besides gcc there is the icc.
Lots of people have brought up icc. Although in the same breath most 
also say nobody is using it anymore. However, I have no idea whether 
that is true. The difference is that clang is widely available in most 
distros; icc is not.



VLAIS is not something they are willing to accept (for various
reasons). There are other patches to LLVM that are still working

Is this not described in C99 6.7.2.1p16?
You're thinking of Flexible Array Members. Essentially the last 
element in a structure can be the equivalent of a zero length array; 
something which both gcc and clang support today.


From C99 6.7.2.1p16:
16 As a special case, the last element of a structure with more than 
one named member may
have an incomplete array type; this is called a flexible array member. 
In most situations,
the flexible array member is ignored. In particular, the size of the 
structure is as if the
flexible array member were omitted except that it may have more 
trailing padding than
the omission would imply. Howev er, when a . (or -) operator has a 
left operand that is
(a pointer to) a structure with a flexible array member and the right 
operand names that
member, it behaves as if that member were replaced with the longest 
array (with the same
element type) that would not make the structure larger than the object 
being accessed; the
offset of the array shall remain that of the flexible array member, 
even if this would differ
from that of the replacement array. If this array would have no 
elements, it behaves as if
it had one element but the behavior is undefined if any attempt is made 
to access that

element or to generate a pointer one past it.


However if we're considering the C99 standard, it also explicitly 
disallows the use of VLAs in structs (VLAIS).


From C99 6.7.2.1 p2:
2 A structure or union shall not contain a member with incomplete or 
function type (hence,
a structure shall not contain an instance of itself, but may contain a 
pointer to an instance
of itself), except that the last member of a structure with more than 
one named member
may have incomplete array type; such a structure (and any union 
containing, possibly
recursively, a member that is such a structure) shall not be a member 
of a structure or an

element of an array.

From C99 6.7.5.2 p4:

4 If the size is not present, the array type is an incomplete type. snip
Essentially all arrays in a structure must not be variable length except 
the last member of the array which can be a zero length array (which can 
be used to approximate a VLA at the end). It also says that a struct 
which uses a flexible array member can't be used inside another struct.



their way upstream that are required to be able to compile Linux as
well.

I hope the other are simple to get in :)
I hope so too. But like the Linux kernel community, the LLVM community 
have standards and procedures for getting code accepted which take time.


Thanks,

Behan

--
Behan Webster
beh...@converseincode.com

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-03 Thread Behan Webster

On 12-12-03 12:57 PM, Sebastian Andrzej Siewior wrote:

On Thu, Nov 01, 2012 at 09:21:16AM +0200, Felipe Balbi wrote:

then we can merge to net tree and handle the conflicts when merging to
Linus, that'd be fine by me as long as people know how to solve the
conflict properly ;-)

Felipe please drop this patch. I don't like this VLAIS patch and its
macro magic. I support the goal of compiling the kernel with clang but I
don't think that this is appropriate. Davem has also his problems with
the netfilter part so I am no the only one.

You are definitely not the only one. :)

But that's okay. The idea is to find an alternative that works for 
everyone. It was the first attempt, which didn't work out. 



*I* think the gadget part of the kernel is something most people won't
use so it won't show up.
Anyway, please drop this completely I try to think of something sane.
He actually didn't take the patch. He was asking the netfilter team to 
take it, and they didn't want to. Fair enough.


However, in order to approximate what gcc is doing in code, obviously 
some math is required. The thought was that macros would hide the worst 
of it, trying not to obfuscate what was actually being done.


One of the project members came up with this alternative. How about 
something like this? Less math, though more string pasting. When 
compiled, the unused variables get optimized away. Otherwise the memory 
packing is identical to using VLAIS in gcc.


#define vla_struct(structname) size_t structname##__##next = 0
#define vla_struct_size(structname) structname##__##next

#define vla_item(structname, type, name, n) \
type * structname##_##name; \
size_t structname##_##name##__##pad = \
(structname##__##next & (__alignof__(type)-1)); \
size_t structname##_##name##__##offset = \
structname##__##next + structname##_##name##__##pad; \
size_t structname##_##name##__##sz = n * sizeof(type); \
structname##__##next = structname##__##next + \
structname##_##name##__##pad + 
structname##_##name##__##sz;


#define vla_ptr(ptr,structname,name) structname##_##name = \
(typeof(structname##_##name))[structname##_##name##__##offset]


Then you can do something like this that looks vaguely struct-like:

vla_struct(foo);
vla_item(foo, char,  vara, 1);
vla_item(foo, short, varb, 10);
vla_item(foo, int,   varc, 5);
vla_item(foo, long,  vard, 3);
size_t total = vla_struct_size(foo);
char buffer[total];

vla_ptr(buffer, foo, varc);
foo_varc = 1;

I've been profiling some sample code around this implementation 
comparing it between compilers, and it approximates the code size and 
speed of using VLAIS in gcc (especially with -O2). It actually performs 
better than the previously proposed macros.


But certainly if anyone has a solution which works for everyone, then 
I'm more than happy to adopt it. The LLVM community has made quite a few 
changes in order to help get Linux working with clang. However, VLAIS is 
not something they are willing to accept (for various reasons). There 
are other patches to LLVM that are still working their way upstream that 
are required to be able to compile Linux as well.


As far as I've been told (though I have been unable to verify), Variable 
Length Arrays In Structs (VLAIS) is a feature which made it into gcc 
along with nested functions by the work on the Ada frontend. By 
contrast, the C99 standard specifies Variable Length Arrays (outside of 
structs) and Flexible Array Members (essentially zero-length arrays at 
the end of a struct), and as such both are supported by gcc and clang.


The LLVMLinux project is a meta-project between the LLVM and Linux 
communities trying to get the toolchain from one working with the code 
from the other.  The LLVMLinux project isn't trying to replace, nor 
break gcc (in fact all the project's kernel patches are tested with gcc 
as well). The idea is to bring another toolchain and set of tools to the 
table and the kernel community.


Behan

--
Behan Webster
beh...@converseincode.com

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-03 Thread Sebastian Andrzej Siewior
On Thu, Nov 01, 2012 at 09:21:16AM +0200, Felipe Balbi wrote:
> then we can merge to net tree and handle the conflicts when merging to
> Linus, that'd be fine by me as long as people know how to solve the
> conflict properly ;-)

Felipe please drop this patch. I don't like this VLAIS patch and its
macro magic. I support the goal of compiling the kernel with clang but I
don't think that this is appropriate. Davem has also his problems with
the netfilter part so I am no the only one.
*I* think the gadget part of the kernel is something most people won't
use so it won't show up.
Anyway, please drop this completely I try to think of something sane.

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-03 Thread Sebastian Andrzej Siewior
On Thu, Nov 01, 2012 at 09:21:16AM +0200, Felipe Balbi wrote:
 then we can merge to net tree and handle the conflicts when merging to
 Linus, that'd be fine by me as long as people know how to solve the
 conflict properly ;-)

Felipe please drop this patch. I don't like this VLAIS patch and its
macro magic. I support the goal of compiling the kernel with clang but I
don't think that this is appropriate. Davem has also his problems with
the netfilter part so I am no the only one.
*I* think the gadget part of the kernel is something most people won't
use so it won't show up.
Anyway, please drop this completely I try to think of something sane.

Sebastian
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-12-03 Thread Behan Webster

On 12-12-03 12:57 PM, Sebastian Andrzej Siewior wrote:

On Thu, Nov 01, 2012 at 09:21:16AM +0200, Felipe Balbi wrote:

then we can merge to net tree and handle the conflicts when merging to
Linus, that'd be fine by me as long as people know how to solve the
conflict properly ;-)

Felipe please drop this patch. I don't like this VLAIS patch and its
macro magic. I support the goal of compiling the kernel with clang but I
don't think that this is appropriate. Davem has also his problems with
the netfilter part so I am no the only one.

You are definitely not the only one. :)

But that's okay. The idea is to find an alternative that works for 
everyone. It was the first attempt, which didn't work out. shrug



*I* think the gadget part of the kernel is something most people won't
use so it won't show up.
Anyway, please drop this completely I try to think of something sane.
He actually didn't take the patch. He was asking the netfilter team to 
take it, and they didn't want to. Fair enough.


However, in order to approximate what gcc is doing in code, obviously 
some math is required. The thought was that macros would hide the worst 
of it, trying not to obfuscate what was actually being done.


One of the project members came up with this alternative. How about 
something like this? Less math, though more string pasting. When 
compiled, the unused variables get optimized away. Otherwise the memory 
packing is identical to using VLAIS in gcc.


#define vla_struct(structname) size_t structname##__##next = 0
#define vla_struct_size(structname) structname##__##next

#define vla_item(structname, type, name, n) \
type * structname##_##name; \
size_t structname##_##name##__##pad = \
(structname##__##next  (__alignof__(type)-1)); \
size_t structname##_##name##__##offset = \
structname##__##next + structname##_##name##__##pad; \
size_t structname##_##name##__##sz = n * sizeof(type); \
structname##__##next = structname##__##next + \
structname##_##name##__##pad + 
structname##_##name##__##sz;


#define vla_ptr(ptr,structname,name) structname##_##name = \
(typeof(structname##_##name))ptr[structname##_##name##__##offset]


Then you can do something like this that looks vaguely struct-like:

vla_struct(foo);
vla_item(foo, char,  vara, 1);
vla_item(foo, short, varb, 10);
vla_item(foo, int,   varc, 5);
vla_item(foo, long,  vard, 3);
size_t total = vla_struct_size(foo);
char buffer[total];

vla_ptr(buffer, foo, varc);
foo_varc = 1;

I've been profiling some sample code around this implementation 
comparing it between compilers, and it approximates the code size and 
speed of using VLAIS in gcc (especially with -O2). It actually performs 
better than the previously proposed macros.


But certainly if anyone has a solution which works for everyone, then 
I'm more than happy to adopt it. The LLVM community has made quite a few 
changes in order to help get Linux working with clang. However, VLAIS is 
not something they are willing to accept (for various reasons). There 
are other patches to LLVM that are still working their way upstream that 
are required to be able to compile Linux as well.


As far as I've been told (though I have been unable to verify), Variable 
Length Arrays In Structs (VLAIS) is a feature which made it into gcc 
along with nested functions by the work on the Ada frontend. By 
contrast, the C99 standard specifies Variable Length Arrays (outside of 
structs) and Flexible Array Members (essentially zero-length arrays at 
the end of a struct), and as such both are supported by gcc and clang.


The LLVMLinux project is a meta-project between the LLVM and Linux 
communities trying to get the toolchain from one working with the code 
from the other.  The LLVMLinux project isn't trying to replace, nor 
break gcc (in fact all the project's kernel patches are tested with gcc 
as well). The idea is to bring another toolchain and set of tools to the 
table and the kernel community.


Behan

--
Behan Webster
beh...@converseincode.com

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-11-01 Thread David Miller
From: Felipe Balbi 
Date: Thu, 1 Nov 2012 09:21:16 +0200

> then we can merge to net tree and handle the conflicts when merging to
> Linus, that'd be fine by me as long as people know how to solve the
> conflict properly ;-)

Like Herbert with the crypto tree, I'm simply not merging this
absolute crap into my networking tree.

These changes are totally not appropriate.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-11-01 Thread Felipe Balbi
Hi,

On Wed, Oct 31, 2012 at 11:33:20AM -0400, Behan Webster wrote:
> On 12-10-31 09:28 AM, Felipe Balbi wrote:
> >hi,
> >
> >On Tue, Oct 30, 2012 at 05:18:56PM -0400, Behan Webster wrote:
> >>The use of variable length arrays in structs (VLAIS) in the Linux Kernel 
> >>code
> >>precludes the use of compilers which don't implement VLAIS (for instance the
> >>Clang compiler). This patch instead calculates offsets into the kmalloc-ed
> >>memory buffer using macros from valign.h.
> >>
> >>Signed-off-by: Behan Webster 
> >this won't apply after the current cleanups I applied to gadget code
> >from Sebastian.
> Makes sense. I'll try it with your repo, and regenerate.
> 
> >If someone takes this patch, it will generate a series of annoying,
> >hard-to-figure-out conflicts (at least judging by the looks of
> >$SUBJECT).
> I just tried the patch on your git.kernel.org repo and thankfully
> there is only one hunk which is rejected, and fortunately the reason
> is trivial (descriptors -> fs_descriptors).
> 
> Was:
> -func->function.descriptors = data->fs_descs;
> +func->function.descriptors = fs_descs;
> 
> Now is:
> -func->function.fs_descriptors = data->fs_descs;
> +func->function.fs_descriptors = fs_descs;
> 
> I will regenerate the patch set, but obviously the new gadget patch
> in the V3 patchset will only apply to the USB repo, and not to the
> netfilter repo.

then we can merge to net tree and handle the conflicts when merging to
Linus, that'd be fine by me as long as people know how to solve the
conflict properly ;-)

-- 
balbi


signature.asc
Description: Digital signature


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-11-01 Thread Felipe Balbi
Hi,

On Wed, Oct 31, 2012 at 11:33:20AM -0400, Behan Webster wrote:
 On 12-10-31 09:28 AM, Felipe Balbi wrote:
 hi,
 
 On Tue, Oct 30, 2012 at 05:18:56PM -0400, Behan Webster wrote:
 The use of variable length arrays in structs (VLAIS) in the Linux Kernel 
 code
 precludes the use of compilers which don't implement VLAIS (for instance the
 Clang compiler). This patch instead calculates offsets into the kmalloc-ed
 memory buffer using macros from valign.h.
 
 Signed-off-by: Behan Webster beh...@converseincode.com
 this won't apply after the current cleanups I applied to gadget code
 from Sebastian.
 Makes sense. I'll try it with your repo, and regenerate.
 
 If someone takes this patch, it will generate a series of annoying,
 hard-to-figure-out conflicts (at least judging by the looks of
 $SUBJECT).
 I just tried the patch on your git.kernel.org repo and thankfully
 there is only one hunk which is rejected, and fortunately the reason
 is trivial (descriptors - fs_descriptors).
 
 Was:
 -func-function.descriptors = data-fs_descs;
 +func-function.descriptors = fs_descs;
 
 Now is:
 -func-function.fs_descriptors = data-fs_descs;
 +func-function.fs_descriptors = fs_descs;
 
 I will regenerate the patch set, but obviously the new gadget patch
 in the V3 patchset will only apply to the USB repo, and not to the
 netfilter repo.

then we can merge to net tree and handle the conflicts when merging to
Linus, that'd be fine by me as long as people know how to solve the
conflict properly ;-)

-- 
balbi


signature.asc
Description: Digital signature


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-11-01 Thread David Miller
From: Felipe Balbi ba...@ti.com
Date: Thu, 1 Nov 2012 09:21:16 +0200

 then we can merge to net tree and handle the conflicts when merging to
 Linus, that'd be fine by me as long as people know how to solve the
 conflict properly ;-)

Like Herbert with the crypto tree, I'm simply not merging this
absolute crap into my networking tree.

These changes are totally not appropriate.
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-10-31 Thread Behan Webster

On 12-10-31 09:28 AM, Felipe Balbi wrote:

hi,

On Tue, Oct 30, 2012 at 05:18:56PM -0400, Behan Webster wrote:

The use of variable length arrays in structs (VLAIS) in the Linux Kernel code
precludes the use of compilers which don't implement VLAIS (for instance the
Clang compiler). This patch instead calculates offsets into the kmalloc-ed
memory buffer using macros from valign.h.

Signed-off-by: Behan Webster 

this won't apply after the current cleanups I applied to gadget code
from Sebastian.

Makes sense. I'll try it with your repo, and regenerate.


If someone takes this patch, it will generate a series of annoying,
hard-to-figure-out conflicts (at least judging by the looks of
$SUBJECT).
I just tried the patch on your git.kernel.org repo and thankfully there 
is only one hunk which is rejected, and fortunately the reason is 
trivial (descriptors -> fs_descriptors).


Was:
-func->function.descriptors = data->fs_descs;
+func->function.descriptors = fs_descs;

Now is:
-func->function.fs_descriptors = data->fs_descs;
+func->function.fs_descriptors = fs_descs;

I will regenerate the patch set, but obviously the new gadget patch in 
the V3 patchset will only apply to the USB repo, and not to the 
netfilter repo.


Thanks,

Behan

--
Behan Webster
beh...@converseincode.com

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-10-31 Thread Felipe Balbi
hi,

On Tue, Oct 30, 2012 at 05:18:56PM -0400, Behan Webster wrote:
> The use of variable length arrays in structs (VLAIS) in the Linux Kernel code
> precludes the use of compilers which don't implement VLAIS (for instance the
> Clang compiler). This patch instead calculates offsets into the kmalloc-ed
> memory buffer using macros from valign.h.
> 
> Signed-off-by: Behan Webster 

this won't apply after the current cleanups I applied to gadget code
from Sebastian.

If someone takes this patch, it will generate a series of annoying,
hard-to-figure-out conflicts (at least judging by the looks of
$SUBJECT).

-- 
balbi


signature.asc
Description: Digital signature


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-10-31 Thread Felipe Balbi
hi,

On Tue, Oct 30, 2012 at 05:18:56PM -0400, Behan Webster wrote:
 The use of variable length arrays in structs (VLAIS) in the Linux Kernel code
 precludes the use of compilers which don't implement VLAIS (for instance the
 Clang compiler). This patch instead calculates offsets into the kmalloc-ed
 memory buffer using macros from valign.h.
 
 Signed-off-by: Behan Webster beh...@converseincode.com

this won't apply after the current cleanups I applied to gadget code
from Sebastian.

If someone takes this patch, it will generate a series of annoying,
hard-to-figure-out conflicts (at least judging by the looks of
$SUBJECT).

-- 
balbi


signature.asc
Description: Digital signature


Re: [PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-10-31 Thread Behan Webster

On 12-10-31 09:28 AM, Felipe Balbi wrote:

hi,

On Tue, Oct 30, 2012 at 05:18:56PM -0400, Behan Webster wrote:

The use of variable length arrays in structs (VLAIS) in the Linux Kernel code
precludes the use of compilers which don't implement VLAIS (for instance the
Clang compiler). This patch instead calculates offsets into the kmalloc-ed
memory buffer using macros from valign.h.

Signed-off-by: Behan Webster beh...@converseincode.com

this won't apply after the current cleanups I applied to gadget code
from Sebastian.

Makes sense. I'll try it with your repo, and regenerate.


If someone takes this patch, it will generate a series of annoying,
hard-to-figure-out conflicts (at least judging by the looks of
$SUBJECT).
I just tried the patch on your git.kernel.org repo and thankfully there 
is only one hunk which is rejected, and fortunately the reason is 
trivial (descriptors - fs_descriptors).


Was:
-func-function.descriptors = data-fs_descs;
+func-function.descriptors = fs_descs;

Now is:
-func-function.fs_descriptors = data-fs_descs;
+func-function.fs_descriptors = fs_descs;

I will regenerate the patch set, but obviously the new gadget patch in 
the V3 patchset will only apply to the USB repo, and not to the 
netfilter repo.


Thanks,

Behan

--
Behan Webster
beh...@converseincode.com

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-10-30 Thread Behan Webster
The use of variable length arrays in structs (VLAIS) in the Linux Kernel code
precludes the use of compilers which don't implement VLAIS (for instance the
Clang compiler). This patch instead calculates offsets into the kmalloc-ed
memory buffer using macros from valign.h.

Signed-off-by: Behan Webster 
---
 drivers/usb/gadget/f_fs.c |  106 ++---
 1 file changed, 61 insertions(+), 45 deletions(-)

diff --git a/drivers/usb/gadget/f_fs.c b/drivers/usb/gadget/f_fs.c
index 64c4ec1..ca6cacc 100644
--- a/drivers/usb/gadget/f_fs.c
+++ b/drivers/usb/gadget/f_fs.c
@@ -22,6 +22,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 
 #include 
@@ -1875,6 +1876,15 @@ error:
return ret;
 }
 
+static void __ffs_init_stringtabs(struct usb_gadget_strings **stringtabs,
+   struct usb_gadget_strings *t, unsigned len)
+{
+   do {
+   *stringtabs++ = t++;
+   } while (--len);
+   *stringtabs = NULL;
+}
+
 static int __ffs_data_got_strings(struct ffs_data *ffs,
  char *const _data, size_t len)
 {
@@ -1911,31 +1921,26 @@ static int __ffs_data_got_strings(struct ffs_data *ffs,
 
/* Allocate everything in one chunk so there's less maintenance. */
{
-   struct {
-   struct usb_gadget_strings *stringtabs[lang_count + 1];
-   struct usb_gadget_strings stringtab[lang_count];
-   struct usb_string strings[lang_count*(needed_count+1)];
-   } *d;
-   unsigned i = 0;
-
-   d = kmalloc(sizeof *d, GFP_KERNEL);
+   int strtabslen = paddedsize(0, lang_count+1,
+   struct usb_gadget_strings *, struct usb_gadget_strings);
+   int strtablen = paddedsize(strtabslen, lang_count,
+   struct usb_gadget_strings, struct usb_string);
+   int strlen = paddedsize(strtabslen + strtablen,
+   lang_count*(needed_count+1), struct usb_string, int);
+
+   stringtabs = kmalloc(strtabslen + strtablen + strlen,
+   GFP_KERNEL);
if (unlikely(!d)) {
kfree(_data);
return -ENOMEM;
}
 
-   stringtabs = d->stringtabs;
-   t = d->stringtab;
-   i = lang_count;
-   do {
-   *stringtabs++ = t++;
-   } while (--i);
-   *stringtabs = NULL;
-
-   stringtabs = d->stringtabs;
-   t = d->stringtab;
-   s = d->strings;
+   t = paddedstart(stringtabs, strtabslen,
+   struct usb_gadget_strings);
+   s = paddedstart(t, strtablen, struct usb_string);
strings = s;
+
+   __ffs_init_stringtabs(stringtabs, t, lang_count);
}
 
/* For each language */
@@ -2209,17 +2214,23 @@ static int ffs_func_bind(struct usb_configuration *c,
 
int ret;
 
-   /* Make it a single chunk, less management later on */
-   struct {
-   struct ffs_ep eps[ffs->eps_count];
-   struct usb_descriptor_header
-   *fs_descs[full ? ffs->fs_descs_count + 1 : 0];
-   struct usb_descriptor_header
-   *hs_descs[high ? ffs->hs_descs_count + 1 : 0];
-   short inums[ffs->interfaces_count];
-   char raw_descs[high ? ffs->raw_descs_length
-   : ffs->raw_fs_descs_length];
-   } *data;
+   struct ffs_ep *eps;
+   struct usb_descriptor_header **fs_descs;
+   struct usb_descriptor_header **hs_descs;
+   short *inums;
+   char *raw_descs;
+
+   int epslen = paddedsize(0, ffs->eps_count,
+   struct ffs_ep, struct usb_descriptor_header);
+   int fsdlen = paddedsize(epslen, full ? ffs->fs_descs_count + 1 : 0,
+   struct usb_descriptor_header,
+   struct usb_descriptor_header);
+   int hsdlen = paddedsize(fsdlen, high ? ffs->hs_descs_count + 1 : 0,
+   struct usb_descriptor_header, short);
+   int inumlen = paddedsize(hsdlen, ffs->interfaces_count, short, char);
+   int rawlen = paddedsize(inumlen,
+   high ? ffs->raw_descs_length : ffs->raw_fs_descs_length,
+   char, int);
 
ENTER();
 
@@ -2227,21 +2238,28 @@ static int ffs_func_bind(struct usb_configuration *c,
if (unlikely(!(full | high)))
return -ENOTSUPP;
 
-   /* Allocate */
-   data = kmalloc(sizeof *data, GFP_KERNEL);
+   /* Allocate a single chunk, less management later on */
+   eps = kmalloc(epslen+fsdlen+hsdlen+inumlen+rawlen, GFP_KERNEL);
if (unlikely(!data))
return -ENOMEM;
 
+   /* Calculate start of each variable in the 

[PATCH V2 2/3] Remove VLAIS usage from gadget code

2012-10-30 Thread Behan Webster
The use of variable length arrays in structs (VLAIS) in the Linux Kernel code
precludes the use of compilers which don't implement VLAIS (for instance the
Clang compiler). This patch instead calculates offsets into the kmalloc-ed
memory buffer using macros from valign.h.

Signed-off-by: Behan Webster beh...@converseincode.com
---
 drivers/usb/gadget/f_fs.c |  106 ++---
 1 file changed, 61 insertions(+), 45 deletions(-)

diff --git a/drivers/usb/gadget/f_fs.c b/drivers/usb/gadget/f_fs.c
index 64c4ec1..ca6cacc 100644
--- a/drivers/usb/gadget/f_fs.c
+++ b/drivers/usb/gadget/f_fs.c
@@ -22,6 +22,7 @@
 #include linux/pagemap.h
 #include linux/export.h
 #include linux/hid.h
+#include linux/valign.h
 #include asm/unaligned.h
 
 #include linux/usb/composite.h
@@ -1875,6 +1876,15 @@ error:
return ret;
 }
 
+static void __ffs_init_stringtabs(struct usb_gadget_strings **stringtabs,
+   struct usb_gadget_strings *t, unsigned len)
+{
+   do {
+   *stringtabs++ = t++;
+   } while (--len);
+   *stringtabs = NULL;
+}
+
 static int __ffs_data_got_strings(struct ffs_data *ffs,
  char *const _data, size_t len)
 {
@@ -1911,31 +1921,26 @@ static int __ffs_data_got_strings(struct ffs_data *ffs,
 
/* Allocate everything in one chunk so there's less maintenance. */
{
-   struct {
-   struct usb_gadget_strings *stringtabs[lang_count + 1];
-   struct usb_gadget_strings stringtab[lang_count];
-   struct usb_string strings[lang_count*(needed_count+1)];
-   } *d;
-   unsigned i = 0;
-
-   d = kmalloc(sizeof *d, GFP_KERNEL);
+   int strtabslen = paddedsize(0, lang_count+1,
+   struct usb_gadget_strings *, struct usb_gadget_strings);
+   int strtablen = paddedsize(strtabslen, lang_count,
+   struct usb_gadget_strings, struct usb_string);
+   int strlen = paddedsize(strtabslen + strtablen,
+   lang_count*(needed_count+1), struct usb_string, int);
+
+   stringtabs = kmalloc(strtabslen + strtablen + strlen,
+   GFP_KERNEL);
if (unlikely(!d)) {
kfree(_data);
return -ENOMEM;
}
 
-   stringtabs = d-stringtabs;
-   t = d-stringtab;
-   i = lang_count;
-   do {
-   *stringtabs++ = t++;
-   } while (--i);
-   *stringtabs = NULL;
-
-   stringtabs = d-stringtabs;
-   t = d-stringtab;
-   s = d-strings;
+   t = paddedstart(stringtabs, strtabslen,
+   struct usb_gadget_strings);
+   s = paddedstart(t, strtablen, struct usb_string);
strings = s;
+
+   __ffs_init_stringtabs(stringtabs, t, lang_count);
}
 
/* For each language */
@@ -2209,17 +2214,23 @@ static int ffs_func_bind(struct usb_configuration *c,
 
int ret;
 
-   /* Make it a single chunk, less management later on */
-   struct {
-   struct ffs_ep eps[ffs-eps_count];
-   struct usb_descriptor_header
-   *fs_descs[full ? ffs-fs_descs_count + 1 : 0];
-   struct usb_descriptor_header
-   *hs_descs[high ? ffs-hs_descs_count + 1 : 0];
-   short inums[ffs-interfaces_count];
-   char raw_descs[high ? ffs-raw_descs_length
-   : ffs-raw_fs_descs_length];
-   } *data;
+   struct ffs_ep *eps;
+   struct usb_descriptor_header **fs_descs;
+   struct usb_descriptor_header **hs_descs;
+   short *inums;
+   char *raw_descs;
+
+   int epslen = paddedsize(0, ffs-eps_count,
+   struct ffs_ep, struct usb_descriptor_header);
+   int fsdlen = paddedsize(epslen, full ? ffs-fs_descs_count + 1 : 0,
+   struct usb_descriptor_header,
+   struct usb_descriptor_header);
+   int hsdlen = paddedsize(fsdlen, high ? ffs-hs_descs_count + 1 : 0,
+   struct usb_descriptor_header, short);
+   int inumlen = paddedsize(hsdlen, ffs-interfaces_count, short, char);
+   int rawlen = paddedsize(inumlen,
+   high ? ffs-raw_descs_length : ffs-raw_fs_descs_length,
+   char, int);
 
ENTER();
 
@@ -2227,21 +2238,28 @@ static int ffs_func_bind(struct usb_configuration *c,
if (unlikely(!(full | high)))
return -ENOTSUPP;
 
-   /* Allocate */
-   data = kmalloc(sizeof *data, GFP_KERNEL);
+   /* Allocate a single chunk, less management later on */
+   eps = kmalloc(epslen+fsdlen+hsdlen+inumlen+rawlen, GFP_KERNEL);
if