Re: target hooks / plugins

[Joseph S. Myers]

On Wed, 13 Jan 2010, Joern Rennecke wrote:

> Duplicating all these changes separately by hand seems nigh impossible.
> I think the best approach is then to take the auto-generated tm.texi as
> the new tm.texi, and packages it up as a patch together with the
> struct member / hook name changes that I made for consistency.
> 
> There is only one issue with using the current auto-generated tm.texi:
> Unless special formatting was in force (e.g. @smallexample), I've removed
> intra-paragraph newlines.  This should work in principle just as will
> as with these newlines for producing output, but it looks somewhat daft
> in tm.texi when you consider it as a source file.

I am not particularly concerned about newlines, if the file is identical 
apart from whitespace.  But your text file lists things such as "was 
undocumented" and "Fixed return value description".  Each such change 
needs its own review, by someone familiar with the relevant part of the 
compiler, and needs its own explanation of the problem posted.  Remember, 
a patch should not contain multiple changes that can logically be 
considered separately, and in this case I expect many different people to 
be appropriate reviewers for changes relating to different hooks, so it's 
important not to mix changes relating to hooks in different areas of the 
compiler.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: target hooks / plugins

[Joern Rennecke]

Quoting "Joseph S. Myers" :


Please note that your initial change to implement automatic doc extraction
should not result in any changes to the Texinfo content of the manual.
Such fixes should all go in either before or after the automatic doc
extraction change, but not at the same time; the doc extraction change
should result in identical text in the manual, but with the Texinfo files
produced in a different way.  I recommend sending such fixes before the
automatic doc extraction change, since they do not depend on the FSF doing
anything.


Duplicating all these changes separately by hand seems nigh impossible.
I think the best approach is then to take the auto-generated tm.texi as
the new tm.texi, and packages it up as a patch together with the
struct member / hook name changes that I made for consistency.

There is only one issue with using the current auto-generated tm.texi:
Unless special formatting was in force (e.g. @smallexample), I've removed
intra-paragraph newlines.  This should work in principle just as will
as with these newlines for producing output, but it looks somewhat daft
in tm.texi when you consider it as a source file.
Putting newlines in the input file would make it harder to read & edit,
since the documentation comes as C strings - and the GNU multiline string
extension has been deprecated some time ago.
I could pipe the documentation through fold -s, but that would also fold
extra-long lines outside the hook documentation, e.g.:
@@ -28,7 +28,8 @@
 @menu
 * Target Structure::The @code{targetm} variable.
 * Driver::  Controlling how the driver runs the  
compilation passes.
-* Run-time Target:: Defining @samp{-m} options like  
@option{-m68000} and @option{-m68020}.

+* Run-time Target:: Defining @samp{-m} options like @option{-m68000} and
+...@option{-m68020}.
 * Per-Function Data::   Defining data structures for per-function  
information.

 * Storage Layout::  Defining sizes and alignments of data.
 * Type Layout:: Defining sizes and properties of basic user  
data types.


So I suppose I'll have to add some fold mechanism into the documentation
output code.

Re: target hooks / plugins

[Joseph S. Myers]

On Wed, 13 Jan 2010, Joern Rennecke wrote:

> Quoting "Joseph S. Myers" :
> > If you want to have documentation extracted from source files, you need to
> > engage with the SC and FSF at an early stage to get suitable license
> > exception wording to permit the relevant text to be used in the manuals as
> > well as the main (GPL) sources.
> 
> I haven't gotten any reply to my proposal from the 5th of January
> http://gcc.gnu.org/ml/gcc/2010-01/msg00082.html
> 
> Is the GCC mailing list no longer the right place to reach the steering
> committee?

It is the right place, but you should start a new thread with just the 
relevant issue and an appropriate subject line marked for the attention of 
the SC.  You should expect it to take at least several months for the FSF 
to prepare an exception (worded as additional permissions under Section 7 
of GPL version 3, like COPYING.RUNTIME, *not* as an old-style "As a 
special exception"), maybe years, if the SC persuades them that there 
should be such an exception.

> > Of course, the ordering and (especially)
> > section divisions in the internals manual are human-designed not arbitrary
> > so you need to retain the ability to put the documentation of each hook in
> > an appropriate position in an appropriate section of the manual.
> 
> I have posted my current patch here:
> http://gcc.gnu.org/ml/gcc-patches/2010-01/msg00559.html
> 
> > If you can make this work it should reduce the risk of people not
> > documenting new hooks
> 
> We actually have a number of instances where the target hook documentation is
> out of sync with the sources.  I have attached a file with my notes on this.

Please note that your initial change to implement automatic doc extraction 
should not result in any changes to the Texinfo content of the manual.  
Such fixes should all go in either before or after the automatic doc 
extraction change, but not at the same time; the doc extraction change 
should result in identical text in the manual, but with the Texinfo files 
produced in a different way.  I recommend sending such fixes before the 
automatic doc extraction change, since they do not depend on the FSF doing 
anything.

> > , but for it to work the FSF agreement is critical
> 
> So who do I have to talk to for this?

The SC.  Not in the middle of a technical thread, but a thread on its own 
marked for the SC and indicating exactly what you want them to raise with 
the FSF.

> tm.texi is half a megabyte, so it would be nice not to have it as a generated
> file in the repository.  Also, it'd be annoying to have to regenerate & check
> it in for every target.def change even though the build works fine with
> the generated file in the build directory.
> Will it be acceptable to have update_web_docs build a generator program
> (written in C) to rebuild tm.texi?

It already builds a generator program in Ada for the Ada manual, so yes.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: target hooks / plugins

[Joern Rennecke]

Quoting "Joseph S. Myers" :

If you want to have documentation extracted from source files, you need to
engage with the SC and FSF at an early stage to get suitable license
exception wording to permit the relevant text to be used in the manuals as
well as the main (GPL) sources.


I haven't gotten any reply to my proposal from the 5th of January
http://gcc.gnu.org/ml/gcc/2010-01/msg00082.html

Is the GCC mailing list no longer the right place to reach the  
steering committee?



 Of course, the ordering and (especially)
section divisions in the internals manual are human-designed not arbitrary
so you need to retain the ability to put the documentation of each hook in
an appropriate position in an appropriate section of the manual.


I have posted my current patch here:
http://gcc.gnu.org/ml/gcc-patches/2010-01/msg00559.html


If you can make this work it should reduce the risk of people not
documenting new hooks


We actually have a number of instances where the target hook documentation is
out of sync with the sources.  I have attached a file with my notes on this.


, but for it to work the FSF agreement is critical


So who do I have to talk to for this?


as
is ensuring it genuinely works as well as the existing system without
regressions (including working in update_web_docs).


tm.texi is half a megabyte, so it would be nice not to have it as a generated
file in the repository.  Also, it'd be annoying to have to regenerate & check
it in for every target.def change even though the build works fine with
the generated file in the build directory.
Will it be acceptable to have update_web_docs build a generator program
(written in C) to rebuild tm.texi?
various hooks: why are we referring to 'the diagnostic message'?
the context does not suggest a particular one, and I thought we had more
than one in gcc.

various documentation used rtx / tree instead of const_rtx / const_tree.

TARGET_ASM_ASSEMBLE_VISIBILITY: const char *...@var{visibility} -> int
clarified current_function_pops_args description and removed comment on
unclear "its arguments" reference.
TARGET_ASM_NAMED_SECTION unsigned int @var{align} -> tree decl
TARGET_ASM_MARK_DECL_PRESERVED tree @var{decl} -> const char *symbol
TARGET_SCHED_ADJUST_COST_2 was undocumented.
TARGET_SCHED_SPECULATE_INSN: Fixed return value description.
TARGET_SCHED_NEEDS_BLOCK_P: int -> bool; rtx @var{insn} -> int dep_status (text 
is still bogus)
TARGET_SCHED_GEN_CHECK -> TARGET_SCHED_GEN_SPEC_CHECK
TARGET_SCHED_SET_SCHED_FLAGS unsigned int *...@var{flags} -> /dev/null
TARGET_VECTORIZE_BUILTIN_VECTORIZED_FUNCTION enum built_in_function @var{code} 
-> unsigned
TARGET_VECTORIZE_BUILTIN_VECTORIZATION_COST was undocumented.
TARGET_[VECTORIZE_]VECTOR_ALIGNMENT_REACHABLE was undocumented.
TARGET_VECTORIZE_BUILTIN_VEC_PERM was undocumented.
TARGET_VECTORIZE_BUILTIN_VEC_PERM_OK was undocumented.
TARGET_EH_RETURN_FILTER_MODE was undocumented.
TARGET_UNWIND_WORD_MODE was undocumented.
TARGET_BUILTIN_FUNCTION -> TARGET_BUILTIN_DECL
TARGET_RESOLVE_OVERLOADED_BUILTIN addded unsigned int /*location_t*/ loc; tree 
@var{arglist} -> void *arglist
TARGET_BUILTIN_RECIPROCAL enum tree_code @var{fn} -> unsigned fn
TARGET_CANNOT_COPY_INSN_P was undocumented.
TARGET_EXPAND_BUILTIN_VA_START was undocumented.
TARGET_GIMPLIFY_VA_ARG_EXPR tree *...@var{pre_p}, tree *...@var{post_p} -> 
gimple_seq *pre_p, gimple_seq *post_p
TARGET_STDARG_OPTIMIZE_HOOK was undocumented.
TARGET_INTERNAL_ARG_POINTER was undocumented.
TARGET_GET_DRAP_RTX: text was garbled, e.g. confusion macro vs. hook.
TARGET_C_MODE_FOR_SUFFIX was undocumented.
TARGET_VALID_OPTION_ATTRIBUTE_P -> TARGET_OPTION_VALID_ATTRIBUTE_P
TARGET_OPTION_PRINT: new parameters FILE *file, int indent
TARGET_OPTION_PRAGMA_PARSE: (target @var{args}) -> tree, tree
TARGET_EXTRA_LIVE_ON_ENTRY: (bitmap *...@var{regs}) -> bitmap
TARGET_HANDLE_PRAGMA_EXTERN_PREFIX was undocumented.
TARGET_HELP bool -> void
TARGET_BUILTIN_SETJMP_FRAME_VALUE bool -> rtx
TARGET_PRETEND_OUTGOING_VARARGS_NAMED: add (CUMULATIVE_ARGS *...@var{ca})
TARGET_LEGITIMATE_ADDRESS_P: returns bool
TARGET_SCHED_REORDER2 @var{clock} -> int clock
TARGET_SCHED_DFA_PRE_CYCLE_INSN int -> rtx
TARGET_SCHED_DFA_POST_CYCLE_INSN int -> rtx
TARGET_SCHED_DFA_PRE_CYCLE_ADVANCE -> TARGET_SCHED_DFA_PRE_ADVANCE_CYCLE
TARGET_SCHED_DFA_POST_CYCLE_ADVANCE -> TARGET_SCHED_DFA_POST_ADVANCE_CYCLE
TARGET_SCHED_ALLOC_SCHED_CONTEXT, TARGET_SCHED_INIT_SCHED_CONTEXT, 
TARGET_SCHED_SET_SCHED_CONTEXT, TARGET_SCHED_CLEAR_SCHED_CONTEXT, 
TARGET_SCHED_FREE_SCHED_CONTEXT were duplicated.
TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC int -> bool
TARGET_SCHED_GET_INSN_SPEC_DS was undocumented.
TARGET_SCHED_GET_INSN_CHECKED_DS was undocumented.
TARGET_SCHED_SKIP_RTX_P was undocumented.
TARGET_ASM_RELOC_RW_MASK: 'returns' void.
TARGET_MANGLE_DECL_ASSEMBLER_NAME: void -> tree
TARGET_ASM_EMIT_UNWIND_LABEL was missing types.
Why is the TARGET_DWARF_CALLING_CONVENTION documentation nested in the
DWARF2_DEBUGGING_IN

Re: target hooks / plugins

[Joern Rennecke]

Quoting "Joseph S. Myers" :


On Wed, 23 Dec 2009, Joern Rennecke wrote:


I've attached what I have so far.


If you want to have documentation extracted from source files, you need to
engage with the SC and FSF at an early stage to get suitable license
exception wording to permit the relevant text to be used in the manuals as
well as the main (GPL) sources.


I suppose I'm reaching the steering commitee by posting to g...@gcc.gnu.org.
What email address should I use to engage the FSF?

I want to move the hook descriptions from tm.texi into target.def .
gentarget.c includes target.def, and gentarget (re-)generates
$(srcdir)/doc/targhooks.texi (the makefile dependency would be keyed on
--enable-maintainer-mode) ; targhooks.texi would then include the hook
descriptions from target.def .

As the person who regenerates targhooks.texi is in possession of target.def ,
I think the (s)he can use the license on target.def to generate a GFDL
(terms according to gcc.texi) targhooks.texi which includes pieces of  
target.def, without requiring the gentarget executable to have a special

license; the executable is only a means to create a (documentation) work
based on target.def .

I've looked at the bison skeleton for inspiration on the license blurb
for target.def .
This is what I have come up with:

/* Target hook definitions.
   Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010
   Free Software Foundation, Inc.

   This program is free software; you can redistribute it and/or modify it
   under the terms of the GNU General Public License as published by the
   Free Software Foundation; either version 3, or (at your option) any
   later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License for more details.

   You should have received a copy of the GNU General Public License
   along with this program; see the file COPYING3.  If not see
   .

   In other words, you are welcome to use, share and improve this program.
   You are forbidden to forbid anyone else to use, share and improve
   what you give them.   Help stamp out software-hoarding!  */

/* As a special exception, you may create documentation that contains
   part or all of this target hooks definitions file and distribute that
   work under the copying terms outlined in doc/gcc.texi .
   Alternatively, if you modify or redistribute this target hooks
   definitions file itself, you may (at your option) remove this special
   exception, which will cause the target hooks definitions file (and any
   program output which incorporates parts of this file) to be licensed
   under the GNU General Public License without this special exception.  */



Of course, the ordering and (especially)
section divisions in the internals manual are human-designed not arbitrary
so you need to retain the ability to put the documentation of each hook in
an appropriate position in an appropriate section of the manual.


Yes, I plan to allow any number of explicit placements of hooks descriptions
in the resulting documentation, and any hooks that are not explicitly placed
will appear as successor of their predecessor in target.def .

RE: target hooks / plugins

[Grigori Fursin]

Hi Joseph,
cTuning-discussions is an open public mailing list -
I have been moderating lots of spam there recently and
mixed up some settings but it is fixed now ...
Cheers,
Grigori

> -Original Message-
> From: Joseph Myers [mailto:jos...@codesourcery.com]
> Sent: Thursday, December 24, 2009 1:26 PM
> To: Joern Rennecke
> Cc: 'GCC Mailing List'; Grigori Fursin; 'Yuanjie Huang'; 'Liang Peng'; 
> 'Zbigniew Chamski';
> 'Yuri Kashnikoff'; 'Diego Novillo'
> Subject: Re: target hooks / plugins
> 
> It appears you CC:ed your message to a closed mailing list
> ctuning-discussi...@googlegroups.com that bounces messages from
> non-subscribers.  Please avoid doing this; messages to public lists should
> not be CC:ed to any list that will send bounces or other automatic replies
> to non-subscribers.  You are free to forward GCC list messages manually to
> such a list, of course.
> 
> --
> Joseph S. Myers
> jos...@codesourcery.com

Re: target hooks / plugins

[Richard Guenther]

On Thu, Dec 24, 2009 at 1:32 AM, Joern Rennecke  wrote:
> Quoting Joern Rennecke :
>>
>> Right now, to make a new target hook, you have to add a new field in
>> target.h, define a new default in target-def.h, place the new macro
>> in exactly the right position there of the right initializer macro,
>> describe the new hook in tm.texi, and if you need a new function with
>> a bunch of parameters returning a constant, you have to add this to
>> hooks.c .
>>
>> I would like to be able to do all this by adding a single entry in a
>> new definition file; and the information should also be usable by
>> plugin sources so that they can automatically make wrappers for all
>> function-type hooks.
>
> I've attached what I have so far.
> There is an issue that the struct gcc_target member names don't always agree
> with the TARGET_* macro names.
>
> Should I rather change one or the other to make them agree, or add an extra
> parameter in the definition file macros to specify these names
> independently?

I didn't look at the patch yet but I'd simply change one or the other to
make the names consistent.

Richard.

>
>

Re: target hooks / plugins

[Joseph S. Myers]

It appears you CC:ed your message to a closed mailing list 
ctuning-discussi...@googlegroups.com that bounces messages from 
non-subscribers.  Please avoid doing this; messages to public lists should 
not be CC:ed to any list that will send bounces or other automatic replies 
to non-subscribers.  You are free to forward GCC list messages manually to 
such a list, of course.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: target hooks / plugins

[Joseph S. Myers]

On Wed, 23 Dec 2009, Joern Rennecke wrote:

> I've attached what I have so far.

If you want to have documentation extracted from source files, you need to 
engage with the SC and FSF at an early stage to get suitable license 
exception wording to permit the relevant text to be used in the manuals as 
well as the main (GPL) sources.  Of course, the ordering and (especially) 
section divisions in the internals manual are human-designed not arbitrary 
so you need to retain the ability to put the documentation of each hook in 
an appropriate position in an appropriate section of the manual.

If you can make this work it should reduce the risk of people not 
documenting new hooks, but for it to work the FSF agreement is critical as 
is ensuring it genuinely works as well as the existing system without 
regressions (including working in update_web_docs).

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: target hooks / plugins

[Joern Rennecke]

Quoting Joern Rennecke :

Right now, to make a new target hook, you have to add a new field in
target.h, define a new default in target-def.h, place the new macro
in exactly the right position there of the right initializer macro,
describe the new hook in tm.texi, and if you need a new function with
a bunch of parameters returning a constant, you have to add this to
hooks.c .

I would like to be able to do all this by adding a single entry in a
new definition file; and the information should also be usable by
plugin sources so that they can automatically make wrappers for all
function-type hooks.


I've attached what I have so far.
There is an issue that the struct gcc_target member names don't always agree
with the TARGET_* macro names.

Should I rather change one or the other to make them agree, or add an extra
parameter in the definition file macros to specify these names independently?


Index: target.def
===
--- target.def  (revision 0)
+++ target.def  (revision 0)
@@ -0,0 +1,263 @@
+/* Target hook definitions.
+   Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009
+   Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify it
+   under the terms of the GNU General Public License as published by the
+   Free Software Foundation; either version 3, or (at your option) any
+   later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; see the file COPYING3.  If not see
+   .
+
+   In other words, you are welcome to use, share and improve this program.
+   You are forbidden to forbid anyone else to use, share and improve
+   what you give them.   Help stamp out software-hoarding!  */
+
+/* The following macros should be provided by the including file:
+
+   DEFHOOK(NAME, DOC, TYPE, PARAMS, INIT): Define a function-valued hook.
+   DEFHOOKPOD(DOC, TYPE, NAME, INIT): Define a piece-of-data 'hook'.  */
+
+/* Defaults for optional macros:
+   DEFHOOKPODX(NAME, NAME, INIT): Like DEFHOOKPOD, but share documentation
+   with the previous 'hook'.  */
+#ifndef DEFHOOKPODX
+#define DEFHOOKPODX(NAME, TYPE, INIT) DEFHOOKPOD (NAME, 0, TYPE, INIT)
+#endif
+   
+/* HOOKSTRUCT(FRAGMENT): Declarator fragments to encapsulate all the
+   members into a struct gcc_target, which in turn contains several
+   sub-structs.  */
+#ifndef HOOKSTRUCT
+#define HOOKSTRUCT(FRAGMENT)
+#endif
+
+HOOKSTRUCT (struct gcc_target {)
+
+/* Functions that output assembler for the target.  */
+#define HOOK_PREFIX "TARGET_ASM_"
+HOOKSTRUCT (struct asm_out {)
+
+DEFHOOKPOD
+(open_paren,
+"These target hooks are C string constants, describing the syntax in the\
+ assembler for grouping arithmetic expressions.  If not overridden, they\
+ default to normal parentheses, which is correct for most assemblers.",
+ const char *, "(")
+DEFHOOKPODX (close_paren, const char *, ")")
+
+DEFHOOKPOD
+(byte_op,
+"@deftypevrx {Target Hook} {const char *} TARGET_ASM_ALIGNED_HI_OP\n"
+"@deftypevrx {Target Hook} {const char *} TARGET_ASM_ALIGNED_SI_OP\n"
+"@deftypevrx {Target Hook} {const char *} TARGET_ASM_ALIGNED_DI_OP\n"
+"@deftypevrx {Target Hook} {const char *} TARGET_ASM_ALIGNED_TI_OP\n"
+"@deftypevrx {Target Hook} {const char *} TARGET_ASM_UNALIGNED_HI_OP\n"
+"@deftypevrx {Target Hook} {const char *} TARGET_ASM_UNALIGNED_SI_OP\n"
+"@deftypevrx {Target Hook} {const char *} TARGET_ASM_UNALIGNED_DI_OP\n"
+"@deftypevrx {Target Hook} {const char *} TARGET_ASM_UNALIGNED_TI_OP\n"
+"These hooks specify assembly directives for creating certain kinds\
+ of integer object.  The @code{TARGET_ASM_BYTE_OP} directive creates a\
+ byte-sized object, the @code{TARGET_ASM_ALIGNED_HI_OP} one creates an\
+ aligned two-byte object, and so on.  Any of the hooks may be\
+ @code{NULL}, indicating that no suitable directive is available.\n\n"
+
+"The compiler will print these strings at the start of a new line,\
+ followed immediately by the object's initial value.  In most cases,\
+ the string should contain a tab, a pseudo-op, and then another tab.",
+ const char *, "\t.byte\t")
+DEFHOOKPOD
+(aligned_op, "*", struct asm_int_op,
+ ({ TARGET_ASM_ALIGNED_HI_OP, TARGET_ASM_ALIGNED_SI_OP,
+TARGET_ASM_ALIGNED_DI_OP, TARGET_ASM_ALIGNED_TI_OP }) )
+DEFHOOKPOD
+(unaligned_op, "*", struct asm_int_op,
+ ({ TARGET_ASM_UNALIGNED_HI_OP, TARGET_ASM_UNALIGNED_SI_OP,
+TARGET_ASM_UNALIGNED_DI_OP, TARGET_ASM_UNALIGNED_TI_OP }) )
+/* Assembler instructions for creating various kinds of integer object.  */
+
+DEFHOOK
+(integer,
+"The @code{assemble_integer} function uses this hook to output an\
+ integer object.  @var{x} is the ob