This patch adds dispatch scheduling for AArch64 by implementing the two target
hooks TARGET_SCHED_DISPATCH and TARGET_SCHED_DISPATCH_DO.
The motivation for this is that cores with out-of-order processing do
most of the reordering to avoid pipeline hazards on the hardware side
using large reorder buffers. For such cores, rather than scheduling
around instruction latencies and throughputs, the compiler should aim to
maximize the utilized dispatch bandwidth by inserting a certain
instruction mix into the frontend dispatch window.
In the following, we will describe the overall implementation:
Recall that the Haifa scheduler makes the following 6 types of queries to
a dispatch scheduling model:
1) targetm.sched.dispatch (NULL, IS_DISPATCH_ON)
2) targetm.sched.dispatch_do (NULL, DISPATCH_INIT)
3) targetm.sched.dispatch (insn, FITS_DISPATCH_WINDOW)
4) targetm.sched.dispatch_do (insn, ADD_TO_DISPATCH_WINDOW)
5) targetm.sched.dispatch (NULL, DISPATCH_VIOLATION)
6) targetm.sched.dispatch (insn, IS_CMP)
For 1), we created the new tune flag AARCH64_EXTRA_TUNE_DISPATCH_SCHED.
For 2-5), we modeled dispatch scheduling using two classes: dispatch_constraint
and dispatch_window.
A dispatch_constraint object represents one rule constraining the dispatch
process, consisting of a function describing the rule and (constant) number
of maximum slots. It also has a non-constant field to keep track of the number
of free slots.
A dispatch_window object models the window of operations that is dispatched
per cycle. It contains an array of dispatch constraints and keeps track of
the available slots. It also exposes functions to ask whether a given
instruction would fit into the dispatch_window or to add an instruction to
the window.
The model operates using only one dispatch_window object that is constructed
when 2) is called. It copies the dispatch information provided in the tuning
model (more details on the changes to tune_params below). During scheduling,
instructions are added according to the dispatch constraints and when the
dispatch_window is full, the free slots in all dispatch_constraint objects are
reset to the maximum.
A dispatch violation occurs when the number of free slots becomes negative
for any dispatch_constraint.
For 6), there is a separate function using the "type"-attribute to check if
a given instruction is a compare instruction.
Dispatch information for a core can be added in its tuning model. We added
two new fields: a *dispatch_constraint (reference to an array of dispatch
constraints) and an int (number of constraints).
All current tuning models were initialized with nullptr and 0.
(In the next patch, dispatch information will be added for Neoverse V2.)
The patch was bootstrapped and tested on aarch64-linux-gnu, no regression.
Signed-off-by: Jennifer Schmitz <jschm...@nvidia.com>
gcc/ChangeLog:
* config.gcc: Add aarch64-sched-dispatch.o to extra_objs.
* config/aarch64/aarch64-protos.h (struct tune_params): New
fields for dispatch scheduling.
* config/aarch64/aarch64-tuning-flags.def
(AARCH64_EXTRA_TUNING_OPTION): New flag to enable dispatch scheduling.
* config/aarch64/aarch64.cc (TARGET_SCHED_DISPATCH): Implement
target hook.
(TARGET_SCHED_DISPATCH_DO): Likewise.
(aarch64_override_options_internal): Add check for definition of
dispatch constraints if dispatch-scheduling tune flag is set.
* config/aarch64/t-aarch64: Add aarch64-sched-dispatch.o.
* config/aarch64/tuning_models/a64fx.h: Initialize fields for
dispatch scheduling in tune_params.
* config/aarch64/tuning_models/ampere1.h: Likewise.
* config/aarch64/tuning_models/ampere1a.h: Likewise.
* config/aarch64/tuning_models/ampere1b.h: Likewise.
* config/aarch64/tuning_models/cortexa35.h: Likewise.
* config/aarch64/tuning_models/cortexa53.h: Likewise.
* config/aarch64/tuning_models/cortexa57.h: Likewise.
* config/aarch64/tuning_models/cortexa72.h: Likewise.
* config/aarch64/tuning_models/cortexa73.h: Likewise.
* config/aarch64/tuning_models/cortexx925.h: Likewise.
* config/aarch64/tuning_models/emag.h: Likewise.
* config/aarch64/tuning_models/exynosm1.h: Likewise.
* config/aarch64/tuning_models/fujitsu_monaka.h: Likewise.
* config/aarch64/tuning_models/generic.h: Likewise.
* config/aarch64/tuning_models/generic_armv8_a.h: Likewise.
* config/aarch64/tuning_models/generic_armv9_a.h: Likewise.
* config/aarch64/tuning_models/neoverse512tvb.h: Likewise.
* config/aarch64/tuning_models/neoversen1.h: Likewise.
* config/aarch64/tuning_models/neoversen2.h: Likewise.
* config/aarch64/tuning_models/neoversen3.h: Likewise.
* config/aarch64/tuning_models/neoversev1.h: Likewise.
* config/aarch64/tuning_models/neoversev2.h: Likewise.
* config/aarch64/tuning_models/neoversev3.h: Likewise.
* config/aarch64/tuning_models/neoversev3ae.h: Likewise.
* config/aarch64/tuning_models/olympus.h: Likewise.
* config/aarch64/tuning_models/qdf24xx.h: Likewise.
* config/aarch64/tuning_models/saphira.h: Likewise.
* config/aarch64/tuning_models/thunderx.h: Likewise.
* config/aarch64/tuning_models/thunderx2t99.h: Likewise.
* config/aarch64/tuning_models/thunderx3t110.h: Likewise.
* config/aarch64/tuning_models/thunderxt88.h: Likewise.
* config/aarch64/tuning_models/tsv110.h: Likewise.
* config/aarch64/tuning_models/xgene1.h: Likewise.
* config/aarch64/aarch64-sched-dispatch.cc: New file for
dispatch scheduling for aarch64.
* config/aarch64/aarch64-sched-dispatch.h: New header file.
---
gcc/config.gcc | 2 +-
gcc/config/aarch64/aarch64-protos.h | 5 +
gcc/config/aarch64/aarch64-sched-dispatch.cc | 272 ++++++++++++++++++
gcc/config/aarch64/aarch64-sched-dispatch.h | 66 +++++
gcc/config/aarch64/aarch64-tuning-flags.def | 3 +
gcc/config/aarch64/aarch64.cc | 14 +
gcc/config/aarch64/t-aarch64 | 8 +
gcc/config/aarch64/tuning_models/a64fx.h | 4 +-
gcc/config/aarch64/tuning_models/ampere1.h | 4 +-
gcc/config/aarch64/tuning_models/ampere1a.h | 4 +-
gcc/config/aarch64/tuning_models/ampere1b.h | 4 +-
gcc/config/aarch64/tuning_models/cortexa35.h | 4 +-
gcc/config/aarch64/tuning_models/cortexa53.h | 4 +-
gcc/config/aarch64/tuning_models/cortexa57.h | 4 +-
gcc/config/aarch64/tuning_models/cortexa72.h | 4 +-
gcc/config/aarch64/tuning_models/cortexa73.h | 4 +-
gcc/config/aarch64/tuning_models/cortexx925.h | 4 +-
gcc/config/aarch64/tuning_models/emag.h | 4 +-
gcc/config/aarch64/tuning_models/exynosm1.h | 4 +-
.../aarch64/tuning_models/fujitsu_monaka.h | 4 +-
gcc/config/aarch64/tuning_models/generic.h | 4 +-
.../aarch64/tuning_models/generic_armv8_a.h | 4 +-
.../aarch64/tuning_models/generic_armv9_a.h | 4 +-
.../aarch64/tuning_models/neoverse512tvb.h | 4 +-
gcc/config/aarch64/tuning_models/neoversen1.h | 4 +-
gcc/config/aarch64/tuning_models/neoversen2.h | 4 +-
gcc/config/aarch64/tuning_models/neoversen3.h | 4 +-
gcc/config/aarch64/tuning_models/neoversev1.h | 4 +-
gcc/config/aarch64/tuning_models/neoversev2.h | 4 +-
gcc/config/aarch64/tuning_models/neoversev3.h | 4 +-
.../aarch64/tuning_models/neoversev3ae.h | 4 +-
gcc/config/aarch64/tuning_models/olympus.h | 4 +-
gcc/config/aarch64/tuning_models/qdf24xx.h | 4 +-
gcc/config/aarch64/tuning_models/saphira.h | 4 +-
gcc/config/aarch64/tuning_models/thunderx.h | 4 +-
.../aarch64/tuning_models/thunderx2t99.h | 4 +-
.../aarch64/tuning_models/thunderx3t110.h | 4 +-
.../aarch64/tuning_models/thunderxt88.h | 4 +-
gcc/config/aarch64/tuning_models/tsv110.h | 4 +-
gcc/config/aarch64/tuning_models/xgene1.h | 4 +-
40 files changed, 468 insertions(+), 34 deletions(-)
create mode 100644 gcc/config/aarch64/aarch64-sched-dispatch.cc
create mode 100644 gcc/config/aarch64/aarch64-sched-dispatch.h
diff --git a/gcc/config.gcc b/gcc/config.gcc
index 0d8dbc4fb19..6ee11313740 100644
--- a/gcc/config.gcc
+++ b/gcc/config.gcc
@@ -351,7 +351,7 @@ aarch64*-*-*)
c_target_objs="aarch64-c.o"
cxx_target_objs="aarch64-c.o"
d_target_objs="aarch64-d.o"
- extra_objs="aarch64-builtins.o aarch-common.o aarch64-elf-metadata.o
aarch64-sve-builtins.o aarch64-sve-builtins-shapes.o
aarch64-sve-builtins-base.o aarch64-sve-builtins-sve2.o
aarch64-sve-builtins-sme.o cortex-a57-fma-steering.o aarch64-speculation.o
aarch-bti-insert.o aarch64-cc-fusion.o aarch64-early-ra.o aarch64-ldp-fusion.o"
+ extra_objs="aarch64-builtins.o aarch-common.o aarch64-elf-metadata.o
aarch64-sve-builtins.o aarch64-sve-builtins-shapes.o
aarch64-sve-builtins-base.o aarch64-sve-builtins-sve2.o
aarch64-sve-builtins-sme.o cortex-a57-fma-steering.o aarch64-speculation.o
aarch-bti-insert.o aarch64-cc-fusion.o aarch64-early-ra.o aarch64-ldp-fusion.o
aarch64-sched-dispatch.o"
target_gtfiles="\$(srcdir)/config/aarch64/aarch64-protos.h
\$(srcdir)/config/aarch64/aarch64-builtins.h
\$(srcdir)/config/aarch64/aarch64-builtins.cc
\$(srcdir)/config/aarch64/aarch64-sve-builtins.h
\$(srcdir)/config/aarch64/aarch64-sve-builtins.cc"
target_has_targetm_common=yes
;;
diff --git a/gcc/config/aarch64/aarch64-protos.h
b/gcc/config/aarch64/aarch64-protos.h
index e946e8da11d..af14d2f6dda 100644
--- a/gcc/config/aarch64/aarch64-protos.h
+++ b/gcc/config/aarch64/aarch64-protos.h
@@ -24,6 +24,7 @@
#include "input.h"
#include "config/arm/aarch-common.h"
+#include "aarch64-sched-dispatch.h"
/* SYMBOL_SMALL_ABSOLUTE: Generate symbol accesses through
high and lo relocs that calculate the base address using a PC
@@ -581,6 +582,10 @@ struct tune_params
/* Define models for the aarch64_ldp_stp_policy. */
enum aarch64_ldp_stp_policy ldp_policy_model, stp_policy_model;
+
+ /* Dispatch constraints for instruction scheduling. */
+ const dispatch_constraint *dispatch_constraints;
+ int num_dispatch_constraints;
};
/* Classifies an address.
diff --git a/gcc/config/aarch64/aarch64-sched-dispatch.cc
b/gcc/config/aarch64/aarch64-sched-dispatch.cc
new file mode 100644
index 00000000000..5fde528377c
--- /dev/null
+++ b/gcc/config/aarch64/aarch64-sched-dispatch.cc
@@ -0,0 +1,272 @@
+/* Dispatch scheduling hooks for AArch64.
+ Copyright The GNU Toolchain Authors.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ GCC is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GCC; see the file COPYING3. If not see
+ <http://www.gnu.org/licenses/>. */
+
+#include "config.h"
+#include "system.h"
+#include "coretypes.h"
+#include "backend.h"
+#include "rtl.h"
+#include "insn-attr.h"
+#include "insn-attr-common.h"
+#include "aarch64-protos.h"
+#include "aarch64-sched-dispatch.h"
+#include "regset.h"
+#include "sched-int.h"
+#include "dumpfile.h"
+#include "print-rtl.h"
+
+/* This file implements the target hooks for dispatch scheduling for AArch64.
+ Instructions are scheduled into the current dispatch window according to
+ dispatch constraints provided by the tuning model.
+
+ To enable dispatch scheduling for a core (see Neoverse V2 for an example):
+ - in the tuning model, add the AARCH64_EXTRA_TUNE_DISPATCH_SCHED tune flag
+ - in the tuning model, add an array of dispatch constraints (more details
+ below) and add its reference and length to the tune_params struct
+ - optionally, create a new instruction attribute to classify instructions
+ into dispatch groups (e.g. neoversev2_dispatch)
+
+ Dispatch constraints are defined as an array of dispatch_constraint objects.
+ Each dispatch_constraint object is constructed with a name, a number of
+ slots (max number of instructions of that type that can be dispatched per
+ cycle), and a callback function.
+ The callback function takes an instruction as argument, determines whether
+ the instruction matches the constraint criteria, and returns the number of
+ slots occupied by the instruction.
+ */
+
+static dispatch_window *current_dispatch_window;
+
+/* Constructor for class dispatch_constraint. */
+dispatch_constraint::dispatch_constraint (const char *name, const int slots,
+ int (*f) (rtx_insn *))
+ : m_name (name), m_max_slots (slots), m_free_slots (slots)
+{
+ this->m_callback = f;
+}
+
+/* Return TRUE if INSN matches the constraint criteria, and there are
+ enough free slots. */
+bool
+dispatch_constraint::can_fit_insn (rtx_insn *insn) const
+{
+ int slots = num_occupied_slots (insn);
+ if (dump_file && (dump_flags & TDF_DETAILS) && slots > 0)
+ {
+ fprintf (dump_file, "Insn matches constraint \'%s\'. ", m_name);
+ fprintf (dump_file, "Slots required: %d; slots free: %d.\n",
+ slots, m_free_slots);
+ }
+ return slots <= m_free_slots;
+}
+
+/* Decrement the free slots by the number of slots occupied by INSN.
+ Return whether the number of free slots is non-negative (no violation). */
+bool
+dispatch_constraint::add_insn_without_violation (rtx_insn *insn)
+{
+ m_free_slots -= num_occupied_slots (insn);
+ return m_free_slots >= 0;
+}
+
+/* Reset the number of free slots to the maximum number of slots.
+ This is called when the next dispatch window is started. */
+void
+dispatch_constraint::reset_slots ()
+{
+ m_free_slots = m_max_slots;
+}
+
+/* Print the dispatch constraint to the given FILE. */
+void
+dispatch_constraint::print_dispatch_constraint (FILE *file) const
+{
+ fprintf (file, "Dispatch constraint \'%s\': %d of %d slots free\n",
+ m_name, m_free_slots, m_max_slots);
+}
+
+/* If INSN matches the dispatch constraint, return the number of slots
+ occupied, else return 0. */
+int
+dispatch_constraint::num_occupied_slots (rtx_insn *insn) const
+{
+ return m_callback (insn);
+}
+
+/* Constructor for class dispatch_window. */
+dispatch_window::dispatch_window (dispatch_constraint *constraints,
+ int num_constraints)
+ : m_constraints (constraints), m_num_constraints (num_constraints),
+ m_violation (false)
+{
+}
+
+/* Return TRUE iff INSN fits into the dispatch window according to
+ to all constraints. */
+bool
+dispatch_window::fits_dispatch_window (rtx_insn *insn) const
+{
+ if (INSN_CODE (insn) >= 0)
+ {
+ if (dump_file)
+ {
+ fprintf (dump_file, "Checking if insn fits into dispatch window:\n");
+ print_rtl_single (dump_file, insn);
+ }
+ for (int i = 0; i < m_num_constraints; i++)
+ {
+ if (!m_constraints[i].can_fit_insn (insn))
+ return false;
+ }
+ }
+ return true;
+}
+
+/* If INSN does not fit into the dispatch window, reset the constraints and
+ start a new dispatch window.
+ Then, add INSN to the dispatch window and set the violation flag
+ if there is a constraint violation. */
+void
+dispatch_window::add_insn_to_window (rtx_insn *insn)
+{
+ if (INSN_CODE (insn) < 0)
+ return;
+
+ if (!fits_dispatch_window (insn))
+ {
+ if (dump_file)
+ fprintf (dump_file, "Window full. Starting new dispatch window.\n");
+ reset_constraints ();
+ }
+
+ for (int i = 0; i < m_num_constraints; i++)
+ {
+ if (!m_constraints[i].add_insn_without_violation (insn))
+ m_violation = true;
+ }
+ if (dump_file)
+ {
+ fprintf (dump_file, "Insn added to dispatch window.\n");
+ if (dump_flags & TDF_DETAILS)
+ print_dispatch_window (dump_file);
+ }
+}
+
+/* Return TRUE iff there is a dispatch violation, i.e. one of the dispatch
+ constraints has a negative number of free slots. */
+bool
+dispatch_window::has_violation () const
+{
+ return m_violation;
+}
+
+/* Print information about the dispatch window to the given FILE. */
+void
+dispatch_window::print_dispatch_window (FILE *file) const
+{
+ fprintf (file, "==== Current dispatch window ====\n");
+ fprintf (file, "Violation: %s\n", m_violation ? "true" : "false");
+ for (int i = 0; i < m_num_constraints; i++)
+ m_constraints[i].print_dispatch_constraint (file);
+ fprintf (file, "\n");
+}
+
+/* For all dispatch constraints, reset the number of free slots to the
+ maximum number of slots.
+ This is called when the next dispatch window is started. */
+void
+dispatch_window::reset_constraints ()
+{
+ for (int i = 0; i < m_num_constraints; i++)
+ m_constraints[i].reset_slots ();
+ m_violation = false;
+}
+
+
+/* Initialize the dispatch window using the constraints from the tuning model.
+ This is called once at the beginning of scheduling. */
+void
+init_dispatch_window (void)
+{
+ const dispatch_constraint *constraints =
+ aarch64_tune_params.dispatch_constraints;
+ int num_constraints = aarch64_tune_params.num_dispatch_constraints;
+ current_dispatch_window =
+ new dispatch_window (const_cast<dispatch_constraint *>(constraints),
+ num_constraints);
+
+ if (dump_file)
+ {
+ fprintf (dump_file, "DISPATCH WINDOW INITIALIZED\n");
+ if (dump_flags & TDF_DETAILS)
+ current_dispatch_window->print_dispatch_window (dump_file);
+ }
+}
+
+/* Return TRUE if INSN is a comparison instruction. */
+bool
+is_cmp_insn (rtx_insn *insn)
+{
+ enum attr_type type;
+ type = get_attr_type (insn);
+ return (type == TYPE_FCMPS
+ || type == TYPE_FCMPD
+ || type == TYPE_FCCMPS
+ || type == TYPE_FCCMPD
+ || GET_CODE (PATTERN (insn)) == COMPARE);
+}
+
+/* The next two functions implement the dispatch-scheduling target hooks
+ for aarch64 and are the drivers of the dispatch scheduler. */
+void
+aarch64_sched_dispatch_do (rtx_insn *insn, int mode)
+{
+ if (mode == DISPATCH_INIT)
+ init_dispatch_window ();
+ else if (mode == ADD_TO_DISPATCH_WINDOW && current_dispatch_window)
+ current_dispatch_window->add_insn_to_window (insn);
+}
+
+bool
+aarch64_sched_dispatch (rtx_insn *insn, int action)
+{
+ /* We only want dispatch scheduling to be enabled during the last
+ scheduling pass, i.e. after reload and sched_fusion. */
+ if ((aarch64_tune_params.extra_tuning_flags
+ & AARCH64_EXTRA_TUNE_DISPATCH_SCHED)
+ && reload_completed &&!sched_fusion)
+ switch (action)
+ {
+ default:
+ return false;
+
+ case IS_DISPATCH_ON:
+ return true;
+
+ case IS_CMP:
+ return is_cmp_insn (insn);
+
+ case DISPATCH_VIOLATION:
+ return current_dispatch_window->has_violation ();
+
+ case FITS_DISPATCH_WINDOW:
+ return current_dispatch_window->fits_dispatch_window (insn);
+ }
+ return false;
+}
\ No newline at end of file
diff --git a/gcc/config/aarch64/aarch64-sched-dispatch.h
b/gcc/config/aarch64/aarch64-sched-dispatch.h
new file mode 100644
index 00000000000..1e0dc420139
--- /dev/null
+++ b/gcc/config/aarch64/aarch64-sched-dispatch.h
@@ -0,0 +1,66 @@
+/* Dispatch scheduling hooks for AArch64.
+ Copyright The GNU Toolchain Authors.
+
+ This file is part of GCC.
+
+ GCC is free software; you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3, or (at your option)
+ any later version.
+
+ GCC is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with GCC; see the file COPYING3. If not see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef GCC_AARCH64_SCHED_DISPATCH_H
+#define GCC_AARCH64_SCHED_DISPATCH_H
+
+void aarch64_sched_dispatch_do (rtx_insn *, int);
+bool aarch64_sched_dispatch (rtx_insn *, int);
+bool is_cmp_insn (rtx_insn *);
+
+/* Describes one dispatch constraint in terms of a callback function
+ and a number of slots per dispatch window. */
+class dispatch_constraint
+{
+public:
+ dispatch_constraint (const char *, const int, int (*) (rtx_insn *));
+ bool can_fit_insn (rtx_insn *) const;
+ bool add_insn_without_violation (rtx_insn *);
+ void reset_slots ();
+ void print_dispatch_constraint (FILE *) const;
+
+private:
+ int num_occupied_slots (rtx_insn *) const;
+
+ const char *m_name;
+ int (*m_callback) (rtx_insn *);
+ const int m_max_slots;
+ int m_free_slots;
+};
+
+/* Describes a dispatch window and keeps track of the dispatch constraints. */
+class dispatch_window
+{
+public:
+ dispatch_window (dispatch_constraint *constraints, int num_constraints);
+
+ bool fits_dispatch_window (rtx_insn *) const;
+ void add_insn_to_window (rtx_insn *);
+ bool has_violation () const;
+ void print_dispatch_window (FILE *) const;
+
+private:
+ void reset_constraints ();
+
+ dispatch_constraint *m_constraints;
+ const int m_num_constraints;
+ bool m_violation;
+};
+
+#endif /* GCC_AARCH64_SCHED_DISPATCH_H */
diff --git a/gcc/config/aarch64/aarch64-tuning-flags.def
b/gcc/config/aarch64/aarch64-tuning-flags.def
index dd91324e9c8..9c9af451b8e 100644
--- a/gcc/config/aarch64/aarch64-tuning-flags.def
+++ b/gcc/config/aarch64/aarch64-tuning-flags.def
@@ -69,4 +69,7 @@ AARCH64_EXTRA_TUNING_OPTION ("cheap_fpmr_write",
CHEAP_FPMR_WRITE)
#define AARCH64_EXTRA_TUNE_BASE (AARCH64_EXTRA_TUNE_CHEAP_SHIFT_EXTEND \
| AARCH64_EXTRA_TUNE_FULLY_PIPELINED_FMA)
+/* Enables dispatch scheduling. */
+AARCH64_EXTRA_TUNING_OPTION ("dispatch_sched", DISPATCH_SCHED)
+
#undef AARCH64_EXTRA_TUNING_OPTION
diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc
index 2c5b6173d83..6cd95d41136 100644
--- a/gcc/config/aarch64/aarch64.cc
+++ b/gcc/config/aarch64/aarch64.cc
@@ -98,6 +98,7 @@
#include "ipa-prop.h"
#include "ipa-fnsummary.h"
#include "hash-map.h"
+#include "aarch64-sched-dispatch.h"
/* This file should be included last. */
#include "target-def.h"
@@ -19078,6 +19079,13 @@ aarch64_override_options_internal (struct gcc_options
*opts)
SET_OPTION_IF_UNSET (opts, &global_options_set, param_fully_pipelined_fma,
1);
+ /* If dispatch scheduling is enabled, dispatch_constraints and
+ num_dispatch_constraints in the tune_params struct must be defined. */
+ if (aarch64_tune_params.extra_tuning_flags
+ & AARCH64_EXTRA_TUNE_DISPATCH_SCHED)
+ gcc_assert (aarch64_tune_params.dispatch_constraints != NULL
+ && aarch64_tune_params.num_dispatch_constraints > 0);
+
/* TODO: SME codegen without SVE2 is not supported, once this support is
added
remove this 'sorry' and the implicit enablement of SVE2 in the checks for
streaming mode above in this function. */
@@ -32244,6 +32252,12 @@ aarch64_libgcc_floating_mode_supported_p
#undef TARGET_SCHED_REASSOCIATION_WIDTH
#define TARGET_SCHED_REASSOCIATION_WIDTH aarch64_reassociation_width
+#undef TARGET_SCHED_DISPATCH
+#define TARGET_SCHED_DISPATCH aarch64_sched_dispatch
+
+#undef TARGET_SCHED_DISPATCH_DO
+#define TARGET_SCHED_DISPATCH_DO aarch64_sched_dispatch_do
+
#undef TARGET_DWARF_FRAME_REG_MODE
#define TARGET_DWARF_FRAME_REG_MODE aarch64_dwarf_frame_reg_mode
diff --git a/gcc/config/aarch64/t-aarch64 b/gcc/config/aarch64/t-aarch64
index 38a8c063725..40c70a90d13 100644
--- a/gcc/config/aarch64/t-aarch64
+++ b/gcc/config/aarch64/t-aarch64
@@ -208,6 +208,14 @@ aarch64-ldp-fusion.o:
$(srcdir)/config/aarch64/aarch64-ldp-fusion.cc \
$(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) \
$(srcdir)/config/aarch64/aarch64-ldp-fusion.cc
+aarch64-sched-dispatch.o: $(srcdir)/config/aarch64/aarch64-sched-dispatch.cc \
+ $(CONFIG_H) $(SYSTEM_H) $(CORETYPES_H) $(BACKEND_H) $(RTL_H) \
+ $(INSN_ATTR_H) $(REGSET_H) sched-int.h $(DUMPFILE_H) \
+ $(srcdir)/config/aarch64/aarch64-protos.h \
+ $(srcdir)/config/aarch64/aarch64-sched-dispatch.h
+ $(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) \
+ $(srcdir)/config/aarch64/aarch64-sched-dispatch.cc
+
comma=,
MULTILIB_OPTIONS = $(subst $(comma),/, $(patsubst %, mabi=%, $(subst
$(comma),$(comma)mabi=,$(TM_MULTILIB_CONFIG))))
MULTILIB_DIRNAMES = $(subst $(comma), ,$(TM_MULTILIB_CONFIG))
diff --git a/gcc/config/aarch64/tuning_models/a64fx.h
b/gcc/config/aarch64/tuning_models/a64fx.h
index 2622cd80346..29ef4a1d33c 100644
--- a/gcc/config/aarch64/tuning_models/a64fx.h
+++ b/gcc/config/aarch64/tuning_models/a64fx.h
@@ -165,7 +165,9 @@ static const struct tune_params a64fx_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&a64fx_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_A64FX. */
diff --git a/gcc/config/aarch64/tuning_models/ampere1.h
b/gcc/config/aarch64/tuning_models/ampere1.h
index f033016d010..ed4f2068d12 100644
--- a/gcc/config/aarch64/tuning_models/ampere1.h
+++ b/gcc/config/aarch64/tuning_models/ampere1.h
@@ -105,7 +105,9 @@ static const struct tune_params ampere1_tunings =
| AARCH64_EXTRA_TUNE_FULLY_PIPELINED_FMA), /* tune_flags. */
&ere1_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_AMPERE1. */
diff --git a/gcc/config/aarch64/tuning_models/ampere1a.h
b/gcc/config/aarch64/tuning_models/ampere1a.h
index 41481a7f077..6aada99913e 100644
--- a/gcc/config/aarch64/tuning_models/ampere1a.h
+++ b/gcc/config/aarch64/tuning_models/ampere1a.h
@@ -57,7 +57,9 @@ static const struct tune_params ampere1a_tunings =
| AARCH64_EXTRA_TUNE_FULLY_PIPELINED_FMA), /* tune_flags. */
&ere1_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_AMPERE1A. */
diff --git a/gcc/config/aarch64/tuning_models/ampere1b.h
b/gcc/config/aarch64/tuning_models/ampere1b.h
index 2ad6003d65c..b0e49a32d88 100644
--- a/gcc/config/aarch64/tuning_models/ampere1b.h
+++ b/gcc/config/aarch64/tuning_models/ampere1b.h
@@ -106,7 +106,9 @@ static const struct tune_params ampere1b_tunings =
| AARCH64_EXTRA_TUNE_AVOID_CROSS_LOOP_FMA), /* tune_flags. */
&ere1b_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_AMPERE1B */
diff --git a/gcc/config/aarch64/tuning_models/cortexa35.h
b/gcc/config/aarch64/tuning_models/cortexa35.h
index 56168c8ebc6..a3294634bb4 100644
--- a/gcc/config/aarch64/tuning_models/cortexa35.h
+++ b/gcc/config/aarch64/tuning_models/cortexa35.h
@@ -56,7 +56,9 @@ static const struct tune_params cortexa35_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&generic_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_CORTEXA35. */
diff --git a/gcc/config/aarch64/tuning_models/cortexa53.h
b/gcc/config/aarch64/tuning_models/cortexa53.h
index 83daae4d38c..ed62ccf0a2a 100644
--- a/gcc/config/aarch64/tuning_models/cortexa53.h
+++ b/gcc/config/aarch64/tuning_models/cortexa53.h
@@ -65,7 +65,9 @@ static const struct tune_params cortexa53_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&generic_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_CORTEXA53. */
diff --git a/gcc/config/aarch64/tuning_models/cortexa57.h
b/gcc/config/aarch64/tuning_models/cortexa57.h
index 8da7fa9d80d..dfca057c71c 100644
--- a/gcc/config/aarch64/tuning_models/cortexa57.h
+++ b/gcc/config/aarch64/tuning_models/cortexa57.h
@@ -102,7 +102,9 @@ static const struct tune_params cortexa57_tunings =
(AARCH64_EXTRA_TUNE_RENAME_FMA_REGS), /* tune_flags. */
&generic_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_CORTEXA57. */
diff --git a/gcc/config/aarch64/tuning_models/cortexa72.h
b/gcc/config/aarch64/tuning_models/cortexa72.h
index f9a330f4a18..953104cade3 100644
--- a/gcc/config/aarch64/tuning_models/cortexa72.h
+++ b/gcc/config/aarch64/tuning_models/cortexa72.h
@@ -54,7 +54,9 @@ static const struct tune_params cortexa72_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&generic_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_CORTEXA72. */
diff --git a/gcc/config/aarch64/tuning_models/cortexa73.h
b/gcc/config/aarch64/tuning_models/cortexa73.h
index 038fd0896b9..dcd2a0ac286 100644
--- a/gcc/config/aarch64/tuning_models/cortexa73.h
+++ b/gcc/config/aarch64/tuning_models/cortexa73.h
@@ -55,7 +55,9 @@ static const struct tune_params cortexa73_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&generic_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
diff --git a/gcc/config/aarch64/tuning_models/cortexx925.h
b/gcc/config/aarch64/tuning_models/cortexx925.h
index f448493b1bc..cbd572cd424 100644
--- a/gcc/config/aarch64/tuning_models/cortexx925.h
+++ b/gcc/config/aarch64/tuning_models/cortexx925.h
@@ -226,7 +226,9 @@ static const struct tune_params cortexx925_tunings =
| AARCH64_EXTRA_TUNE_AVOID_LDAPUR), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_CORTEXX925. */
diff --git a/gcc/config/aarch64/tuning_models/emag.h
b/gcc/config/aarch64/tuning_models/emag.h
index 264a2810299..ccb68e471aa 100644
--- a/gcc/config/aarch64/tuning_models/emag.h
+++ b/gcc/config/aarch64/tuning_models/emag.h
@@ -54,7 +54,9 @@ static const struct tune_params emag_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&xgene1_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_EMAG. */
diff --git a/gcc/config/aarch64/tuning_models/exynosm1.h
b/gcc/config/aarch64/tuning_models/exynosm1.h
index 71876df7785..6ae27f65866 100644
--- a/gcc/config/aarch64/tuning_models/exynosm1.h
+++ b/gcc/config/aarch64/tuning_models/exynosm1.h
@@ -138,7 +138,9 @@ static const struct tune_params exynosm1_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&exynosm1_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_EXYNOSM1. */
diff --git a/gcc/config/aarch64/tuning_models/fujitsu_monaka.h
b/gcc/config/aarch64/tuning_models/fujitsu_monaka.h
index 5dc40243fe3..041e907d38f 100644
--- a/gcc/config/aarch64/tuning_models/fujitsu_monaka.h
+++ b/gcc/config/aarch64/tuning_models/fujitsu_monaka.h
@@ -58,7 +58,9 @@ static const struct tune_params fujitsu_monaka_tunings =
| AARCH64_EXTRA_TUNE_MATCHED_VECTOR_THROUGHPUT), /* tune_flags. */
&generic_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_FUJITSU_MONAKA. */
diff --git a/gcc/config/aarch64/tuning_models/generic.h
b/gcc/config/aarch64/tuning_models/generic.h
index a822c055fe9..a962f8f72ac 100644
--- a/gcc/config/aarch64/tuning_models/generic.h
+++ b/gcc/config/aarch64/tuning_models/generic.h
@@ -186,7 +186,9 @@ static const struct tune_params generic_tunings =
(AARCH64_EXTRA_TUNE_CSE_SVE_VL_CONSTANTS), /* tune_flags. */
&generic_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_GENERIC. */
diff --git a/gcc/config/aarch64/tuning_models/generic_armv8_a.h
b/gcc/config/aarch64/tuning_models/generic_armv8_a.h
index 01080cade46..e7d512a4423 100644
--- a/gcc/config/aarch64/tuning_models/generic_armv8_a.h
+++ b/gcc/config/aarch64/tuning_models/generic_armv8_a.h
@@ -186,7 +186,9 @@ static const struct tune_params generic_armv8_a_tunings =
| AARCH64_EXTRA_TUNE_MATCHED_VECTOR_THROUGHPUT), /* tune_flags. */
&generic_armv8_a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_GENERIC_ARMV8_A. */
diff --git a/gcc/config/aarch64/tuning_models/generic_armv9_a.h
b/gcc/config/aarch64/tuning_models/generic_armv9_a.h
index f76a2506f38..cff40882cfd 100644
--- a/gcc/config/aarch64/tuning_models/generic_armv9_a.h
+++ b/gcc/config/aarch64/tuning_models/generic_armv9_a.h
@@ -254,7 +254,9 @@ static const struct tune_params generic_armv9_a_tunings =
| AARCH64_EXTRA_TUNE_MATCHED_VECTOR_THROUGHPUT), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_GENERIC_ARMV9_A. */
diff --git a/gcc/config/aarch64/tuning_models/neoverse512tvb.h
b/gcc/config/aarch64/tuning_models/neoverse512tvb.h
index 964b4ac284a..8c4ec3ac733 100644
--- a/gcc/config/aarch64/tuning_models/neoverse512tvb.h
+++ b/gcc/config/aarch64/tuning_models/neoverse512tvb.h
@@ -161,7 +161,9 @@ static const struct tune_params neoverse512tvb_tunings =
| AARCH64_EXTRA_TUNE_AVOID_PRED_RMW), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_NEOVERSE512TVB. */
diff --git a/gcc/config/aarch64/tuning_models/neoversen1.h
b/gcc/config/aarch64/tuning_models/neoversen1.h
index 9dc37bd7fd6..d5425d5bf99 100644
--- a/gcc/config/aarch64/tuning_models/neoversen1.h
+++ b/gcc/config/aarch64/tuning_models/neoversen1.h
@@ -54,7 +54,9 @@ static const struct tune_params neoversen1_tunings =
(AARCH64_EXTRA_TUNE_BASE), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_NEOVERSEN1. */
diff --git a/gcc/config/aarch64/tuning_models/neoversen2.h
b/gcc/config/aarch64/tuning_models/neoversen2.h
index 9fbc059ea12..d52b0404ec5 100644
--- a/gcc/config/aarch64/tuning_models/neoversen2.h
+++ b/gcc/config/aarch64/tuning_models/neoversen2.h
@@ -223,7 +223,9 @@ static const struct tune_params neoversen2_tunings =
| AARCH64_EXTRA_TUNE_AVOID_PRED_RMW), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_NEOVERSEN2. */
diff --git a/gcc/config/aarch64/tuning_models/neoversen3.h
b/gcc/config/aarch64/tuning_models/neoversen3.h
index 78177e78e07..ce5cef6fb24 100644
--- a/gcc/config/aarch64/tuning_models/neoversen3.h
+++ b/gcc/config/aarch64/tuning_models/neoversen3.h
@@ -222,7 +222,9 @@ static const struct tune_params neoversen3_tunings =
| AARCH64_EXTRA_TUNE_MATCHED_VECTOR_THROUGHPUT), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_NEOVERSEN3. */
diff --git a/gcc/config/aarch64/tuning_models/neoversev1.h
b/gcc/config/aarch64/tuning_models/neoversev1.h
index f1ec7dcdda7..2f224bfb2ca 100644
--- a/gcc/config/aarch64/tuning_models/neoversev1.h
+++ b/gcc/config/aarch64/tuning_models/neoversev1.h
@@ -232,7 +232,9 @@ static const struct tune_params neoversev1_tunings =
| AARCH64_EXTRA_TUNE_AVOID_PRED_RMW), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
diff --git a/gcc/config/aarch64/tuning_models/neoversev2.h
b/gcc/config/aarch64/tuning_models/neoversev2.h
index 266d8f190a2..faf06d8e7ed 100644
--- a/gcc/config/aarch64/tuning_models/neoversev2.h
+++ b/gcc/config/aarch64/tuning_models/neoversev2.h
@@ -224,7 +224,9 @@ static const struct tune_params neoversev2_tunings =
| AARCH64_EXTRA_TUNE_AVOID_LDAPUR), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_NEOVERSEV2. */
diff --git a/gcc/config/aarch64/tuning_models/neoversev3.h
b/gcc/config/aarch64/tuning_models/neoversev3.h
index f5566d270da..19281096006 100644
--- a/gcc/config/aarch64/tuning_models/neoversev3.h
+++ b/gcc/config/aarch64/tuning_models/neoversev3.h
@@ -224,7 +224,9 @@ static const struct tune_params neoversev3_tunings =
| AARCH64_EXTRA_TUNE_AVOID_LDAPUR), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_NEOVERSEV3. */
diff --git a/gcc/config/aarch64/tuning_models/neoversev3ae.h
b/gcc/config/aarch64/tuning_models/neoversev3ae.h
index 5796e52a266..329e004eab8 100644
--- a/gcc/config/aarch64/tuning_models/neoversev3ae.h
+++ b/gcc/config/aarch64/tuning_models/neoversev3ae.h
@@ -224,7 +224,9 @@ static const struct tune_params neoversev3ae_tunings =
| AARCH64_EXTRA_TUNE_AVOID_LDAPUR), /* tune_flags. */
&generic_armv9a_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_NEOVERSEV3AE. */
diff --git a/gcc/config/aarch64/tuning_models/olympus.h
b/gcc/config/aarch64/tuning_models/olympus.h
index 268789db019..6f471a1c5b7 100644
--- a/gcc/config/aarch64/tuning_models/olympus.h
+++ b/gcc/config/aarch64/tuning_models/olympus.h
@@ -204,7 +204,9 @@ static struct tune_params olympus_tunings =
| AARCH64_EXTRA_TUNE_AVOID_PRED_RMW), /* tune_flags. */
&olympus_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_OLYMPUS. */
diff --git a/gcc/config/aarch64/tuning_models/qdf24xx.h
b/gcc/config/aarch64/tuning_models/qdf24xx.h
index 583d30a4d17..6ef115a0882 100644
--- a/gcc/config/aarch64/tuning_models/qdf24xx.h
+++ b/gcc/config/aarch64/tuning_models/qdf24xx.h
@@ -130,7 +130,9 @@ static const struct tune_params qdf24xx_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&qdf24xx_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_QDF24XX. */
diff --git a/gcc/config/aarch64/tuning_models/saphira.h
b/gcc/config/aarch64/tuning_models/saphira.h
index 684f3951a60..c37a731649d 100644
--- a/gcc/config/aarch64/tuning_models/saphira.h
+++ b/gcc/config/aarch64/tuning_models/saphira.h
@@ -56,7 +56,9 @@ static const struct tune_params saphira_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&generic_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_SAPHIRA. */
diff --git a/gcc/config/aarch64/tuning_models/thunderx.h
b/gcc/config/aarch64/tuning_models/thunderx.h
index d79c1391b8c..f94ef397bdc 100644
--- a/gcc/config/aarch64/tuning_models/thunderx.h
+++ b/gcc/config/aarch64/tuning_models/thunderx.h
@@ -111,7 +111,9 @@ static const struct tune_params thunderx_tunings =
(AARCH64_EXTRA_TUNE_CHEAP_SHIFT_EXTEND), /* tune_flags. */
&thunderx_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_THUNDERX. */
diff --git a/gcc/config/aarch64/tuning_models/thunderx2t99.h
b/gcc/config/aarch64/tuning_models/thunderx2t99.h
index 513c6158084..ea5cc39f0e4 100644
--- a/gcc/config/aarch64/tuning_models/thunderx2t99.h
+++ b/gcc/config/aarch64/tuning_models/thunderx2t99.h
@@ -131,7 +131,9 @@ static const struct tune_params thunderx2t99_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&thunderx2t99_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_THUNDERX2T99. */
diff --git a/gcc/config/aarch64/tuning_models/thunderx3t110.h
b/gcc/config/aarch64/tuning_models/thunderx3t110.h
index a2547b8e54d..a3a3d72ce43 100644
--- a/gcc/config/aarch64/tuning_models/thunderx3t110.h
+++ b/gcc/config/aarch64/tuning_models/thunderx3t110.h
@@ -130,7 +130,9 @@ static const struct tune_params thunderx3t110_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&thunderx3t110_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_THUNDERX3T110. */
diff --git a/gcc/config/aarch64/tuning_models/thunderxt88.h
b/gcc/config/aarch64/tuning_models/thunderxt88.h
index 6be5c526fc1..85b06e96233 100644
--- a/gcc/config/aarch64/tuning_models/thunderxt88.h
+++ b/gcc/config/aarch64/tuning_models/thunderxt88.h
@@ -66,7 +66,9 @@ static const struct tune_params thunderxt88_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&thunderxt88_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALIGNED, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALIGNED /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALIGNED, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_THUNDERXT88. */
diff --git a/gcc/config/aarch64/tuning_models/tsv110.h
b/gcc/config/aarch64/tuning_models/tsv110.h
index 458286e0e80..450f13390d0 100644
--- a/gcc/config/aarch64/tuning_models/tsv110.h
+++ b/gcc/config/aarch64/tuning_models/tsv110.h
@@ -131,7 +131,9 @@ static const struct tune_params tsv110_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&tsv110_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_TSV110. */
diff --git a/gcc/config/aarch64/tuning_models/xgene1.h
b/gcc/config/aarch64/tuning_models/xgene1.h
index b4f01ee92f5..f69a5de6fe2 100644
--- a/gcc/config/aarch64/tuning_models/xgene1.h
+++ b/gcc/config/aarch64/tuning_models/xgene1.h
@@ -139,7 +139,9 @@ static const struct tune_params xgene1_tunings =
(AARCH64_EXTRA_TUNE_NONE), /* tune_flags. */
&xgene1_prefetch_tune,
AARCH64_LDP_STP_POLICY_ALWAYS, /* ldp_policy_model. */
- AARCH64_LDP_STP_POLICY_ALWAYS /* stp_policy_model. */
+ AARCH64_LDP_STP_POLICY_ALWAYS, /* stp_policy_model. */
+ nullptr, /* dispatch_constraints. */
+ 0 /* num_dispatch_constraints. */
};
#endif /* GCC_AARCH64_H_XGENE1. */
--
2.34.1
