[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #25 from rguenth at gcc dot gnu dot org 2007-03-31 14:42 --- Yes. -- rguenth at gcc dot gnu dot org changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution||FIXED http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #23 from rguenth at gcc dot gnu dot org 2007-01-14 15:06 --- Can it be the patch changes result of alias analysis? I get (big) runtime differences (but maybe due to unrelated changes) with the tester from Jan 13 (patched) vs. Jan 14 (unpatched). -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #21 from rguenth at gcc dot gnu dot org 2007-01-13 23:02 --- The patch fixed the freefem memory regression. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
Re: [Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
okay, i'll update changelog, submit and commit. On 13 Jan 2007 23:02:13 -, rguenth at gcc dot gnu dot org [EMAIL PROTECTED] wrote: --- Comment #21 from rguenth at gcc dot gnu dot org 2007-01-13 23:02 --- The patch fixed the freefem memory regression. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #22 from dberlin at gcc dot gnu dot org 2007-01-14 01:22 --- Subject: Re: Compiling FreeFem3d uses unreasonable amount of time and memory okay, i'll update changelog, submit and commit. On 13 Jan 2007 23:02:13 -, rguenth at gcc dot gnu dot org [EMAIL PROTECTED] wrote: --- Comment #21 from rguenth at gcc dot gnu dot org 2007-01-13 23:02 --- The patch fixed the freefem memory regression. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089 -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #20 from rguenth at gcc dot gnu dot org 2007-01-11 18:12 --- Again tonight - Mark broke bootstrap. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #19 from rguenth at gcc dot gnu dot org 2007-01-10 18:39 --- We'll see with tonights run of the tester. Thanks. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #16 from rguenth at gcc dot gnu dot org 2007-01-09 21:17 --- Pling! -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
Re: [Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
Try the attached, let me know how it goes.
On 9 Jan 2007 21:17:05 -, rguenth at gcc dot gnu dot org
[EMAIL PROTECTED] wrote:
--- Comment #16 from rguenth at gcc dot gnu dot org 2007-01-09 21:17
---
Pling!
--
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
--- gcc/tree.h (/mirror/gcc-trunk) (revision 1114)
+++ gcc/tree.h (/local/gcc-clean) (revision 1114)
@@ -2449,10 +2449,14 @@ struct tree_decl_minimal GTY(())
struct tree_memory_tag GTY(())
{
struct tree_decl_minimal common;
+
+ bitmap GTY ((skip)) aliases;
+
unsigned int is_global:1;
};
#define MTAG_GLOBAL(NODE) (TREE_MEMORY_TAG_CHECK (NODE)-mtag.is_global)
+#define MTAG_ALIASES(NODE) (TREE_MEMORY_TAG_CHECK (NODE)-mtag.aliases)
struct tree_struct_field_tag GTY(())
{
--- gcc/tree-ssa-alias.c (/mirror/gcc-trunk) (revision 1114)
+++ gcc/tree-ssa-alias.c (/local/gcc-clean) (revision 1114)
@@ -90,6 +90,7 @@ struct alias_stats_d
/* Local variables. */
static struct alias_stats_d alias_stats;
+static bitmap_obstack alias_bitmap_obstack;
/* Local functions. */
static void compute_flow_insensitive_aliasing (struct alias_info *);
@@ -99,7 +100,7 @@ static bool may_alias_p (tree, HOST_WIDE
static tree create_memory_tag (tree type, bool is_type_tag);
static tree get_smt_for (tree, struct alias_info *);
static tree get_nmt_for (tree);
-static void add_may_alias (tree, tree, struct pointer_set_t *);
+static void add_may_alias (tree, tree);
static struct alias_info *init_alias_info (void);
static void delete_alias_info (struct alias_info *);
static void compute_flow_sensitive_aliasing (struct alias_info *);
@@ -194,19 +195,21 @@ static void
mark_aliases_call_clobbered (tree tag, VEC (tree, heap) **worklist,
VEC (int, heap) **worklist2)
{
+ bitmap aliases;
+ bitmap_iterator bi;
unsigned int i;
- VEC (tree, gc) *ma;
tree entry;
var_ann_t ta = var_ann (tag);
if (!MTAG_P (tag))
return;
- ma = may_aliases (tag);
- if (!ma)
+ aliases = may_aliases (tag);
+ if (!aliases)
return;
- for (i = 0; VEC_iterate (tree, ma, i, entry); i++)
+ EXECUTE_IF_SET_IN_BITMAP (aliases, 0, i, bi)
{
+ entry = referenced_var (i);
if (!unmodifiable_var_p (entry))
{
add_to_worklist (entry, worklist, worklist2, ta-escape_mask);
@@ -264,7 +267,8 @@ compute_tag_properties (void)
changed = false;
for (k = 0; VEC_iterate (tree, taglist, k, tag); k++)
{
- VEC (tree, gc) *ma;
+ bitmap ma;
+ bitmap_iterator bi;
unsigned int i;
tree entry;
bool tagcc = is_call_clobbered (tag);
@@ -277,8 +281,9 @@ compute_tag_properties (void)
if (!ma)
continue;
- for (i = 0; VEC_iterate (tree, ma, i, entry); i++)
+ EXECUTE_IF_SET_IN_BITMAP (ma, 0, i, bi)
{
+ entry = referenced_var (i);
/* Call clobbered entries cause the tag to be marked
call clobbered. */
if (!tagcc is_call_clobbered (entry))
@@ -508,8 +513,9 @@ sort_mp_info (VEC(mp_info_t,heap) *list)
static void
create_partition_for (mp_info_t mp_p)
{
+ bitmap_iterator bi;
tree mpt, sym;
- VEC(tree,gc) *aliases;
+ bitmap aliases;
unsigned i;
if (mp_p-num_vops = (long) MAX_ALIASED_VOPS)
@@ -556,11 +562,12 @@ create_partition_for (mp_info_t mp_p)
else
{
aliases = may_aliases (mp_p-var);
- gcc_assert (VEC_length (tree, aliases) 1);
+ gcc_assert (!bitmap_empty_p (aliases));
mpt = NULL_TREE;
- for (i = 0; VEC_iterate (tree, aliases, i, sym); i++)
+ EXECUTE_IF_SET_IN_BITMAP (aliases, 0, i, bi)
{
+ sym = referenced_var (i);
/* Only set the memory partition for aliased symbol SYM if
SYM does not belong to another partition. */
if (memory_partition (sym) == NULL_TREE)
@@ -614,11 +621,10 @@ rewrite_alias_set_for (tree tag, bitmap
else
{
/* Create a new alias set for TAG with the new partitions. */
- var_ann_t ann;
- ann = var_ann (tag);
- for (i = 0; VEC_iterate (tree, ann-may_aliases, i, sym); i++)
+ EXECUTE_IF_SET_IN_BITMAP (MTAG_ALIASES (tag), 0, i, bi)
{
+ sym = referenced_var (i);
mpt = memory_partition (sym);
if (mpt)
bitmap_set_bit (new_aliases, DECL_UID (mpt));
@@ -627,9 +633,7 @@ rewrite_alias_set_for (tree tag, bitmap
}
/* Rebuild the may-alias array for TAG. */
- VEC_free (tree, gc, ann-may_aliases);
- EXECUTE_IF_SET_IN_BITMAP (new_aliases, 0, i, bi)
- VEC_safe_push (tree, gc, ann-may_aliases, referenced_var (i));
+ bitmap_copy (MTAG_ALIASES (tag), new_aliases);
}
}
@@ -691,7 +695,10 @@ compute_memory_partitions (void)
/* Each reference to VAR will produce as many VOPs as elements
exist in its alias set. */
mp.var = var;
- mp.num_vops = VEC_length (tree, may_aliases (var));
+ if (!may_aliases (var))
+ mp.num_vops = 0;
+ else
+ mp.num_vops = bitmap_count_bits (may_aliases (var));
/* No point grouping singleton alias sets. */
if
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #17 from dberlin at gcc dot gnu dot org 2007-01-09 21:42 --- Subject: Re: Compiling FreeFem3d uses unreasonable amount of time and memory Try the attached, let me know how it goes. On 9 Jan 2007 21:17:05 -, rguenth at gcc dot gnu dot org [EMAIL PROTECTED] wrote: --- Comment #16 from rguenth at gcc dot gnu dot org 2007-01-09 21:17 --- Pling! -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089 --- Comment #18 from dberlin at gcc dot gnu dot org 2007-01-09 21:42 --- Created an attachment (id=12874) -- (http://gcc.gnu.org/bugzilla/attachment.cgi?id=12874action=view) -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #13 from hubicka at gcc dot gnu dot org 2006-12-23 14:26 --- Note that we've got another noticeable jump in memory consumption today (well at least it would be very important jump if we used just 28MB of memory for aliasing :). Is that also aliasing or shall be analyzed? Honza -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #14 from hubicka at gcc dot gnu dot org 2006-12-23 14:27 --- Well, actually the testcase now runs out of memory and ICE... -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #15 from dberlin at gcc dot gnu dot org 2006-12-23 16:21 --- Subject: Re: Compiling FreeFem3d uses unreasonable amount of time and memory On 23 Dec 2006 14:26:00 -, hubicka at gcc dot gnu dot org [EMAIL PROTECTED] wrote: --- Comment #13 from hubicka at gcc dot gnu dot org 2006-12-23 14:26 --- Note that we've got another noticeable jump in memory consumption today (well at least it would be very important jump if we used just 28MB of memory for aliasing :). Is that also aliasing or shall be analyzed? It's possible it was my aliasing fix, but it's hard to say. I had only seen cases where it increased mem usage ~3-5% (this was on large testcases too). It certainly shouldn't run out of memory. In any case, i'm working on testing bitmaps for may-aliases right now. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #8 from dnovillo at gcc dot gnu dot org 2006-12-13 17:32 --- http://gcc.gnu.org/ml/gcc-patches/2006-12/msg00959.html fixes the ICE in the operand scanner. The alias times should be back to saner values, but the memory consumption problem is still there. Still looking into that. -- dnovillo at gcc dot gnu dot org changed: What|Removed |Added AssignedTo|dnovillo at redhat dot com |dnovillo at gcc dot gnu dot ||org Status|UNCONFIRMED |ASSIGNED Ever Confirmed|0 |1 Last reconfirmed|-00-00 00:00:00 |2006-12-13 17:32:28 date|| http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #9 from dnovillo at gcc dot gnu dot org 2006-12-13 23:50 --- The memory problem is quite simple: We just have a *lot* of pointers and a *lot* of addressable symbols. Here is a breakdown of what happens on the first call to compute_may_aliases: During the first call to compute_may_aliases: 1- Size of cc1plus is 339Mb 2- Call to compute_points_to_sets grows to 355Mb (+4.72%) 3- Call to compute_flow_insensitive_aliasing grows to 364Mb (+2.54%) 4- Call to compute_flow_sensitive_aliasing grows to 667Mb (+83.2%) The reason for this tremendous growth is quite simple. There are 39,010 SSA name pointers and many of them have their own points-to set, which we store in that name's may-alias set. We grow to 667Mb in the last loop of compute_flow_sensitive_aliasing. One thing we could do is just not use flow-sensitive information in these cases. If we don't set SSA_NAME_PTR_INFO, everything will default to flow-insensitive information and things will Just Work. Perhaps using sparse bitmaps for the may-alias sets might help with memory consumption, but I found these bitmaps to slow down the operand scanner quite a bit in the past. May be worth a try. Danny, thoughts? -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #10 from dnovillo at gcc dot gnu dot org 2006-12-13 23:54 --- (In reply to comment #9) The memory problem is quite simple: We just have a *lot* of pointers and a *lot* of addressable symbols. Here is a breakdown of what happens on the first call to compute_may_aliases: During the first call to compute_may_aliases: This is in the function ffparse, BTW. Which has alias sets with 4.2 million elements. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #11 from dberlin at gcc dot gnu dot org 2006-12-14 01:11 --- Subject: Re: Compiling FreeFem3d uses unreasonable amount of time and memory On 13 Dec 2006 23:50:17 -, dnovillo at gcc dot gnu dot org [EMAIL PROTECTED] wrote: --- Comment #9 from dnovillo at gcc dot gnu dot org 2006-12-13 23:50 --- The memory problem is quite simple: We just have a *lot* of pointers and a *lot* of addressable symbols. Here is a breakdown of what happens on the first call to compute_may_aliases: During the first call to compute_may_aliases: 1- Size of cc1plus is 339Mb 2- Call to compute_points_to_sets grows to 355Mb (+4.72%) 3- Call to compute_flow_insensitive_aliasing grows to 364Mb (+2.54%) 4- Call to compute_flow_sensitive_aliasing grows to 667Mb (+83.2%) The reason for this tremendous growth is quite simple. There are 39,010 SSA name pointers and many of them have their own points-to set, which we store in that name's may-alias set. If they had their own set, then compute_points_to_set would have required more memory. We grow to 667Mb in the last loop of compute_flow_sensitive_aliasing. One thing we could do is just not use flow-sensitive information in these cases. If we don't set SSA_NAME_PTR_INFO, everything will default to flow-insensitive information and things will Just Work. Perhaps using sparse bitmaps for the may-alias sets might help with memory consumption, but I found these bitmaps to slow down the operand scanner quite a bit in the past. May be worth a try. Danny, thoughts? If compute_points_to_sets only grows memory by 26 meg, then that's all we need to represent the points-to sets of all these variables So we shoudl be able to do this without using more memory than that. Sadly, we create copies of the result of find_what_p_points_to, then transform it into an array. I'll give the bitmaps a try for the operand scanner and see how it works. I really don't think we should have to turn off information that is only taking 26 meg to store originally :) -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #12 from dnovillo at gcc dot gnu dot org 2006-12-14 02:50 --- (In reply to comment #11) I'll give the bitmaps a try for the operand scanner and see how it works. OK. Hopefully that won't introduce a huge slowdown in the operand scanner. Assigning back to you. -- dnovillo at gcc dot gnu dot org changed: What|Removed |Added AssignedTo|dnovillo at gcc dot gnu dot |dberlin at gcc dot gnu dot |org |org http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #7 from rguenth at gcc dot gnu dot org 2006-12-12 16:44 --- We're now ICEing in internal compiler error: in ssa_operand_alloc, at tree-ssa-operands.c:365 for the second testcase. -- rguenth at gcc dot gnu dot org changed: What|Removed |Added AssignedTo|unassigned at gcc dot gnu |dnovillo at redhat dot com |dot org | http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #5 from rguenth at gcc dot gnu dot org 2006-12-07 12:40 --- Numbers with mainline r119612 are FiniteElementMethod.cpp: 346MB, 46.86s parse.ff.cc: 1GB, 1236.53s so actually the latter is the biggest offender here ;) The compiler was built with release checking (but not bootstrapped, built with gcc 4.1.2). Flags for building parse.ff.cc are -O2 -fno-strict-aliasing (in case this makes a difference). -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #6 from rguenth at gcc dot gnu dot org 2006-12-07 14:07 --- tree alias analysis :1125.41 (91%) usr 1.57 (31%) sys1127.32 (91%) wall 199468 kB (19%) ggc PRE : 61.16 ( 5%) usr 0.83 (16%) sys 62.01 ( 5%) wall 2073 kB ( 0%) ggc TOTAL :1239.40 5.05 1244.82 1038165 kB So, easy whom to blame ;) -- rguenth at gcc dot gnu dot org changed: What|Removed |Added CC||dberlin at gcc dot gnu dot ||org http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #1 from rguenth at gcc dot gnu dot org 2006-12-06 17:17 --- Created an attachment (id=12758) -- (http://gcc.gnu.org/bugzilla/attachment.cgi?id=12758action=view) first testcase -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #2 from rguenth at gcc dot gnu dot org 2006-12-06 17:18 --- Created an attachment (id=12759) -- (http://gcc.gnu.org/bugzilla/attachment.cgi?id=12759action=view) second testcase -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #3 from rguenth at gcc dot gnu dot org 2006-12-06 17:20 --- Before the last big regressions on the mainline the first one took 350MB and 52s to build with -O2 on x86_64, the second one 685MB and 147s. That was g++ (GCC) 4.3.0 20061122 (experimental). -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
[Bug tree-optimization/30089] Compiling FreeFem3d uses unreasonable amount of time and memory
--- Comment #4 from dberlin at gcc dot gnu dot org 2006-12-07 04:48 --- On my machine, with an unoptimized cc1plus (IE stage1), the first one, at -O2 takes 150meg of memory total, and 221 seconds, with most of the time being verifiers. This is with local PTA changes to speed up PTA TOTAL : 221.5817.76 271.63 note: tree SSA verifier : 51.90 (23%) usr 0.89 ( 5%) sys 56.19 (21%) wall 74 kB ( 0%) ggc tree STMT verifier: 40.15 (18%) usr 0.88 ( 5%) sys 41.12 (15%) wall etc The *other* case is spending all it's alias time checking for dups in add_may_alias, which diego's patch should fix. There is also a ton of time elsewhere: tree alias analysis : 71.28 (21%) usr 1.66 ( 9%) sys 114.75 (26%) wall 18776 kB ( 5%) ggc tree SSA verifier : 33.40 (10%) usr 0.43 ( 2%) sys 34.55 ( 8%) wall 259 kB ( 0%) ggc tree STMT verifier: 30.43 ( 9%) usr 0.64 ( 3%) sys 31.04 ( 7%) wall 0 kB ( 0%) ggc PRE : 64.49 (19%) usr 0.64 ( 3%) sys 75.84 (17%) wall 1086 kB ( 0%) ggc scheduling 2 : 46.21 (14%) usr 0.35 ( 2%) sys 62.71 (14%) wall 2328 kB ( 1%) ggc TOTAL : 339.0919.05 444.66 (it was in a debugger for about 100s to get some idea of what was going on). But on my machine, it still only uses 350 meg of memory On x86_64, i expect about double memory usage. I also expect if i tested bootstrapped optimized compilers, i'd get the same times you are expecting, excluding checking time This leaves a few possibilities: 1 My local PTA improvements are helping this 2 Something is very different on x86_64 3 My preprocessing using Apple G++ 4.0.1 is giving very different code to play with than mainline does 4 The regression is already fixed :) I can either send you the patch to test for 1, or you can wait a few days for me to commit it -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=30089
