[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[rguenth at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

Richard Guenther rguenth at gcc dot gnu.org changed:

   What|Removed |Added

   Target Milestone|--- |4.8.0

--- Comment #5 from Richard Guenther rguenth at gcc dot gnu.org 2012-08-21 
08:26:03 UTC ---
(In reply to comment #4)
 It was introduced between revision 189101 and revision 189664
 on cxx-conversion branch.  Unfortunately, since branch was broken
 between those 2 revisions, I can't bisect further.

I only see

r189664 | dnovillo | 2012-07-19 16:40:11 +0200 (Thu, 19 Jul 2012) | 3 lines


Merge from trunk rev 189106.

inbetween those two revisions. r189668 is another merge from trunk,
so is r188963.  So I suspect a mismerge in r189106, r188963 was

r188963 | dnovillo | 2012-06-25 23:10:33 +0200 (Mon, 25 Jun 2012) | 7 lines

Merge from trunk

Merged revisions
188725-188729,188731,188733,188738-188749,188751-188755,188759,
188764-188765,188771,188776,188778,188780-188789,188791-188796,188802-188808,188
814-188815,188818,188820-188824,188826-188827,188829,188833,188838,188840-188843
,188847,188849,188852-188853,188856-188860,188865-188872,188874-188876,10-18
8881,14-15,188891,188893,188900-188902,188906-188909,188913,188915-18891
8,188922 via svnmerge from 
svn+ssh://gcc.gnu.org/svn/gcc/trunk

HJ, maybe you can help narrowing down things by trying different optimization
levels?  I see that using a memleak checker may not be possible with
10G memory usage.

Note that -fmem-report does not track all allocations, esp. heap hashtables
are not tracked.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #6 from dnovillo at google dot com dnovillo at google dot com 
2012-08-21 13:38:24 UTC ---
On 2012-08-20 22:59 , hjl.tools at gmail dot com wrote:
 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

 --- Comment #4 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 02:59:15 
 UTC ---
 It was introduced between revision 189101 and revision 189664
 on cxx-conversion branch.  Unfortunately, since branch was broken
 between those 2 revisions, I can't bisect further.


There was no rev 189101 in cxx-conversion.  That is a trunk revision. 
In that range of revisions, there are only merges from trunk until rev 
188129, which introduces the new hash table.

Prior to that, we have rev 188059, which makes cosmetic changes to 
configure.ac.

If it's related to the hash table, then comparing rev 188059 vs rev 
188129 may show the regression.


Diego.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #7 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 13:58:05 
UTC ---
(In reply to comment #6)
 
 If it's related to the hash table, then comparing rev 188059 vs rev 
 188129 may show the regression.
 

Neither rev 188059 nor rev 188129 will build:

../../../../gcc/gcc/graphite-sese-to-poly.c: In function \u2018void
build_sese_conditions_before(dom_walk_data*, basic_block)\u2019:
../../../../gcc/gcc/graphite-sese-to-poly.c:1357:2: error: call of overloaded
\u2018VEC_safe_push_1(vec_tgimple_statement_d***, NULL, const char [44], int,
const char [29])\u2019 is ambiguous
../../../../gcc/gcc/graphite-sese-to-poly.c:1357:2: note: candidates are: 
In file included from ../../../../gcc/gcc/basic-block.h:25:0,
 from ../../../../gcc/gcc/tree-flow.h:27,
 from ../../../../gcc/gcc/graphite-sese-to-poly.c:24:
../../../../gcc/gcc/vec.h:674:1: note: T VEC_safe_push_1(vec_tT**, T, const
char*, unsigned int, const char*) [with T = gimple_statement_d*;
vec_allocation_t A = (vec_allocation_t)0u]
../../../../gcc/gcc/vec.h:682:1: note: T* VEC_safe_push_1(vec_tT**, const T*,
const char*, unsigned int, const char*) [with T = gimple_statement_d*;
vec_allocation_t A = (vec_allocation_t)0u]
make[2]: *** [graphite-sese-to-poly.o] Error 1
  2692,40   99%

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #8 from dnovillo at google dot com dnovillo at google dot com 
2012-08-21 14:06:34 UTC ---
On 2012-08-21 09:58 , hjl.tools at gmail dot com wrote:
 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

 --- Comment #7 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 13:58:05 
 UTC ---
 (In reply to comment #6)

 If it's related to the hash table, then comparing rev 188059 vs rev
 188129 may show the regression.


 Neither rev 188059 nor rev 188129 will build:

 ../../../../gcc/gcc/graphite-sese-to-poly.c: In function \u2018void
 build_sese_conditions_before(dom_walk_data*, basic_block)\u2019:
 ../../../../gcc/gcc/graphite-sese-to-poly.c:1357:2: error: call of overloaded
 \u2018VEC_safe_push_1(vec_tgimple_statement_d***, NULL, const char [44], 
 int,
 const char [29])\u2019 is ambiguous
 ../../../../gcc/gcc/graphite-sese-to-poly.c:1357:2: note: candidates are:
 In file included from ../../../../gcc/gcc/basic-block.h:25:0,
   from ../../../../gcc/gcc/tree-flow.h:27,
   from ../../../../gcc/gcc/graphite-sese-to-poly.c:24:
 ../../../../gcc/gcc/vec.h:674:1: note: T VEC_safe_push_1(vec_tT**, T, const
 char*, unsigned int, const char*) [with T = gimple_statement_d*;
 vec_allocation_t A = (vec_allocation_t)0u]
 ../../../../gcc/gcc/vec.h:682:1: note: T* VEC_safe_push_1(vec_tT**, const 
 T*,
 const char*, unsigned int, const char*) [with T = gimple_statement_d*;
 vec_allocation_t A = (vec_allocation_t)0u]
 make[2]: *** [graphite-sese-to-poly.o] Error 1
2692,40   
 99%


Huh, odd.  Can you try this patchlet on top of those revs?  It builds 
for me with this applied:

diff --git a/gcc/graphite-sese-to-poly.c b/gcc/graphite-sese-to-poly.c
index cdabd73..5712e58 100644
--- a/gcc/graphite-sese-to-poly.c
+++ b/gcc/graphite-sese-to-poly.c
@@ -1354,7 +1354,7 @@ build_sese_conditions_before (struct dom_walk_data 
*dw_data,
if (e-flags  EDGE_TRUE_VALUE)
 VEC_safe_push (gimple, heap, *cases, stmt);
else
-   VEC_safe_push (gimple, heap, *cases, NULL);
+   VEC_safe_push (gimple, heap, *cases, (gimple) NULL);
  }

gbb = gbb_from_bb (bb);

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #9 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 16:20:37 
UTC ---
Revision 188059 is bad:

f951: out of memory allocating 36872 bytes after a total of 583266304 bytes

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #10 from dnovillo at google dot com dnovillo at google dot com 
2012-08-21 16:44:10 UTC ---
On Tue, Aug 21, 2012 at 12:20 PM, hjl.tools at gmail dot com
gcc-bugzi...@gcc.gnu.org wrote:
 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

 --- Comment #9 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 16:20:37 
 UTC ---
 Revision 188059 is bad:

 f951: out of memory allocating 36872 bytes after a total of 583266304 bytes

Thanks.  Does rev 188129 show the same thing?  The next revisions to try are:

188040 (TREE_CHECK macros)
187954 (merge from trunk)
187836 (initial VEC conversion)
187735 (merge from trunk)

I now have access to SPEC2006, I'll try a build.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

H.J. Lu hjl.tools at gmail dot com changed:

   What|Removed |Added

 Status|UNCONFIRMED |NEW
   Last reconfirmed|2012-08-20 00:00:00 |2012-08-21
 Ever Confirmed|0   |1

--- Comment #11 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 16:51:24 
UTC ---
It is caused by revision 187836:

http://gcc.gnu.org/ml/gcc-cvs/2012-05/msg00833.html

The C++ implementation of vec.[ch] has a massive memory leak.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

Diego Novillo dnovillo at gcc dot gnu.org changed:

   What|Removed |Added

 Status|NEW |ASSIGNED

--- Comment #12 from Diego Novillo dnovillo at gcc dot gnu.org 2012-08-21 
16:55:34 UTC ---
Thanks.  I'll work on a fix.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #13 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 17:10:09 
UTC ---
It can be reproduced with -frecord-marker=4 -O -funswitch-loops.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #14 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 17:41:10 
UTC ---
It failed even with

diff --git a/gcc/tree-ssa-loop.c b/gcc/tree-ssa-loop.c
index 3d650bf..30ac4b5 100644
--- a/gcc/tree-ssa-loop.c
+++ b/gcc/tree-ssa-loop.c
@@ -149,7 +149,7 @@ tree_ssa_loop_unswitch (void)
 static bool
 gate_tree_ssa_loop_unswitch (void)
 {
-  return flag_unswitch_loops != 0;
+  return false;
 }

 struct gimple_opt_pass pass_tree_unswitch =

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #15 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 17:57:59 
UTC ---
It failed with

diff --git a/gcc/passes.c b/gcc/passes.c
index b6fe18e..10174c4 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -1449,7 +1449,6 @@ init_optimization_passes (void)
 NEXT_PASS (pass_lim);
 NEXT_PASS (pass_copy_prop);
 NEXT_PASS (pass_dce_loop);
-NEXT_PASS (pass_tree_unswitch);
 NEXT_PASS (pass_scev_cprop);
 NEXT_PASS (pass_record_bounds);
 NEXT_PASS (pass_check_data_deps);

Somehow just processing the -funswitch-loops command-line option
triggers this problem.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #16 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 18:08:49 
UTC ---
There are:

opts.c:typedef char *char_p; /* For DEF_VEC_P.  */
opts.c:DEF_VEC_P(char_p);
opts.c:DEF_VEC_ALLOC_P(char_p,heap);
opts-global.c:typedef const char *const_char_p; /* For DEF_VEC_P.  */
opts-global.c:DEF_VEC_P(const_char_p);
opts-global.c:DEF_VEC_ALLOC_P(const_char_p,heap);

Will they cause problems if other files define similar types?

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #17 from dnovillo at google dot com dnovillo at google dot com 
2012-08-21 18:19:10 UTC ---
On 2012-08-21 14:08 , hjl.tools at gmail dot com wrote:
 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

 --- Comment #16 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 18:08:49 
 UTC ---
 There are:

 opts.c:typedef char *char_p; /* For DEF_VEC_P.  */
 opts.c:DEF_VEC_P(char_p);
 opts.c:DEF_VEC_ALLOC_P(char_p,heap);
 opts-global.c:typedef const char *const_char_p; /* For DEF_VEC_P.  */
 opts-global.c:DEF_VEC_P(const_char_p);
 opts-global.c:DEF_VEC_ALLOC_P(const_char_p,heap);

 Will they cause problems if other files define similar types?


They shouldn't.  DEF_VEC_* expands to a NOP now.  The allocation 
strategy is only needed during the actual allocation call.  So, in the 
case of opts.c, that would be in add_comma_separated_to_vector() (the 
call to VEC_safe_push).

Those two vectors are only used for -finstrument-options..., though.  So 
that does not seem related.

The call to postpone_unknown_option_warning in opts-global.c seems also 
unrelated.  It's only used when processing unknown options.  That vector 
is popped when the unknown options are freed, so that can't be it either.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #18 from dnovillo at google dot com dnovillo at google dot com 
2012-08-21 18:31:51 UTC ---
OK, I think this is the hunk that's causing grief:

diff --git a/gcc/df-scan.c b/gcc/df-scan.c
index 39f444f..35100d1 100644
--- a/gcc/df-scan.c
+++ b/gcc/df-scan.c
@@ -4392,7 +4392,6 @@ df_bb_verify (basic_block bb)
if (!INSN_P (insn))
  continue;
df_insn_refs_verify (collection_rec, bb, insn, true);
-  df_free_collection_rec (collection_rec);
  }

/* Do the artificial defs and uses.  */


I remember that I ran into this during the VEC conversion 
(http://gcc.gnu.org/ml/gcc/2012-05/msg00271.html) and after some 
discussion I ended up convincing myself that taking it out was harmless. 
  Clearly, I was wrong.

I've hooked gdb to the running f951 and it's stuck in df_bb_verify().

Odd that this has not triggered anywhere else.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #19 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 18:54:45 
UTC ---
(In reply to comment #15)
 It failed with
 
 diff --git a/gcc/passes.c b/gcc/passes.c
 index b6fe18e..10174c4 100644
 --- a/gcc/passes.c
 +++ b/gcc/passes.c
 @@ -1449,7 +1449,6 @@ init_optimization_passes (void)
  NEXT_PASS (pass_lim);
  NEXT_PASS (pass_copy_prop);
  NEXT_PASS (pass_dce_loop);
 -NEXT_PASS (pass_tree_unswitch);
  NEXT_PASS (pass_scev_cprop);
  NEXT_PASS (pass_record_bounds);
  NEXT_PASS (pass_check_data_deps);
 
 Somehow just processing the -funswitch-loops command-line option
 triggers this problem.

With --enable-gather-detailed-mem-stats, I got

Alloc-pool Kind Elt size  Pools  Allocated (elts)Peak
(elts)Leak (elts)

-df_scan ref base   64 18   24808192(387628)   11869056(   
185454)  0( 0)
-df_scan ref artificial 72 18   15168528(210674)2044944(   
 28402)  0( 0)
+df_scan ref base   64 18  513091264(   8017051)  500077440(  
7813710)  0( 0)
+df_scan ref artificial 72 18  599905368(   8332019)2044944(   
 28402)  0( 0)
 elt_loc_list   32 277982112(249441)2399488(   
 74984)  0( 0)  
-df_scan ref regular72 18   71483184(992822)   45955584(   
638272)  0( 0)
+df_scan ref regular72 18 2091195360(  29044380) 2065579776( 
28688608)  0( 0)
 df_scan insn   56 187681016(137161)3340848(   
 59658)  0( 0)

-Total  15775  253131240
+Total  16067 3345899232

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #20 from dnovillo at google dot com dnovillo at google dot com 
2012-08-21 19:07:33 UTC ---
On 2012-08-21 14:54 , hjl.tools at gmail dot com wrote:

 With --enable-gather-detailed-mem-stats, I got

 Alloc-pool Kind Elt size  Pools  Allocated (elts)Peak
 (elts)Leak (elts)

 -df_scan ref base   64 18   24808192(387628)   11869056(
 185454)  0( 0)
 -df_scan ref artificial 72 18   15168528(210674)2044944(
   28402)  0( 0)
 +df_scan ref base   64 18  513091264(   8017051)  500077440(
 7813710)  0( 0)
 +df_scan ref artificial 72 18  599905368(   8332019)2044944(
   28402)  0( 0)
   elt_loc_list   32 277982112(249441)2399488(
   74984)  0( 0)
 -df_scan ref regular72 18   71483184(992822)   45955584(
 638272)  0( 0)
 +df_scan ref regular72 18 2091195360(  29044380) 2065579776(
 28688608)  0( 0)
   df_scan insn   56 187681016(137161)3340848(
   59658)  0( 0)

 -Total  15775  253131240
 +Total  16067 3345899232


That agrees with what I found, thanks.  I've added a link to the 
discussion about the df verifier.  The vectors need to be cleared, but I 
can't just free the vectors:

Stack vectors must be initially allocated with VEC_stack_alloc.
gcc/df-scan.c: In function 'unsigned int df_count_refs(bool, bool, bool)':
gcc/df-scan.c:1507:1: internal compiler error: in vec_reserve, at vec.h:
  }

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[steven at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #21 from Steven Bosscher steven at gcc dot gnu.org 2012-08-21 
19:19:58 UTC ---
(In reply to comment #18)
 Odd that this has not triggered anywhere else.

It may have triggered elsewhere, see PR54343 ...

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #22 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 19:27:50 
UTC ---
This seems to work:

diff --git a/gcc/df-scan.c b/gcc/df-scan.c
index 35100d1..39f444f 100644
--- a/gcc/df-scan.c
+++ b/gcc/df-scan.c
@@ -4392,6 +4392,7 @@ df_bb_verify (basic_block bb)
   if (!INSN_P (insn))
 continue;
   df_insn_refs_verify (collection_rec, bb, insn, true);
+  df_free_collection_rec (collection_rec);
 }

   /* Do the artificial defs and uses.  */
diff --git a/gcc/vec.h b/gcc/vec.h
index cc7e819..3a298ff 100644
--- a/gcc/vec.h
+++ b/gcc/vec.h
@@ -1031,21 +1031,9 @@ vec_reserve (vec_tT *vec_, int reserve MEM_STAT_DECL)
   sizeof (T), false
   PASS_MEM_STAT);
   else
-{
-  /* Only allow stack vectors when re-growing them.  The initial
- allocation of stack vectors must be done with the
- VEC_stack_alloc macro, because it uses alloca() for the
- allocation.  */
-  if (vec_ == NULL)
-{
-  fprintf (stderr, Stack vectors must be initially allocated 
-   with VEC_stack_alloc.\n);
-  gcc_unreachable ();
-}
-  return (vec_tT *) vec_stack_o_reserve (vec_, reserve,
-   offsetof (vec_tT, vec),
-   sizeof (T) PASS_MEM_STAT);
-}
+return (vec_tT *) vec_stack_o_reserve (vec_, reserve,
+ offsetof (vec_tT, vec),
+ sizeof (T) PASS_MEM_STAT);
 }

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #23 from dnovillo at google dot com dnovillo at google dot com 
2012-08-21 19:50:12 UTC ---
On 2012-08-21 15:27 , hjl.tools at gmail dot com wrote:
 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

 --- Comment #22 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 19:27:50 
 UTC ---
 This seems to work:

 diff --git a/gcc/df-scan.c b/gcc/df-scan.c
 index 35100d1..39f444f 100644
 --- a/gcc/df-scan.c
 +++ b/gcc/df-scan.c
 @@ -4392,6 +4392,7 @@ df_bb_verify (basic_block bb)
 if (!INSN_P (insn))
   continue;
 df_insn_refs_verify (collection_rec, bb, insn, true);
 +  df_free_collection_rec (collection_rec);
   }

 /* Do the artificial defs and uses.  */
 diff --git a/gcc/vec.h b/gcc/vec.h
 index cc7e819..3a298ff 100644
 --- a/gcc/vec.h
 +++ b/gcc/vec.h
 @@ -1031,21 +1031,9 @@ vec_reserve (vec_tT *vec_, int reserve MEM_STAT_DECL)
 sizeof (T), false
 PASS_MEM_STAT);
 else
 -{
 -  /* Only allow stack vectors when re-growing them.  The initial
 - allocation of stack vectors must be done with the
 - VEC_stack_alloc macro, because it uses alloca() for the
 - allocation.  */
 -  if (vec_ == NULL)
 -{
 -  fprintf (stderr, Stack vectors must be initially allocated 
 -   with VEC_stack_alloc.\n);
 -  gcc_unreachable ();
 -}
 -  return (vec_tT *) vec_stack_o_reserve (vec_, reserve,
 -   offsetof (vec_tT, vec),
 -   sizeof (T) PASS_MEM_STAT);
 -}
 +return (vec_tT *) vec_stack_o_reserve (vec_, reserve,
 + offsetof (vec_tT, vec),
 + sizeof (T) PASS_MEM_STAT);
   }


The problem with this is that you are switching a stack vec into a heap 
vec.  This may not always be what the caller wanted.


The other alternative is to truncate the vectors after the call to 
df_insn_refs_verify().

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #24 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 19:53:14 
UTC ---
(In reply to comment #23)
 
 The problem with this is that you are switching a stack vec into a heap 
 vec.  This may not always be what the caller wanted.

My patch just restores the old behavior.

 
 The other alternative is to truncate the vectors after the call to 
 df_insn_refs_verify().

This should be a separate patch, not the part of C++ conversion.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[dnovillo at google dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #25 from dnovillo at google dot com dnovillo at google dot com 
2012-08-21 20:49:16 UTC ---
On 2012-08-21 15:53 , hjl.tools at gmail dot com wrote:
 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

 --- Comment #24 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 19:53:14 
 UTC ---
 (In reply to comment #23)

 The problem with this is that you are switching a stack vec into a heap
 vec.  This may not always be what the caller wanted.

 My patch just restores the old behavior.

You are right.  This was always the case.  I added the extra check to 
guard against inadvertent *initial* heap allocations for stack vectors.

But now that I see the old code, this was always the case.  The 
subsequent stack operations after the first round around the loop will 
move the stacks into the heap.

The patch is OK with a ChangeLog and bootstrap testing.


Thanks!  Diego.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #26 from hjl at gcc dot gnu.org hjl at gcc dot gnu.org 2012-08-21 
21:07:07 UTC ---
Author: hjl
Date: Tue Aug 21 21:07:01 2012
New Revision: 190576

URL: http://gcc.gnu.org/viewcvs?root=gccview=revrev=190576
Log:
Restore df_free_collection_rec call in df_bb_verify

PR middle-end/54332
* df-scan.c (df_bb_verify): Restore df_free_collection_rec call
inside the insn traversal loop.

* vec.h (vec_reserve): Remove the stack allocation check.

Modified:
trunk/gcc/ChangeLog
trunk/gcc/df-scan.c
trunk/gcc/vec.h

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

H.J. Lu hjl.tools at gmail dot com changed:

   What|Removed |Added

 Status|ASSIGNED|RESOLVED
 Resolution||FIXED

--- Comment #27 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 21:11:11 
UTC ---
Fixed.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

H.J. Lu hjl.tools at gmail dot com changed:

   What|Removed |Added

 CC||areg.melikadamyan at gmail
   ||dot com, dnovillo at google
   ||dot com

--- Comment #1 from H.J. Lu hjl.tools at gmail dot com 2012-08-20 20:53:14 
UTC ---
It is caused by revision 190402:

http://gcc.gnu.org/ml/gcc-cvs/2012-08/msg00379.html

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #2 from H.J. Lu hjl.tools at gmail dot com 2012-08-20 21:18:00 
UTC ---
Revision 190401 takes 512MB virtual memory to compile module_domain.fppized.f90
while revision 190402 takes 10GB. This is a 20x increase.

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[steven at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

Steven Bosscher steven at gcc dot gnu.org changed:

   What|Removed |Added

   Keywords||memory-hog
 Status|UNCONFIRMED |NEW
   Last reconfirmed||2012-08-20
 CC||steven at gcc dot gnu.org
 Ever Confirmed|0   |1

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

H.J. Lu hjl.tools at gmail dot com changed:

   What|Removed |Added

 Status|NEW |UNCONFIRMED
 Ever Confirmed|1   |0

--- Comment #3 from H.J. Lu hjl.tools at gmail dot com 2012-08-20 21:41:46 
UTC ---
Revision 190402 memory stat:

Memory consumption before IPA
Number of expanded macros: 0

Line Table allocations during the compilation process
Number of ordinary maps used:3 
Ordinary map used size:120 
Number of ordinary maps allocated: 409 
Ordinary maps allocated size:   15k
Number of macro maps used:   0 
Macro maps used size:0 
Macro maps locations size:   0 
Macro maps size: 0 
Duplicated maps locations size:  0 
Total allocated maps size:  15k
Total used maps size:  120 

Memory still allocated at the end of the compilation process
Size   AllocatedUsedOverhead
8 36k 11k   1080 
1696k 82k   2112 
32   132k 62k   2376 
64  1460k   1338k 22k
256  440k432k   6160 
512 81923584 112 
1024  16k 11k224 
2048  12k   8192 168 
4096  84k 84k   1176 
8192  32k 32k224 
16384 48k 48k168 
65536 64k 64k 56 
131072128k128k 56 
262144512k512k112 
2468k 32k   1224 
40  3740k   1482k 58k
48  2108k876k 32k
56  1396k913k 21k
7228k   2232 392 
80   104k101k   1456 
8816k   2200 224 
96  4252k   4140k 58k
112   16k 12k224 
120 81925160 112 
184   72k 67k   1008 
128   36k 14k504 
152 4324k   4168k 59k
168  788k362k 10k
160  504k487k   7056 
104 2836k   2770k 38k
312  212k208k   2968 
136 40962448  56 
Total 23M 18M331k

String pool
entries10710
identifiers7074 (66.05%)
slots16384
deleted3636
bytes86k (17592186044415M overhead)
table size128k
coll/search0.6146
ins/search0.2503
avg. entry8.25 bytes (+/- 8.27)
longest entry46
(No per-node statistics)
Type hash: size 2039, 996 elements, 1.024948 collisions
DECL_DEBUG_EXPR  hash: size 1021, 294 elements, 0.128814 collisions
DECL_VALUE_EXPR  hash: size 1021, 0 elements, 0.00 collisions
No gimple statistics
No RTX statistics

Alias oracle query stats:
  refs_may_alias_p: 0 disambiguations, 0 queries
  ref_maybe_used_by_call_p: 0 disambiguations, 0 queries
  call_may_clobber_ref_p: 0 disambiguations, 0 queries

PTA query stats:
  pt_solution_includes: 0 disambiguations, 0 queries
  pt_solutions_intersect: 0 disambiguations, 0 queries
Memory consumption after IPA
Number of expanded macros: 0

Line Table allocations during the compilation process
Number of ordinary maps used:3 
Ordinary map used size:120 
Number of ordinary maps allocated: 409 
Ordinary maps allocated size:   15k
Number of macro maps used:   0 
Macro maps used size:0 
Macro maps locations size:   0 
Macro maps size: 0 
Duplicated maps locations size:  0 
Total allocated maps size:  15k
Total used maps size:  120 

Memory still allocated at the end of the compilation process
Size   AllocatedUsedOverhead
8 36k 11k   1080 
1696k 82k   2112 
32   132k 70k   2376 
64  1460k   1160k 22k
256 1724k   1722k 23k
512 81925120 112 
1024  28k 27k392 
2048  12k   8192 168 
4096 116k116k   1624 
8192  32k 32k224 
16384   4544k   4544k 15k
65536 64k 64k 56 
131072128k128k 56 
262144512k512k112 
524288512k512k 56 
24   200k 74k   3600 
40  3740k   1280k 58k
48  2108k425k 32k
56  1396k910k 21k
7228k   2232 392 
80  4824k   2840k   

[Bug fortran/54332] [4.8 Regression] 481.wrf in SPEC CPU 2006 takes 10GB memory to compile

[hjl.tools at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54332

--- Comment #4 from H.J. Lu hjl.tools at gmail dot com 2012-08-21 02:59:15 
UTC ---
It was introduced between revision 189101 and revision 189664
on cxx-conversion branch.  Unfortunately, since branch was broken
between those 2 revisions, I can't bisect further.