>
> I've managed to fix nearly all reported missed merged types for cc1.
> Remaining are those we'll never be able to merge (merging would
> change the SCC shape) and those that eventually end up refering
> to a TYPE_STUB_DECL with a make_anon_name () IDENTIFIER_NODE.
> For the latter we should find a middle-end solution as a followup
> in case it really matters.
>
> WPA statistics for stage2 cc1 are
>
> [WPA] read 2495082 SCCs of average size 2.380088
> [WPA] 5938514 tree bodies read in total
> [WPA] tree SCC table: size 524287, 260253 elements, collision ratio:
> 0.804380
> [WPA] tree SCC max chain length 11 (size 1)
> [WPA] Compared 429412 SCCs, 7039 collisions (0.016392)
> [WPA] Merged 426111 SCCs
> [WPA] Merged 3313709 tree bodies
> [WPA] Merged 225079 types
> [WPA] 162844 types prevailed (488124 associated trees)
> [WPA] Old merging code merges an additional 22412 types of which 21492 are
> in the same SCC with their prevailing variant (345831 and 323276
> associated trees)
>
> which shows there are 920 such TYPE_STUB_DECL issues and 21492
> merges the old code did that destroyed SCCs.
>
> Compared to the old code which only unified types and some selected
> trees (INTEGER_CSTs), the new code can immediately ggc_free the
> unified SCCs after they have been read which results in 55% of
> all tree bodies input into WPA stage to be freed (rather than hoping
> on secondary GC walk effects as the old code relied on), 58% of
> all types are recycled.
>
> Compile-time is at least on-par with the old code now and disk-usage
> grows by a moderate 10% due to the streaming format change.
On Firefox we now get
[WPA] read 43144472 SCCs of average size 2.270524
[WPA] 97960575 tree bodies read in total
[WPA] tree SCC table: size 8388593, 3936571 elements, collision ratio: 0.727773
[WPA] tree SCC max chain length 88 (size 1)
[WPA] Compared 19030240 SCCs, 337719 collisions (0.017746)
[WPA] Merged 18957101 SCCs
[WPA] Merged 58202930 tree bodies
[WPA] Merged 11800337 types
[WPA] 4506307 types prevailed (13699881 associated trees)
[WPA] Old merging code merges an additional 2174796 types of which 141104 are
in the same SCC with their prevailing variant (12811826 and 6367853 associated
trees)
[WPA] GIMPLE canonical type table: size 131071, 77871 elements, 4506442
searches, 1130903 collisions (ratio: 0.250953)
[WPA] GIMPLE canonical type hash table: size 8388593, 4506386 elements,
15712947 searches, 12879021 collisions (ratio: 0.819644)
and about 5GB of GGC memory after merging, overall the footprint is still
around 10GB.
It is notable improvmenet over old code however, where we needed 16GB.
[LTRANS] read 319710 SCCs of average size 6.184039
[LTRANS] 1977099 tree bodies read in total
[LTRANS] GIMPLE canonical type table: size 16381, 9569 elements, 473131
searches, 24899 collisions (ratio: 0.052626)
[LTRANS] GIMPLE canonical type hash table: size 1048573, 473076 elements,
1611909 searches, 1340396 collisions (ratio: 0.831558)
CPU: AMD64 family10, speed 2100 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask
of 0x00 (No unit mask) count 750000
samples % app name symbol name
45047 11.7420 lto1 inflate_fast
34224 8.9209 lto1 streamer_read_uhwi(lto_input_block*)
24630 6.4201 lto1 compare_tree_sccs_1(tree_node*,
tree_node*, tree_node***)
23205 6.0487 lto1 pointer_map_insert(pointer_map_t*,
void const*)
20829 5.4293 lto1 unpack_value_fields(data_in*,
bitpack_d*, tree_node*)
13545 3.5307 lto1 ht_lookup_with_hash(ht*, unsigned
char const*, unsigned long, unsigned int, ht_lookup_option)
12841 3.3472 libc-2.11.1.so memset
11840 3.0862 lto1 htab_find_slot_with_hash
11397 2.9708 lto1
streamer_tree_cache_insert_1(streamer_tree_cache_d*, tree_node*, unsigned int,
unsigned int*, bool)
11086 2.8897 lto1 lto_input_tree(lto_input_block*,
data_in*)
10522 2.7427 lto1 lto_input_tree_1(lto_input_block*,
data_in*, LTO_tags, unsigned int)
8853 2.3076 lto1 unify_scc(streamer_tree_cache_d*,
unsigned int, unsigned int, unsigned int, unsigned int)
8539 2.2258 lto1 hash_table<tree_scc_hasher,
xcallocator>::find_slot_with_hash(tree_scc const*, unsigned int, insert_option)
7987 2.0819 lto1 adler32
7743 2.0183 lto1
streamer_read_tree_body(lto_input_block*, data_in*, tree_node*)
Can't we free the pointer map in streamer after every SCC?
phase stream in : 244.05 (47%) usr 7.14 (25%) sys 252.74 (46%) wall
3478752 kB (93%) ggc
phase stream out : 222.50 (43%) usr 21.22 (73%) sys 243.97 (44%) wall
7160 kB ( 0%) ggc
garbage collection : 12.88 ( 2%) usr 0.00 ( 0%) sys 12.88 ( 2%) wall
0 kB ( 0%) ggc
ipa lto decl in : 177.88 (34%) usr 4.87 (17%) sys 184.28 (34%) wall
3887482 kB (103%) ggc
ipa lto decl out : 199.66 (39%) usr 11.30 (39%) sys 211.05 (38%) wall
0 kB ( 0%) ggc
ipa inlining heuristics : 29.49 ( 6%) usr 0.70 ( 2%) sys 30.21 ( 6%) wall
1353230 kB (36%) ggc
ipa lto decl merge : 26.57 ( 5%) usr 0.00 ( 0%) sys 26.58 ( 5%) wall
8269 kB ( 0%) ggc
ipa lto cgraph merge : 12.78 ( 2%) usr 0.04 ( 0%) sys 12.82 ( 2%) wall
142112 kB ( 4%) ggc
TOTAL : 517.75 29.13 548.71
3758407 kB
The longest running ltrans add another 400 seconds.
combiner : 16.16 ( 4%) usr 0.08 ( 1%) sys 16.53 ( 4%) wall
205251 kB ( 6%) ggc
integrated RA : 47.97 (12%) usr 0.21 ( 3%) sys 48.39 (12%) wall
391655 kB (12%) ggc
LRA hard reg assignment : 158.64 (39%) usr 0.02 ( 0%) sys 158.74 (38%) wall
0 kB ( 0%) ggc
TOTAL : 404.51 8.39 414.01
3215235 kB
Honza
