Re: [PATCH] Fix compiling large files
On 03/15/2016 04:31 PM, Richard Henderson wrote: On 03/10/2016 08:20 PM, DJ Delorie wrote: I'm moving on to Plan C but I put a copy of the file on .../dj/foo.c.gz (195Mb) if anyone wants to find out why there's a 16Gb limit compiling it... With just the following, we successfully compile your file. It takes about 25 minutes and memory use tops out around 40GB. Which still seems insane for a 1.6GB input file consisting primarily of data for a static array, but that's a different problem. At this point we usually have a PR to go with all stage4 changes. But a meaningful PR is difficult to create, since the attachment would be too large. Perhaps a generator could be created, but since it wouldn't go in the testsuite it seems like a waste of time. Thoughts? r~ * line-map.c (new_linemap): Make alloc_size a size_t. OK. jeff
Re: [PATCH] Fix compiling large files
> At this point we usually have a PR to go with all stage4 > changes. But a meaningful PR is difficult to create, since > the attachment would be too large. Perhaps a generator could > be created, but since it wouldn't go in the testsuite it seems > like a waste of time. > > Thoughts? CPP macros grow exponentially, we could do it there, if we can get it to preserve (inject?) line breaks. But I wouldn't want to be the poor developer on a 4Gb 32-bit system trying to run it...
Re: [PATCH] Fix compiling large files
On Tue, Mar 15, 2016 at 03:31:44PM -0700, Richard Henderson wrote:
> On 03/10/2016 08:20 PM, DJ Delorie wrote:
> >I'm moving on to Plan C but I put a copy of the file on
> >.../dj/foo.c.gz (195Mb) if anyone wants to find out
> >why there's a 16Gb limit compiling it...
>
> With just the following, we successfully compile your file.
>
> It takes about 25 minutes and memory use tops out around 40GB.
> Which still seems insane for a 1.6GB input file consisting
> primarily of data for a static array, but that's a
> different problem.
>
> At this point we usually have a PR to go with all stage4
> changes. But a meaningful PR is difficult to create, since
> the attachment would be too large. Perhaps a generator could
> be created, but since it wouldn't go in the testsuite it seems
> like a waste of time.
>
> Thoughts?
>
>
> r~
>
>
> * line-map.c (new_linemap): Make alloc_size a size_t.
Ok for stage4.
> diff --git a/libcpp/line-map.c b/libcpp/line-map.c
> index 1fb634a..80d4e6b 100644
> --- a/libcpp/line-map.c
> +++ b/libcpp/line-map.c
> @@ -376,7 +376,7 @@ new_linemap (struct line_maps *set,
>if (LINEMAPS_USED (set, macro_map_p) == LINEMAPS_ALLOCATED (set,
> macro_map_p))
> {
>/* We ran out of allocated line maps. Let's allocate more. */
> - unsigned alloc_size;
> + size_t alloc_size;
>
>/* Cast away extern "C" from the type of xrealloc. */
>line_map_realloc reallocator = (set->reallocator
Jakub
