[webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Daniel Bratell]

Hi WebKittens,

As you might be aware, we at Opera now have Chromium based products which  
means that we compile WebKit quite a lot. A big issue for us and our  
automatic systems has been the long time needed to compile WebKit (inside  
Chromium), especially in Windows.


The big compilation time difference between Linux and Windows didn't seem  
to make sense so we did some analysis of what was going on and we think  
that we have found a cause. In WebKit include directives are without path,  
and instead the compiler is given a very long list of directories to  
search through. That process takes a lot of time in Windows. It must take  
some time in OSX and in Linux too but probably less.


As an experiment we took the (chromium) project webcore_dom, that normally  
compiles in 56 seconds in Windows on a generic computer and "fixed" it.  
Removing the many include paths in the build system and instead specifying  
the path in the include directives changed that to 42 seconds, a 25%  
reduction.


There is no reason to think the same reduction doesn't apply to all  
projects and then there are many many minutes to save for developers and  
build servers here.


Caveat: I don't know if the resulting binary is correct.

Is this something that has been talked about in the past, and would you be  
interested in replacing the long list of directories to search for every  
include with paths (relative some good base) directly in the include  
directives?


/Daniel
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Ryosuke Niwa]

On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell  wrote:

> Is this something that has been talked about in the past, and would you be
> interested in replacing the long list of directories to search for every
> include with paths (relative some good base) directly in the include
> directives?
>

Using explicit paths to include files has been talked about in the past;
e.g. https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html

The most convincing counter argument I can remember is that it'll make
refactoring harder because you'll have to update all #include's when you
move headers.

It would be great if we can figure out if this also improves the build time
on Mac/Linux.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Jarred Nicholls]

On Tue, Mar 26, 2013 at 2:37 PM, Ryosuke Niwa  wrote:

> On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell wrote:
>
>> Is this something that has been talked about in the past, and would you
>> be interested in replacing the long list of directories to search for every
>> include with paths (relative some good base) directly in the include
>> directives?
>>
>
> Using explicit paths to include files has been talked about in the past;
> e.g.
> https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html
>
> The most convincing counter argument I can remember is that it'll make
> refactoring harder because you'll have to update all #include's when you
> move headers.
>

A few tailored scripts make that an easier job.  We're all engineers after
all.


>
> It would be great if we can figure out if this also improves the build
> time on Mac/Linux.
>

Agreed.  Somewhat apples and oranges of course, but the noticeable slow
down when compiling WebKit versus the rest of Chromium, file to file, is
enormous.  Chromium does a full path include directive from the base of the
src by convention.  I'd love to see Windows builds sped up; definitely the
biggest pain point in continuous integration.


>
> - R. Niwa
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Ryosuke Niwa]

On Tue, Mar 26, 2013 at 11:42 AM, Jarred Nicholls  wrote:

> On Tue, Mar 26, 2013 at 2:37 PM, Ryosuke Niwa  wrote:
>
>> On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell wrote:
>>
>>> Is this something that has been talked about in the past, and would you
>>> be interested in replacing the long list of directories to search for every
>>> include with paths (relative some good base) directly in the include
>>> directives?
>>>
>>
>> Using explicit paths to include files has been talked about in the past;
>> e.g.
>> https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html
>>
>> The most convincing counter argument I can remember is that it'll make
>> refactoring harder because you'll have to update all #include's when you
>> move headers.
>>
>
> A few tailored scripts make that an easier job.  We're all engineers after
> all.
>

We have been talking about this for years, and we even have bugs:
https://bugs.webkit.org/show_bug.cgi?id=61772
Given that, I'm somewhat skeptical if we ever get around to it in
foreseeable future.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Geoffrey Garen]

> In WebKit include directives are without path, and instead the compiler is 
> given a very long list of directories to search through. That process takes a 
> lot of time in Windows. It must take some time in OSX and in Linux too but 
> probably less.

Can we make a first-order improvement just by making sure that paths are 
searched in order of likelihood?

Geoff
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Dirk Pranke]

On Tue, Mar 26, 2013 at 11:47 AM, Ryosuke Niwa  wrote:

> On Tue, Mar 26, 2013 at 11:42 AM, Jarred Nicholls wrote:
>
>> On Tue, Mar 26, 2013 at 2:37 PM, Ryosuke Niwa  wrote:
>>
>>> On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell wrote:
>>>
 Is this something that has been talked about in the past, and would you
 be interested in replacing the long list of directories to search for every
 include with paths (relative some good base) directly in the include
 directives?

>>>
>>> Using explicit paths to include files has been talked about in the past;
>>> e.g.
>>> https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html
>>>
>>> The most convincing counter argument I can remember is that it'll make
>>> refactoring harder because you'll have to update all #include's when you
>>> move headers.
>>>
>>
>> A few tailored scripts make that an easier job.  We're all engineers
>> after all.
>>
>
> We have been talking about this for years, and we even have bugs:
> https://bugs.webkit.org/show_bug.cgi?id=61772
> Given that, I'm somewhat skeptical if we ever get around to it in
> foreseeable future.
>
>
Scripts that update include paths from Source/WebCore/foo.h to
Source/Platform/foo.h are substantially easier than something to modify
XCode projects robustly ;)

If we have consensus that we should just switch to paths relative to Source
(or maybe a couple different options), that would be (IMO) a big win. It
sounds like Daniel & co. have already done the big bang conversion.


On Tue, Mar 26, 2013 at 11:58 AM, Geoffrey Garen  wrote:

> > In WebKit include directives are without path, and instead the compiler
> is given a very long list of directories to search through. That process
> takes a lot of time in Windows. It must take some time in OSX and in Linux
> too but probably less.
>
> Can we make a first-order improvement just by making sure that paths are
> searched in order of likelihood?
>

That actually sounds harder and more fragile to me than switch to full
paths across the board.

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Benjamin Poulain]

On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>
> If we have consensus that we should just switch to paths relative to
> Source (or maybe a couple different options), that would be (IMO) a big
> win. It sounds like Daniel & co. have already done the big bang conversion.
>

I think using full path would be a serious hit regarding hackability.

I would rather spend some time tweaking my compiler to cache each directory
content than waste time finding where is every single header I need to
include.

Benjamin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Ryosuke Niwa]

On Tue, Mar 26, 2013 at 11:37 AM, Ryosuke Niwa  wrote:

> On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell wrote:
>
>> Is this something that has been talked about in the past, and would you
>> be interested in replacing the long list of directories to search for every
>> include with paths (relative some good base) directly in the include
>> directives?
>>
>
> Using explicit paths to include files has been talked about in the past;
> e.g.
> https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html
>
> The most convincing counter argument I can remember is that it'll make
> refactoring harder because you'll have to update all #include's when you
> move headers.
>
> It would be great if we can figure out if this also improves the build
> time on Mac/Linux.
>

Another idea.  What you copied all headers into one directory (in a pre
compilation step) and used that as the single include directory?

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Dirk Pranke]

On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain wrote:

> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>
>> If we have consensus that we should just switch to paths relative to
>> Source (or maybe a couple different options), that would be (IMO) a big
>> win. It sounds like Daniel & co. have already done the big bang conversion.
>>
>
> I think using full path would be a serious hit regarding hackability.
>
> I would rather spend some time tweaking my compiler to cache each
> directory content than waste time finding where is every single header I
> need to include.
>
>
Interesting. I have the exact opposite experience, having to paw around to
figure out where "Font.h" actually lives rather than just seeing
"WebCore/platform/graphics/Font.h".

At any rate, to be clear, I would be in favor of that change but I'm not
expecting it to happen :).

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Alexis Menard]

On Tue, Mar 26, 2013 at 5:40 PM, Dirk Pranke  wrote:
> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
> wrote:
>>
>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>>>
>>> If we have consensus that we should just switch to paths relative to
>>> Source (or maybe a couple different options), that would be (IMO) a big win.
>>> It sounds like Daniel & co. have already done the big bang conversion.
>>
>>
>> I think using full path would be a serious hit regarding hackability.
>>
>> I would rather spend some time tweaking my compiler to cache each
>> directory content than waste time finding where is every single header I
>> need to include.
>>
>
> Interesting. I have the exact opposite experience, having to paw around to
> figure out where "Font.h" actually lives rather than just seeing
> "WebCore/platform/graphics/Font.h".

But a modern IDE can easily solve that for you. You just click on the
header and it opens it for you. Xcode, QtCreator handles it very well.

That said if you don't use an IDE then I understand your point :).

>
> At any rate, to be clear, I would be in favor of that change but I'm not
> expecting it to happen :).
>
> -- Dirk
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>



-- 
Software Engineer @
Intel Open Source Technology Center
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Dirk Pranke]

On Tue, Mar 26, 2013 at 1:42 PM, Alexis Menard  wrote:

> On Tue, Mar 26, 2013 at 5:40 PM, Dirk Pranke  wrote:
> > On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
> > wrote:
> >>
> >> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke 
> wrote
> >>>
> >>> If we have consensus that we should just switch to paths relative to
> >>> Source (or maybe a couple different options), that would be (IMO) a
> big win.
> >>> It sounds like Daniel & co. have already done the big bang conversion.
> >>
> >>
> >> I think using full path would be a serious hit regarding hackability.
> >>
> >> I would rather spend some time tweaking my compiler to cache each
> >> directory content than waste time finding where is every single header I
> >> need to include.
> >>
> >
> > Interesting. I have the exact opposite experience, having to paw around
> to
> > figure out where "Font.h" actually lives rather than just seeing
> > "WebCore/platform/graphics/Font.h".
>
> But a modern IDE can easily solve that for you. You just click on the
> header and it opens it for you. Xcode, QtCreator handles it very well.
>
> That said if you don't use an IDE then I understand your point :).
>
>
I think to me the fact that the paths are spelled out is both one less
action I have to do to get my question answered (I don't have to  click),
and a helpful reminder of the general layout of the code (and which
directories I might need to care about in relation to a particular file).

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[James Robinson]

Also keep in mind that currently different build systems hack the include
path up to have the same #include point to different headers depending on
the build configuration, so the path expansion for a given #include will
not be the same for all ports.  It's basically a very non-obvious way to do
#if PLATFORM() guards at include sites without looking like it.  For
instance there are 7 different versions of AuthenticationChallenge.h but
only one #include statement in ResourceLoader.cpp.

Consider:

$ find Source/WebCore -name "*.h" -printf "%f\n" | wc -l
3383

$ find Source/WebCore -name "*.h" -printf "%f\n" | sort | uniq | wc -l
3288


- James


On Tue, Mar 26, 2013 at 1:40 PM, Dirk Pranke  wrote:

> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain wrote:
>
>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>>
>>> If we have consensus that we should just switch to paths relative to
>>> Source (or maybe a couple different options), that would be (IMO) a big
>>> win. It sounds like Daniel & co. have already done the big bang conversion.
>>>
>>
>> I think using full path would be a serious hit regarding hackability.
>>
>> I would rather spend some time tweaking my compiler to cache each
>> directory content than waste time finding where is every single header I
>> need to include.
>>
>>
> Interesting. I have the exact opposite experience, having to paw around to
> figure out where "Font.h" actually lives rather than just seeing
> "WebCore/platform/graphics/Font.h".
>
> At any rate, to be clear, I would be in favor of that change but I'm not
> expecting it to happen :).
>
> -- Dirk
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Hugo Parente Lima]

On Tuesday, March 26, 2013 01:40:56 PM Dirk Pranke wrote:
> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
wrote:
> > On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
> > 
> >> If we have consensus that we should just switch to paths relative to
> >> Source (or maybe a couple different options), that would be (IMO) a big
> >> win. It sounds like Daniel & co. have already done the big bang
> >> conversion.
> > 
> > I think using full path would be a serious hit regarding hackability.
> > 
> > I would rather spend some time tweaking my compiler to cache each
> > directory content than waste time finding where is every single header I
> > need to include.
> 
> Interesting. I have the exact opposite experience, having to paw around to
> figure out where "Font.h" actually lives rather than just seeing
> "WebCore/platform/graphics/Font.h".
> 
> At any rate, to be clear, I would be in favor of that change but I'm not
> expecting it to happen :).

"I would be in favor of that change but I'm not expecting it to happen :)"

This summarizes my feel on any attempt to do a relative big change on WebKit 
project.

> -- Dirk

signature.asc
Description: This is a digitally signed message part.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] Fwd: Compiling WebKit up to 25% faster in Windows?

[Ryosuke Niwa]

Resent from the right email address.

On Tue, Mar 26, 2013 at 1:40 PM, Dirk Pranke  wrote:

> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain wrote:
>
>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>>
>>> If we have consensus that we should just switch to paths relative to
>>> Source (or maybe a couple different options), that would be (IMO) a big
>>> win. It sounds like Daniel & co. have already done the big bang conversion.
>>>
>>
>> I think using full path would be a serious hit regarding hackability.
>>
>> I would rather spend some time tweaking my compiler to cache each
>> directory content than waste time finding where is every single header I
>> need to include.
>>
>>
> Interesting. I have the exact opposite experience, having to paw around to
> figure out where "Font.h" actually lives rather than just seeing
> "WebCore/platform/graphics/Font.h".
>

On TextMate or Xcode, all I need need to is to type in Font.h and they'll
automatically find the right file for me.

I'll be annoyed if people started using same filenames in multiple
directories (this has already happened :( ) because then various tools will
start listing multiple files from all those directories.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Filip Pizlo]

On Mar 26, 2013, at 1:40 PM, Dirk Pranke  wrote:

> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain  wrote:
> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
> If we have consensus that we should just switch to paths relative to Source 
> (or maybe a couple different options), that would be (IMO) a big win. It 
> sounds like Daniel & co. have already done the big bang conversion.
> 
> I think using full path would be a serious hit regarding hackability.
> 
> I would rather spend some time tweaking my compiler to cache each directory 
> content than waste time finding where is every single header I need to 
> include.
> 
> 
> Interesting. I have the exact opposite experience, having to paw around to 
> figure out where "Font.h" actually lives rather than just seeing 
> "WebCore/platform/graphics/Font.h".
> 
> At any rate, to be clear, I would be in favor of that change but I'm not 
> expecting it to happen :).

I'm with Dirk on this.  Full path would help hackability for me.

I don't use an IDE, so I'll be typing more.  But I spend more time reading code 
than typing code.

Also we have a lot of stupid in header file naming right now.  For example the 
DFG calls the JSC::DFG::Node header "DFGNode.h", and puts it in 
JavaScriptCore/dfg/DFGNode.h.  So we duplicate the namespacing of 
JSC::DFG::Node in both the filename and the directory name.  Ridiculous!  If we 
had a discipline to always include using paths relative to Source, then we 
could just rename it to JavaScriptCore/dfg/Node.h.  That would make me happy.

-F



> 
> -- Dirk 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Hugo Parente Lima]

On Tuesday, March 26, 2013 01:47:26 PM James Robinson wrote:
> Also keep in mind that currently different build systems hack the include
> path up to have the same #include point to different headers depending on
> the build configuration, so the path expansion for a given #include will
> not be the same for all ports.  It's basically a very non-obvious way to do
> #if PLATFORM() guards at include sites without looking like it.  For
> instance there are 7 different versions of AuthenticationChallenge.h but
> only one #include statement in ResourceLoader.cpp.

I think this is another issue, but for sure a blocking issue for the one being 
discussed. 

> Consider:
> 
> $ find Source/WebCore -name "*.h" -printf "%f\n" | wc -l
> 3383
> 
> $ find Source/WebCore -name "*.h" -printf "%f\n" | sort | uniq | wc -l
> 3288
> 
> 
> - James
> 
> On Tue, Mar 26, 2013 at 1:40 PM, Dirk Pranke  wrote:
> > On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
wrote:
> >> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
> >> 
> >>> If we have consensus that we should just switch to paths relative to
> >>> Source (or maybe a couple different options), that would be (IMO) a big
> >>> win. It sounds like Daniel & co. have already done the big bang
> >>> conversion.
> >> 
> >> I think using full path would be a serious hit regarding hackability.
> >> 
> >> I would rather spend some time tweaking my compiler to cache each
> >> directory content than waste time finding where is every single header I
> >> need to include.
> > 
> > Interesting. I have the exact opposite experience, having to paw around to
> > figure out where "Font.h" actually lives rather than just seeing
> > "WebCore/platform/graphics/Font.h".
> > 
> > At any rate, to be clear, I would be in favor of that change but I'm not
> > expecting it to happen :).
> > 
> > -- Dirk
> > 
> > ___
> > webkit-dev mailing list
> > webkit-dev@lists.webkit.org
> > https://lists.webkit.org/mailman/listinfo/webkit-dev

signature.asc
Description: This is a digitally signed message part.
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Levi Weintraub]

+1. I also like that this will make layering violations clearer.


On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo  wrote:

>
> On Mar 26, 2013, at 1:40 PM, Dirk Pranke  wrote:
>
> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
> wrote:
>
>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>>
>>> If we have consensus that we should just switch to paths relative to
>>> Source (or maybe a couple different options), that would be (IMO) a big
>>> win. It sounds like Daniel & co. have already done the big bang conversion.
>>>
>>
>> I think using full path would be a serious hit regarding hackability.
>>
>> I would rather spend some time tweaking my compiler to cache each
>> directory content than waste time finding where is every single header I
>> need to include.
>>
>>
> Interesting. I have the exact opposite experience, having to paw around to
> figure out where "Font.h" actually lives rather than just seeing
> "WebCore/platform/graphics/Font.h".
>
> At any rate, to be clear, I would be in favor of that change but I'm not
> expecting it to happen :).
>
>
> I'm with Dirk on this.  Full path would help hackability for me.
>
> I don't use an IDE, so I'll be typing more.  But I spend more time reading
> code than typing code.
>
> Also we have a lot of stupid in header file naming right now.  For example
> the DFG calls the JSC::DFG::Node header "DFGNode.h", and puts it in
> JavaScriptCore/dfg/DFGNode.h.  So we duplicate the namespacing of
> JSC::DFG::Node in both the filename and the directory name.  Ridiculous!
>  If we had a discipline to always include using paths relative to Source,
> then we could just rename it to JavaScriptCore/dfg/Node.h.  That would make
> me happy.
>
> -F
>
>
>
>
> -- Dirk
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Ryosuke Niwa]

On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo  wrote:

>
> On Mar 26, 2013, at 1:40 PM, Dirk Pranke  wrote:
>
> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
> wrote:
>
>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>>
>>> If we have consensus that we should just switch to paths relative to
>>> Source (or maybe a couple different options), that would be (IMO) a big
>>> win. It sounds like Daniel & co. have already done the big bang conversion.
>>>
>>
>> I think using full path would be a serious hit regarding hackability.
>>
>> I would rather spend some time tweaking my compiler to cache each
>> directory content than waste time finding where is every single header I
>> need to include.
>>
>>
> Interesting. I have the exact opposite experience, having to paw around to
> figure out where "Font.h" actually lives rather than just seeing
> "WebCore/platform/graphics/Font.h".
>
> At any rate, to be clear, I would be in favor of that change but I'm not
> expecting it to happen :).
>
>
> I'm with Dirk on this.  Full path would help hackability for me.
>
> I don't use an IDE, so I'll be typing more.  But I spend more time reading
> code than typing code.
>
> Also we have a lot of stupid in header file naming right now.  For example
> the DFG calls the JSC::DFG::Node header "DFGNode.h", and puts it in
> JavaScriptCore/dfg/DFGNode.h.  So we duplicate the namespacing of
> JSC::DFG::Node in both the filename and the directory name.  Ridiculous!
>  If we had a discipline to always include using paths relative to Source,
> then we could just rename it to JavaScriptCore/dfg/Node.h.  That would make
> me happy.
>

That'll make me sad because then Xcode will then find both
WebCore/dom/Node.h and JavaScriptCode/dfg/Node.h when I type Node.h.

Unfortunately, we can't name Node.h in WebCore/dom DOMNode.h because
DOMNode.h already exists for Objective C bindings.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Jesus Sanchez-Palencia]

2013/3/26 Ryosuke Niwa :
> On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo  wrote:
>>
>>
>> On Mar 26, 2013, at 1:40 PM, Dirk Pranke  wrote:
>>
>> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
>> wrote:
>>>
>>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote

 If we have consensus that we should just switch to paths relative to
 Source (or maybe a couple different options), that would be (IMO) a big 
 win.
 It sounds like Daniel & co. have already done the big bang conversion.
>>>
>>>
>>> I think using full path would be a serious hit regarding hackability.
>>>
>>> I would rather spend some time tweaking my compiler to cache each
>>> directory content than waste time finding where is every single header I
>>> need to include.
>>>
>>
>> Interesting. I have the exact opposite experience, having to paw around to
>> figure out where "Font.h" actually lives rather than just seeing
>> "WebCore/platform/graphics/Font.h".
>>
>> At any rate, to be clear, I would be in favor of that change but I'm not
>> expecting it to happen :).
>>
>>
>> I'm with Dirk on this.  Full path would help hackability for me.
>>
>> I don't use an IDE, so I'll be typing more.  But I spend more time reading
>> code than typing code.
>>
>> Also we have a lot of stupid in header file naming right now.  For example
>> the DFG calls the JSC::DFG::Node header "DFGNode.h", and puts it in
>> JavaScriptCore/dfg/DFGNode.h.  So we duplicate the namespacing of
>> JSC::DFG::Node in both the filename and the directory name.  Ridiculous!  If
>> we had a discipline to always include using paths relative to Source, then
>> we could just rename it to JavaScriptCore/dfg/Node.h.  That would make me
>> happy.
>
>
> That'll make me sad because then Xcode will then find both
> WebCore/dom/Node.h and JavaScriptCode/dfg/Node.h when I type Node.h.
>
> Unfortunately, we can't name Node.h in WebCore/dom DOMNode.h because
> DOMNode.h already exists for Objective C bindings.


IMHO, we should be favoring code readability instead of a tool's feature.

+1 for full paths.

-jesus


>
> - R. Niwa
>
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Mark Rowe]

On 2013-03-26, at 11:37, Ryosuke Niwa  wrote:

> On Tue, Mar 26, 2013 at 11:21 AM, Daniel Bratell  wrote:
> Is this something that has been talked about in the past, and would you be 
> interested in replacing the long list of directories to search for every 
> include with paths (relative some good base) directly in the include 
> directives?
> 
> Using explicit paths to include files has been talked about in the past; e.g. 
> https://lists.webkit.org/pipermail/webkit-dev/2011-November/018632.html
> 
> The most convincing counter argument I can remember is that it'll make 
> refactoring harder because you'll have to update all #include's when you move 
> headers. 
> 
> It would be great if we can figure out if this also improves the build time 
> on Mac/Linux.

I’d be very interested on what effect, if any, this has on Mac too. My initial 
impression is that any improvement should be minimal since a header map is used 
to optimize for precisely this situation. If that mechanism isn’t helping then 
it would be useful to know.

- Mark

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Jarred Nicholls]

On Tue, Mar 26, 2013 at 5:17 PM, Jesus Sanchez-Palencia wrote:

> 2013/3/26 Ryosuke Niwa :
> > On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo  wrote:
> >>
> >>
> >> On Mar 26, 2013, at 1:40 PM, Dirk Pranke  wrote:
> >>
> >> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
> >> wrote:
> >>>
> >>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke 
> wrote
> 
>  If we have consensus that we should just switch to paths relative to
>  Source (or maybe a couple different options), that would be (IMO) a
> big win.
>  It sounds like Daniel & co. have already done the big bang conversion.
> >>>
> >>>
> >>> I think using full path would be a serious hit regarding hackability.
> >>>
> >>> I would rather spend some time tweaking my compiler to cache each
> >>> directory content than waste time finding where is every single header
> I
> >>> need to include.
> >>>
> >>
> >> Interesting. I have the exact opposite experience, having to paw around
> to
> >> figure out where "Font.h" actually lives rather than just seeing
> >> "WebCore/platform/graphics/Font.h".
> >>
> >> At any rate, to be clear, I would be in favor of that change but I'm not
> >> expecting it to happen :).
> >>
> >>
> >> I'm with Dirk on this.  Full path would help hackability for me.
> >>
> >> I don't use an IDE, so I'll be typing more.  But I spend more time
> reading
> >> code than typing code.
> >>
> >> Also we have a lot of stupid in header file naming right now.  For
> example
> >> the DFG calls the JSC::DFG::Node header "DFGNode.h", and puts it in
> >> JavaScriptCore/dfg/DFGNode.h.  So we duplicate the namespacing of
> >> JSC::DFG::Node in both the filename and the directory name.
>  Ridiculous!  If
> >> we had a discipline to always include using paths relative to Source,
> then
> >> we could just rename it to JavaScriptCore/dfg/Node.h.  That would make
> me
> >> happy.
> >
> >
> > That'll make me sad because then Xcode will then find both
> > WebCore/dom/Node.h and JavaScriptCode/dfg/Node.h when I type Node.h.
> >
> > Unfortunately, we can't name Node.h in WebCore/dom DOMNode.h because
> > DOMNode.h already exists for Objective C bindings.
>
>
> IMHO, we should be favoring code readability instead of a tool's feature.
>
> +1 for full paths.
>
> -jesus
>
>
My sentiment exactly.  And I agree with Dirk and Filip, seeing full paths
improves (my) hackability and clearly shows layer dependencies (and
violations).  I'd say that it would also help people understand where
certain headers live (educational) and to know which header specifically is
being included at compile time rather than it being a guessing game.  This
is just my opinion, but its rendered through experience working in both
chromium code and webkit code.  Having a few files named the same hasn't at
all effected my ability to find the right file (Sublime Text 2), though I
understand that argument.  In fact I'm forced to learn the difference and
distinguish the locations between similarly named files in the code base;
I've found it very helpful.

Jarred


>
> >
> > - R. Niwa
> >
> >
> > ___
> > webkit-dev mailing list
> > webkit-dev@lists.webkit.org
> > https://lists.webkit.org/mailman/listinfo/webkit-dev
> >
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Maciej Stachowiak]

On Mar 26, 2013, at 2:34 PM, Jarred Nicholls  wrote:

> On Tue, Mar 26, 2013 at 5:17 PM, Jesus Sanchez-Palencia  
> wrote:
> 2013/3/26 Ryosuke Niwa :
> > On Tue, Mar 26, 2013 at 1:53 PM, Filip Pizlo  wrote:
> >>
> >>
> >> On Mar 26, 2013, at 1:40 PM, Dirk Pranke  wrote:
> >>
> >> On Tue, Mar 26, 2013 at 1:29 PM, Benjamin Poulain 
> >> wrote:
> >>>
> >>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
> 
>  If we have consensus that we should just switch to paths relative to
>  Source (or maybe a couple different options), that would be (IMO) a big 
>  win.
>  It sounds like Daniel & co. have already done the big bang conversion.
> >>>
> >>>
> >>> I think using full path would be a serious hit regarding hackability.
> >>>
> >>> I would rather spend some time tweaking my compiler to cache each
> >>> directory content than waste time finding where is every single header I
> >>> need to include.
> >>>
> >>
> >> Interesting. I have the exact opposite experience, having to paw around to
> >> figure out where "Font.h" actually lives rather than just seeing
> >> "WebCore/platform/graphics/Font.h".
> >>
> >> At any rate, to be clear, I would be in favor of that change but I'm not
> >> expecting it to happen :).
> >>
> >>
> >> I'm with Dirk on this.  Full path would help hackability for me.
> >>
> >> I don't use an IDE, so I'll be typing more.  But I spend more time reading
> >> code than typing code.
> >>
> >> Also we have a lot of stupid in header file naming right now.  For example
> >> the DFG calls the JSC::DFG::Node header "DFGNode.h", and puts it in
> >> JavaScriptCore/dfg/DFGNode.h.  So we duplicate the namespacing of
> >> JSC::DFG::Node in both the filename and the directory name.  Ridiculous!  
> >> If
> >> we had a discipline to always include using paths relative to Source, then
> >> we could just rename it to JavaScriptCore/dfg/Node.h.  That would make me
> >> happy.
> >
> >
> > That'll make me sad because then Xcode will then find both
> > WebCore/dom/Node.h and JavaScriptCode/dfg/Node.h when I type Node.h.
> >
> > Unfortunately, we can't name Node.h in WebCore/dom DOMNode.h because
> > DOMNode.h already exists for Objective C bindings.
> 
> 
> IMHO, we should be favoring code readability instead of a tool's feature.
> 
> +1 for full paths.
> 
> -jesus
> 
> 
> My sentiment exactly.  And I agree with Dirk and Filip, seeing full paths 
> improves (my) hackability and clearly shows layer dependencies (and 
> violations).  I'd say that it would also help people understand where certain 
> headers live (educational) and to know which header specifically is being 
> included at compile time rather than it being a guessing game.  This is just 
> my opinion, but its rendered through experience working in both chromium code 
> and webkit code.  Having a few files named the same hasn't at all effected my 
> ability to find the right file (Sublime Text 2), though I understand that 
> argument.  In fact I'm forced to learn the difference and distinguish the 
> locations between similarly named files in the code base; I've found it very 
> helpful.

I realize that this is partly a matter of taste, and I do not feel too 
strongly. But I prefer being able to include headers by the short name without 
the include path. To me, listing the path feels like visual noise and doesn't 
aid my understanding. But I realize others may feel differently.

I also dislike having files with the same name, whether or not you include them 
by full path. If you do that, then you need to disambiguate to even talk about 
a file. 

Regards,
Maciej

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Daniel Bratell]

Den 2013-03-26 21:29:32 skrev Benjamin Poulain :


On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote


If we have consensus that we should just switch to paths relative to
Source (or maybe a couple different options), that would be (IMO) a big
win. It sounds like Daniel & co. have already done the big bang  
conversion.




I think using full path would be a serious hit regarding hackability.

I would rather spend some time tweaking my compiler to cache each  
directory

content than waste time finding where is every single header I need to
include.


I guess you mean that it will be more job moving files around, but that is  
a rather rare operation, while reading an include directive and wondering  
what it's part of is rather common (both for compilers, tools and humans).


I like the paths as a tool to indicate "module" dependencies. You can more  
easily see that a file depends on foo and bar (but not on fie) if you see:


#include "foo/object.h"
#include "foo/thing.h"
#include "bar/stuff.h"

That will tell you useful things, and avoid making layering violations by  
accident.


But I realize it's a question of style and as such there is not a right  
and a wrong, unless there are other factors. And here we have the  
seemingly heavy compilation time cost for it which I think is a strong  
argument against delegating the task of finding the header files to the  
compiler.


/Daniel
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Daniel Bratell]

Den 2013-03-26 21:20:10 skrev Dirk Pranke :

If we have consensus that we should just switch to paths relative to  
Source

(or maybe a couple different options), that would be (IMO) a big win. It
sounds like Daniel & co. have already done the big bang conversion.


Not the whole conversion, no. Just enough to confirm the suspicion that it  
affects compile time in Windows.


As other people in the thread have mentioned there are some tricky areas  
so before investing too much time in this, it seemed to be best to check  
with people with longer experience of the code if there was something  
we've missed.


Though the conversion is scripted so it can possibly be applied to large  
portions of the source code and then the tricky areas can be manually  
fixed.


/Daniel
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Ryosuke Niwa]

On Tue, Mar 26, 2013 at 4:35 PM, Daniel Bratell  wrote:

> Den 2013-03-26 21:29:32 skrev Benjamin Poulain :
>
>
>  On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>>
>>>
>>> If we have consensus that we should just switch to paths relative to
>>> Source (or maybe a couple different options), that would be (IMO) a big
>>> win. It sounds like Daniel & co. have already done the big bang
>>> conversion.
>>>
>>>
>> I think using full path would be a serious hit regarding hackability.
>>
>> I would rather spend some time tweaking my compiler to cache each
>> directory
>> content than waste time finding where is every single header I need to
>> include.
>>
>
> I guess you mean that it will be more job moving files around, but that is
> a rather rare operation, while reading an include directive and wondering
> what it's part of is rather common (both for compilers, tools and humans).
>
> I like the paths as a tool to indicate "module" dependencies. You can more
> easily see that a file depends on foo and bar (but not on fie) if you see:
>
> #include "foo/object.h"
> #include "foo/thing.h"
> #include "bar/stuff.h"
>
> That will tell you useful things, and avoid making layering violations by
> accident.
>

Sure, but we don't really have multiple modules/components inside WebCore.
I can definitely see being able to spot this across different projects in
Source (e.g. between WTF, JSC, WebCore, & WebKit) but we already do this so
I'm not sure what kind of layering violations this is going to help us
detect.  Do you have a specific use case in mind where this would have
helped us?

I can see that this would be helpful in separating WebCore and Platform
(currently misplaced in WebCore/platform/) but we don't have many
self-contained components like Platform in WebCore.

It's also worth noting that we can't easily use relative file paths to
specify headers across WTF, JSC, WebCore, WebKit, etc.. due to build system
constriants.

- R. Niwa
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Martin Robinson]

On Tue, Mar 26, 2013 at 1:31 PM, Ryosuke Niwa  wrote:
> Another idea.  What you copied all headers into one directory (in a pre
> compilation step) and used that as the single include directory?

I don't dare chime in about full paths versus short paths, but I'd
like to point out that many ports already have tooling in place to
emulate Mac style framework includes for WebKit2.

For instance, #include  works via a script that scans
source files and creates (hard, I believe) links in a single
directory. This script is surely easily adaptable to use on a
project-wide basis. So, if we decide that a shorter search path is a
win, it doesn't necessarily mean full paths.

--Martin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Benjamin Poulain]

On Tue, Mar 26, 2013 at 4:35 PM, Daniel Bratell  wrote:

> Den 2013-03-26 21:29:32 skrev Benjamin Poulain :
>
>> On Tue, Mar 26, 2013 at 1:20 PM, Dirk Pranke  wrote
>>
>>>
>>> If we have consensus that we should just switch to paths relative to
>>> Source (or maybe a couple different options), that would be (IMO) a big
>>> win. It sounds like Daniel & co. have already done the big bang
>>> conversion.
>>>
>>>
>> I think using full path would be a serious hit regarding hackability.
>>
>> I would rather spend some time tweaking my compiler to cache each
>> directory
>> content than waste time finding where is every single header I need to
>> include.
>>
>
> I guess you mean that it will be more job moving files around, but that is
> a rather rare operation, while reading an include directive and wondering
> what it's part of is rather common (both for compilers, tools and humans).
>

My personal issue with full path is I don't think of the code base in term
of files but in terms of classes.
I don't care where the files are, it is a detail.

The problem of moving file is also obvious.
There is already a huge cost associated with moving and renaming files (all
the build systems). It is to the point that people will prefer leaving
broken name rather than renaming headers. By having full path, you would
increase that cost further, making refactoring even harder.


> I like the paths as a tool to indicate "module" dependencies. You can more
> easily see that a file depends on foo and bar (but not on fie) if you see:


> #include "foo/object.h"
> #include "foo/thing.h"
> #include "bar/stuff.h"
>
> That will tell you useful things, and avoid making layering violations by
> accident.
>

I don't understand this argument.

We already have WTF, WebCore, WebKit and we use them as prefix when
including headers.
The last problem is platform. But it should be fixed by moving it outside
WebCore, not by changing everything else.

I think is already silly to have wtf/text as a weird exception.

But I realize it's a question of style and as such there is not a right and
> a wrong, unless there are other factors. And here we have the seemingly
> heavy compilation time cost for it which I think is a strong argument
> against delegating the task of finding the header files to the compiler.


Hackabilty is a project goal. Compile time is not.
If the change means people are afraid to move/rename files, it is a bad
idea.

If such a change comes with the appropriate tooling for moving/renaming
files, I am not opposed to it.

Benjamin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Compiling WebKit up to 25% faster in Windows?

[Nico Weber]

On Tue, Mar 26, 2013 at 4:51 PM, Daniel Bratell  wrote:

> Den 2013-03-26 21:20:10 skrev Dirk Pranke :
>
>
>  If we have consensus that we should just switch to paths relative to
>> Source
>> (or maybe a couple different options), that would be (IMO) a big win. It
>> sounds like Daniel & co. have already done the big bang conversion.
>>
>
> Not the whole conversion, no. Just enough to confirm the suspicion that it
> affects compile time in Windows.
>
> As other people in the thread have mentioned there are some tricky areas
> so before investing too much time in this, it seemed to be best to check
> with people with longer experience of the code if there was something we've
> missed.
>
> Though the conversion is scripted so it can possibly be applied to large
> portions of the source code and then the tricky areas can be manually fixed.


Can you share your scripts, so that we can measure how this changes things
on other platforms?

Thanks!
Nico


>
>
> /Daniel
>
> __**_
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/**mailman/listinfo/webkit-dev
>
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] Fwd: Compiling WebKit up to 25% faster in Windows?

[Brent Fulgham]

Hi,

On Mar 26, 2013, at 1:50 PM, Ryosuke Niwa  wrote:

>> Interesting. I have the exact opposite experience, having to paw around to 
>> figure out where "Font.h" actually lives rather than just seeing 
>> "WebCore/platform/graphics/Font.h".
> 
> On TextMate or Xcode, all I need need to is to type in Font.h and they'll 
> automatically find the right file for me.
> 
> I'll be annoyed if people started using same filenames in multiple 
> directories (this has already happened :( ) because then various tools will 
> start listing multiple files from all those directories.

This can be a problem for implementation files, too, at least under windows. 
Can I share the joy of tracking down linker errors due to multiple compilation 
units with the name "Foo.obj" :)

That said, I really do prefer the segmented include paths for the same reasons 
Dirk shared.

Thanks,

-Brent
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev