RE: pch support
Just FYI here are some tests I made in November: http://www.reactos.org/archives/public/ros-dev/2005-November/006273.html Casper -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Steven Edwards Sent: 13. januar 2006 17:01 To: Rolf Kalbermatter Cc: wine-devel@winehq.org; Marcus Meissner Subject: Re: pch support Hi, On 1/13/06, Rolf Kalbermatter <[EMAIL PROTECTED]> wrote: > This solution has been starting to form in my mind as well. It is even > portable to MSVC it seems with a few small modifications to winapi/msvcmaker. I came up with a much more generic solution that will work on all compilers and does not cause a dependancy issue. It requires some minor code adjustments and there is a slight issue if you have errors in linking. The error location is wrong so you need to have a way to disable this if you have a real error to track the problem down, but once I fixed ReactOS shell32.dll to use my hack my compile times on gcc4 went from 30+ secs down to 9sec. MSVC went from 22 to less than 1 secound. In short each module would have its own autogenerated master C file that would include all of the others like the one I have attached. Minus the #define hacks. I personally think its more of a nasty hack but a configure switch could be added like --enable-compilation-units Because each header is only processed once, it give the same effect as PCH. -- Steven Edwards - ReactOS and Wine developer "There is one thing stronger than all the armies in the world, and that is an idea whose time has come." - Victor Hugo
RE: pch support
Just FYI here are some tests I made in November: http://www.reactos.org/archives/public/ros-dev/2005-November/006273.html Casper -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Steven Edwards Sent: 13. januar 2006 17:01 To: Rolf Kalbermatter Cc: wine-devel@winehq.org; Marcus Meissner Subject: Re: pch support Hi, On 1/13/06, Rolf Kalbermatter <[EMAIL PROTECTED]> wrote: > This solution has been starting to form in my mind as well. It is even > portable to MSVC it seems with a few small modifications to winapi/msvcmaker. I came up with a much more generic solution that will work on all compilers and does not cause a dependancy issue. It requires some minor code adjustments and there is a slight issue if you have errors in linking. The error location is wrong so you need to have a way to disable this if you have a real error to track the problem down, but once I fixed ReactOS shell32.dll to use my hack my compile times on gcc4 went from 30+ secs down to 9sec. MSVC went from 22 to less than 1 secound. In short each module would have its own autogenerated master C file that would include all of the others like the one I have attached. Minus the #define hacks. I personally think its more of a nasty hack but a configure switch could be added like --enable-compilation-units Because each header is only processed once, it give the same effect as PCH. -- Steven Edwards - ReactOS and Wine developer "There is one thing stronger than all the armies in the world, and that is an idea whose time has come." - Victor Hugo
Re: pch support
Hi, On 1/13/06, Rolf Kalbermatter <[EMAIL PROTECTED]> wrote: > This solution has been starting to form in my mind as well. It is even > portable to MSVC it seems with a few small modifications to winapi/msvcmaker. I came up with a much more generic solution that will work on all compilers and does not cause a dependancy issue. It requires some minor code adjustments and there is a slight issue if you have errors in linking. The error location is wrong so you need to have a way to disable this if you have a real error to track the problem down, but once I fixed ReactOS shell32.dll to use my hack my compile times on gcc4 went from 30+ secs down to 9sec. MSVC went from 22 to less than 1 secound. In short each module would have its own autogenerated master C file that would include all of the others like the one I have attached. Minus the #define hacks. I personally think its more of a nasty hack but a configure switch could be added like --enable-compilation-units Because each header is only processed once, it give the same effect as PCH. -- Steven Edwards - ReactOS and Wine developer "There is one thing stronger than all the armies in the world, and that is an idea whose time has come." - Victor Hugo #include #define NONAMELESSUNION #define NONAMELESSSTRUCT #include "authors.c" #include "autocomplete.c" #include "brsfolder.c" #define critsect_debug critsect_debug_changenotify #include "changenotify.c" #undef critsect_debug #include "classes.c" #include "clipboard.c" #include "control.c" #define vt_PersistFolder2 vt_PersistFolder2_cpanelfolder #define vt_ShellFolder2 vt_ShellFolder2_cpanelfolder #define impl_from_IPersistFolder2 impl_from_IPersistFolder2_cpanelfolder #include "cpanelfolder.c" #undef impl_from_IPersistFolder2 #undef vt_ShellFolder2 #undef vt_PersistFolder2 #include "dataobject.c" #include "dde.c" #include "debughlp.c" #include "dialogs.c" #include "dragdrophelper.c" #include "enumidlist.c" #define pfvt pfvt_folders #define eivt eivt_folders #define impl_from_IPersistFile impl_from_IPersistFile_folders #include "folders.c" #undef impl_from_IPersistFile #undef eivt #undef pfvt #define critsect_debug critsect_debug_iconcache #include "iconcache.c" #undef critsect_debug #include "pidl.c" #include "regsvr.c" #include "ros-systray.c" #include "shell32_main.c" #define pfvt pfvt_shelllink #define eivt eivt_shelllink #define impl_from_IPersistFile impl_from_IPersistFile_shelllink #include "shelllink.c" #undef impl_from_IPersistFile #undef eivt #undef pfvt #include "shellole.c" #include "shellord.c" #include "shellpath.c" #include "shellreg.c" #include "shellstring.c" #include "shfldr_desktop.c" #define dtvt dtvt_shfldr_fs #define impl_from_IDropTarget impl_from_IDropTarget_shfldr_fs #define IGenericSFImpl IGenericSFImpl_fs #include "shfldr_fs.c" #undef impl_from_IDropTarget #undef IGenericSFImpl #undef dtvt #define vt_PersistFolder2 vt_PersistFolder2_shfldr_mycomp #define vt_ShellFolder2 vt_ShellFolder2_shfldr_mycomp #define IGenericSFImpl IGenericSFImpl_shfldr_mycomp #include "shfldr_mycomp.c" #undef IGenericSFImpl #undef vt_ShellFolder2 #undef vt_PersistFolder2 #include "shfldr_unixfs.c" #include "shlexec.c" #include "shlfileop.c" #include "shlfolder.c" #include "shlfsbind.c" #include "shlmenu.c" #define dtvt dtvt_shlview #define impl_from_IDropTarget impl_from_IDropTarget_shlview #include "shlview.c" #undef impl_from_IDropTarget #undef dtvt #include "shpolicy.c" #define cmvt cmvt_shv_bg_cmenu #include "shv_bg_cmenu.c" #undef cmvt #define cmvt cmvt_shv_item_cmenu #include "shv_item_cmenu.c" #undef cmvt
RE: pch support
> Hmm, perhaps use a CFLAGS "-include autogenerated.h" trick. > > Generate "autogenerated.h" by collecting all possible includes via > a script on "make depend", then force it into every file with > -include autogenerated.h (previously compiled to pch). > > Has the advantage that the magic is all in the buildsystem, no > source files need to be touched. This solution has been starting to form in my mind as well. It is even portable to MSVC it seems with a few small modifications to winapi/msvcmaker. For platforms not supporting PCH the build system would simply behave as is. Watch out though that you probably do not want to include every possible public header file, due to some conflicts. But for the majority of windows.h and friends this could be a good solution. Rolf Kalbermatter
Re: pch support
On Fri, Jan 13, 2006 at 10:47:43AM +0100, Rolf Kalbermatter wrote: > > > On Thu, Jan 12, 2006 at 09:02:42PM +0100, Rolf Kalbermatter wrote: > > > Thomas Weidenmueller wrote: > > > > > > > We've been using PCH with GCC for a long time in ReactOS, > > it's been > > > > working very well and reliable. Compiling ReactOS is *a lot* > > > > faster with > > > > PCH enabled. > > > > > > Basically what I get in MSVC 6 when modifing a header somewhere and > > > PCH is enabled is that MSVC keeps mocking about errors in the > > > headers, my change was supposed to fix. Disabling PCH (and > > > sometimes deleting the *.pch file) always fixes those issues. > > > > Unless, of course, your filesystem is samba-mounted from a linux box, > > when PCH tends to cause internal compiler errors. > > Well, I finally read a bit about how precompiled headers are supposed to work > and it seems MSVC does work in a similar way than gcc. > Basically you can only have one pch file per working unit. > > My problem was that when you create a new project in MSVC 6, PCH is by > default enabled and set to automatic but no specific header > file is filled in in the project settings. What this seems to do is taking > some (which?) header files and putting them in an > automatically generated pch file in your project output directory. Of course > for more than 1 source file in a project this is likely > each time different so build times will be even worse than without PCH. Also > it seems that header dependancy tracking isn't reliable > in such a setup. > > >From some explanations, I gathered that the only useful PCH setting in MSVC > >is as well to use one specific pch include file and even > more specifically configure one source file to create that pch file and all > others only to use it, otherwise MSVC tends to recompile > the pch file on some arbitrary conditions multiple times for different source > files, basically reverting the purpose of PCH more or > less or even make it worse. > > One thing to watch out also is that any statements in a source file before > the specifically set precompiled header file are > completely, and without any indication (well except the strange resulting > compile errors of course) ignored. Hmm, perhaps use a CFLAGS "-include autogenerated.h" trick. Generate "autogenerated.h" by collecting all possible includes via a script on "make depend", then force it into every file with -include autogenerated.h (previously compiled to pch). Has the advantage that the magic is all in the buildsystem, no source files need to be touched. Ciao, Marcus
RE: pch support
> On Thu, Jan 12, 2006 at 09:02:42PM +0100, Rolf Kalbermatter wrote: > > Thomas Weidenmueller wrote: > > > > > We've been using PCH with GCC for a long time in ReactOS, > it's been > > > working very well and reliable. Compiling ReactOS is *a lot* > > > faster with > > > PCH enabled. > > > > Basically what I get in MSVC 6 when modifing a header somewhere and > > PCH is enabled is that MSVC keeps mocking about errors in the > > headers, my change was supposed to fix. Disabling PCH (and > > sometimes deleting the *.pch file) always fixes those issues. > > Unless, of course, your filesystem is samba-mounted from a linux box, > when PCH tends to cause internal compiler errors. Well, I finally read a bit about how precompiled headers are supposed to work and it seems MSVC does work in a similar way than gcc. Basically you can only have one pch file per working unit. My problem was that when you create a new project in MSVC 6, PCH is by default enabled and set to automatic but no specific header file is filled in in the project settings. What this seems to do is taking some (which?) header files and putting them in an automatically generated pch file in your project output directory. Of course for more than 1 source file in a project this is likely each time different so build times will be even worse than without PCH. Also it seems that header dependancy tracking isn't reliable in such a setup. >From some explanations, I gathered that the only useful PCH setting in MSVC is >as well to use one specific pch include file and even more specifically configure one source file to create that pch file and all others only to use it, otherwise MSVC tends to recompile the pch file on some arbitrary conditions multiple times for different source files, basically reverting the purpose of PCH more or less or even make it worse. One thing to watch out also is that any statements in a source file before the specifically set precompiled header file are completely, and without any indication (well except the strange resulting compile errors of course) ignored. Rolf Kalbermatter
Re: pch support
On Thu, Jan 12, 2006 at 09:02:42PM +0100, Rolf Kalbermatter wrote: > Thomas Weidenmueller wrote: > > > We've been using PCH with GCC for a long time in ReactOS, it's been > > working very well and reliable. Compiling ReactOS is *a lot* > > faster with > > PCH enabled. > > Basically what I get in MSVC 6 when modifing a header somewhere and > PCH is enabled is that MSVC keeps mocking about errors in the > headers, my change was supposed to fix. Disabling PCH (and > sometimes deleting the *.pch file) always fixes those issues. Unless, of course, your filesystem is samba-mounted from a linux box, when PCH tends to cause internal compiler errors. David -- David Laight: [EMAIL PROTECTED]
RE: pch support
Thomas Weidenmueller wrote: > We've been using PCH with GCC for a long time in ReactOS, it's been > working very well and reliable. Compiling ReactOS is *a lot* > faster with > PCH enabled. Basically what I get in MSVC 6 when modifing a header somewhere and PCH is enabled is that MSVC keeps mocking about errors in the headers, my change was supposed to fix. Disabling PCH (and sometimes deleting the *.pch file) always fixes those issues. The issues about getting all the necessary headers in one single include file is, that sometimes certain headers conflict but one is used in one particular source file and another one in another source file. Of course you can always fix those issues by adding precompiler conditions around certain declarations or even complete include declarations, and that would be the prefered solution but sometimes this can't be easily done because you might break compilation on a different platform or with a different header set/compiler. I think PCH is fine as long as you do not need to modify anything about the source files and the preprocessor creating the pch file is smart enough to correctly track all possible dependancies. Apparently this seems not possible: MSVC seems to have trouble tracking dependancies reliable but does not require to put all used headers in a special include file, and gcc does require this modification but does not have trouble to track proper dependancies (or I have never understood how PCH is supposed to work under MSVC, but for me not having projects containing million lines of c code, disabling PCH always was the most simple and reliable solution). Rolf Kalbermatter
Re: pch support
Peter Åstrand wrote: > Disabling precompiled headers is the first thing I do when starting a > new project. We are using Visual C++ 2003/7.1, which often fails if you > are building from a network share (this is more or less unsupported > though, see > http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&oe=UTF-8&selm=umBJLvIYDHA.2632%40TK2MSFTNGP09.phx.gbl). We've been using PCH with GCC for a long time in ReactOS, it's been working very well and reliable. Compiling ReactOS is *a lot* faster with PCH enabled. I don't really understand what the problem is. Nothing really should change except that all includes are grouped into one header file. It works the same with MSVC. The only difference is that dependency tracking needs to use one minor hack so it works properly with MSVC. Even ancient versions of GCC don't have a problem with it and should compile everything properly without any additional hacks (without PCH support of course). - Thomas
Re: RFC: pch support
Hello Steven, > Alexandre did not like the needed changes required to add support for > this but I figured I would throw it out for debate anyway in case > anyone else can make him change his mind (duck). The issue is he does > not like needing a single header per-program or per dll. After doing > some research it seems to me that even MSVC does not support multiple > PCH files per project so I don't know how it could be fixed in gcc to > do it Microsoft has not figured it out. Why don't you create only one single precompiled header file for ? The speedup effect will not that big as in your example. But the code changes are minimal. The include is used nearly everywhere, and it contains a great bunch of declarations. So it can be used for most of all programs and DLLs. Perhaps this change is more easy to sell. Regards, martin
RE: pch support
On Thu, 12 Jan 2006, Rolf Kalbermatter wrote: I've had issues with header changes with MSVC precompiled headers too. Sometimes the only solution is to disable precompiled headers and delete the pch file. For fairness I have to say that I'm still using MSVC 6 so there is a good chance that more recent MSVC systems have better dependency tracking in precompiled headers. Disabling precompiled headers is the first thing I do when starting a new project. We are using Visual C++ 2003/7.1, which often fails if you are building from a network share (this is more or less unsupported though, see http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&oe=UTF-8&selm=umBJLvIYDHA.2632%40TK2MSFTNGP09.phx.gbl). -- Peter Åstrand ThinLinc Chief Developer Cendio www.cendio.se Teknikringen 3 583 30 LinköpingPhone: +46-13-21 46 00
RE: pch support
Steven Edwards wrote: > Alexandre did not like the needed changes required to add support for > this but I figured I would throw it out for debate anyway in case > anyone else can make him change his mind (duck). The issue is he does > not like needing a single header per-program or per dll. After doing > some research it seems to me that even MSVC does not support multiple > PCH files per project so I don't know how it could be fixed in gcc to > do it Microsoft has not figured it out. Well, having a single header file to include indeed looks not nice. This would grow and grow as more modules are added and eventually you would be adding the entire SDK headers to every source file indirectly. It is also not the way MSVC does do it. Instead the preparser or something seems to create a separate pch file from all the specifically included headers in that project and it seems smart enough to allow for different headers in different source files that can conflict if used together. I'm not sure how you would suppose to avoid possible conflicts of headers in this way. Maybe you consider maintaining one single precompile.h file per module but that's one more file to maintain and it still won't solve the issue of Conflicting header files used in different source files inside that module. The Wine/MS SDK/crt headers are already a nightmare to combine sometimes and your single header would only increase that issue. And Wine attempting to be compilable on many different systems besides standard Linux, has certainly a lot more possible difficulties with header conflicts than most other systems. > problem: > ccache works great for speeding up compiles but seems to still have > issue with some header changes. Also the ReactOS guys really want to > add pch support to the Wine dlls that are shared so the job falls to > me to sell it to Wine. I've had issues with header changes with MSVC precompiled headers too. Sometimes the only solution is to disable precompiled headers and delete the pch file. For fairness I have to say that I'm still using MSVC 6 so there is a good chance that more recent MSVC systems have better dependency tracking in precompiled headers. Rolf Kalbermatter
RFC: pch support
Hi,
Alexandre did not like the needed changes required to add support for
this but I figured I would throw it out for debate anyway in case
anyone else can make him change his mind (duck). The issue is he does
not like needing a single header per-program or per dll. After doing
some research it seems to me that even MSVC does not support multiple
PCH files per project so I don't know how it could be fixed in gcc to
do it Microsoft has not figured it out.
problem:
ccache works great for speeding up compiles but seems to still have
issue with some header changes. Also the ReactOS guys really want to
add pch support to the Wine dlls that are shared so the job falls to
me to sell it to Wine.
Solution:
Below are the benchmarks from my box and a patch. Simply patch your
tree and then ./configure and cd to the regedit folder and do a 'time
make','make clean','make pch', and 'time make' if you want to see the
speedup on your box. I expect we could get a 2x speedup in fresh
compiles of Wine with pch support
Regedit compile without pch
real0m5.632s
user0m5.000s
sys 0m0.524s
Regedit compile with pch
real0m2.797s
user0m2.352s
sys 0m0.428s
--
Steven Edwards - ReactOS and Wine developer
"There is one thing stronger than all the armies in the world, and
that is an idea whose time has come." - Victor Hugo
--- /dev/null 1994-07-17 19:46:18.0 -0400
+++ programs/regedit/precomp.h 2006-01-11 14:19:54.0 -0500
@@ -0,0 +1,21 @@
+#include
+#include
+#include
+#include
+#include
+
+#define NONAMELESSUNION
+#define NONAMELESSSTRUCT
+#define WIN32_LEAN_AND_MEAN /* Exclude rarely-used stuff from Windows headers */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "main.h"
+#include "regproc.h"
+
+#include "wine/debug.h"
Index: Make.rules.in
===
RCS file: /home/wine/wine/Make.rules.in,v
retrieving revision 1.193
diff -u -r1.193 Make.rules.in
--- Make.rules.in 6 Oct 2005 16:06:04 - 1.193
+++ Make.rules.in 12 Jan 2006 04:49:29 -
@@ -106,7 +106,7 @@
prog_manext = 1
api_manext = 3w
conf_manext = 5
-CLEAN_FILES = *.o *.a *.so *.ln *.$(LIBEXT) \\\#*\\\# *~ *% .\\\#* *.bak *.orig *.rej \
+CLEAN_FILES = *.gch *.o *.a *.so *.ln *.$(LIBEXT) \\\#*\\\# *~ *% .\\\#* *.bak *.orig *.rej \
*.flc *.tab.c *.tab.h @[EMAIL PROTECTED] core
OBJS = $(C_SRCS:.c=.o) $(EXTRA_OBJS)
Index: programs/Makeprog.rules.in
===
RCS file: /home/wine/wine/programs/Makeprog.rules.in,v
retrieving revision 1.41
diff -u -r1.41 Makeprog.rules.in
--- programs/Makeprog.rules.in 28 Sep 2005 18:34:00 - 1.41
+++ programs/Makeprog.rules.in 12 Jan 2006 04:49:35 -
@@ -22,6 +22,13 @@
all: $(MODULE)$(DLLEXT) $(BASEMODULE)$(EXEEXT)
+# Rule for generating a precompiled header
+
+$(H_SRCS).gch: Makefile.in
+ $(CC) -g $(H_SRCS) $(ALLCFLAGS) -o $@
+
+pch::$(H_SRCS).gch
+
# Rules for .so main module
$(MODULE).so: $(OBJS) $(RC_SRCS:.rc=.res) Makefile.in
Index: programs/regedit/Makefile.in
===
RCS file: /home/wine/wine/programs/regedit/Makefile.in,v
retrieving revision 1.22
diff -u -r1.22 Makefile.in
--- programs/regedit/Makefile.in 11 Aug 2005 17:12:18 - 1.22
+++ programs/regedit/Makefile.in 12 Jan 2006 04:49:36 -
@@ -10,6 +10,8 @@
EXTRADEFS = -DNO_LIBWINE_PORT
MODCFLAGS = @BUILTINFLAG@
+H_SRCS = precomp.h
+
C_SRCS = \
about.c \
childwnd.c \
Index: programs/regedit/about.c
===
RCS file: /home/wine/wine/programs/regedit/about.c,v
retrieving revision 1.4
diff -u -r1.4 about.c
--- programs/regedit/about.c 22 Aug 2005 09:17:37 - 1.4
+++ programs/regedit/about.c 12 Jan 2006 04:49:36 -
@@ -18,13 +18,7 @@
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
-#define WIN32_LEAN_AND_MEAN /* Exclude rarely-used stuff from Windows headers */
-#include
-#include
-#include
-#include
-
-#include "main.h"
+#include
void ShowAboutBox(HWND hWnd)
{
Index: programs/regedit/childwnd.c
===
RCS file: /home/wine/wine/programs/regedit/childwnd.c,v
retrieving revision 1.13
diff -u -r1.13 childwnd.c
--- programs/regedit/childwnd.c 5 Jan 2006 16:55:13 - 1.13
+++ programs/regedit/childwnd.c 12 Jan 2006 04:49:36 -
@@ -18,16 +18,7 @@
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
-#define WIN32_LEAN_AND_MEAN /* Exclude rarely-used stuff from Windows headers */
-#include
-#include
-#include
