Re: [naviserver-devel] Naviserver hangs on Windows
On 08.10.14 20:25, Gustaf Neumann wrote: I've revived an old installation of win7 based on VirtualBox and Visual Studio 11, used the version of naviserver from bitbucket, compiled tcl, switched the line for the win-makefile, compiled nsd using nmake, and to my big surprise, nsd,exe -c worked! did some more testing with nsd.exe under win7-64 (version from the main trunk). Starting nsd with the standard nsd-config file works nicely (i just had to replace .so by .dll for nscp, nssock, nslog, nscgi, and nsdb), the modules are loading fine, browsing through the documentation via the windows binary works nice. At least the basic functionality works without any problems. The only quirks is currently the installation, i had to create the install directories by hand and had to copy much of the content there (i hope for a better make install from Andrew's version). -g PS: Btw, i am using now the windows-native version of NS_GetTime (using 2x long) and mutex-timings turned on. Since Andrew uses this as well on 32bit-windows, we should be able to use this in the main naviserver branch as well. -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Dear Andrew, Sorry for the late reply, but have been away of my computer. 1. _USE_32BIT_TIME_T Indeed, you are absolutely write. My Windows distribution had since 2011 to 2013 both Windows 32 and Windows 64 binaries. I was using more or less the same make and configuration files to build it. The define _USE_32BIT_TIME_T was used for the Windows 32 build but not for the Windows 64. The current version of my distribution only contains Windows 64 binaries and in fact the define is commented out in the makefile. Sorry for the time I made you waste on this point. 2. About the configuration I am using the tools made available by Visual Studio 2013 Professional Edition in the VS2013 X64 Native Tools Command Prompt. That is: CL: Microsoft (R) C/C++ Optimizing Compiler Version 18.00.21005.1 for x64 LINK: Microsoft (R) Incremental Linker Version 12.00.21005.1 This is what I use and your questions below: COPTS = /EHsc /W3 /nologo /c DEFS= $(DEFS) /D _WINDOWS /D TCL_THREADS=1 /D WIN32 /D _WIN32 \ /D FD_SETSIZE=128 /D NO_CONST=1 /D _MBCS # /D _USE_32BIT_TIME_T LOPTS = /NOLOGO /SUBSYSTEM:CONSOLE /OPT:NOREF /OPT:NOICF Did you comment out your _USE_32BIT_TIME_T there? [MM] Absolutely, I already explained what I did in point 1 and once again apologies. What does /D _WIN32 do? [MM] This should be set automatically by the compiler, so in theory it is redundant... but one never knows... What C run-time library are you using? [MM] Visual C++ Redistributable Packages for Visual Studio 2013 (Release, No Debug) http://www.microsoft.com/en-us/download/details.aspx?id=40784 I do agree with your remarks in your mail, I had a look at the Visual Studio Project Files you use, the only relevant differences I can see are the following: 1. you are targeting as Machine_X64, I am targeting AMD64 2. you use Active TCL, I did compile from scratch also TCL, using the same compiler options (I believe this is very important as we have no control on how Active TCL is built) Hope this helps, Maurizio -Original Message- From: Andrew Piskorski [mailto:a...@piskorski.com] Sent: 07 October 2014 22:36 To: naviserver-devel@lists.sourceforge.net Subject: Re: [naviserver-devel] Naviserver hangs on Windows On Mon, Oct 06, 2014 at 05:18:08PM -0400, Andrew Piskorski wrote: On Sun, Oct 05, 2014 at 09:37:39PM +0200, Maurizio Martignano wrote: Did you use the define _USE_32BIT_TIME_T yes or not? I haven't tried using _USE_32BIT_TIME_T yet, but I think using it would be INCORRECT on Windows-64. If I do pass the /D _USE_32BIT_TIME_T flag, I get this: C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\INCLUDE\crtdefs.h(463) : fatal error C1189: #error : You cannot use 32-bit time_t (_USE_32BIT_TIME_T) with _WIN64 So that seems pretty clear, I must NOT set that flag. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Andrew, these are good news!
the advantage of approach 2 (disable mutex timings, use ) is that this
works for 32bit and 64 bit, while (1)
works most likely only for 64bit (if i look at the constant). The
mutex timings is an additional feature in naviserver, so at least
for the time being, one can life without that and with the
deeper knowledge, we might be able to activate this later again.
The problem seems to be: DllMain() calls certain functions
before the normal entry point main() is called, which initializes
tcl etc. The Windows man page [1] says:
* Warning* There are serious limits on what you can do in a DLL entry
point.
... not without a reason. It seems, that nn the windows side, one
has to be very careful, what is called inside these routines, since
all calls to Tcl* are potentially unsafe.
changing the Ns_Time.sec (in nsthread.h) to be time_t rather than long
fixes nearly everything.
The reason, naviserver uses 2x long in Ns_Time is probably due to
Tcl, using:
typedef struct Tcl_Time {
long sec; /* Seconds. */
long usec; /* Microseconds. */
} Tcl_Time;
So at least, when using Tcl_GetTime(),
which returns the time as Tcl_Time, everything should be
fine. Note, that in your native time implementation, you
assign to timePtr-sec with (time_t), which might be
part of the problem you are experiencing.
I have to look at this closer before i have a better seeing
of consequences of changing the types in Ns_Time, i would
certainly prefer to stick to the Tcl conventions, since we depend
strongly on tcl behavior all over the place.
Probably, i should get a windows installation running, but
the setup costs are quite high in terms of time and lack
on windows-knowlege on my side.
All together, things look already much better.
-g
[1]
http://msdn.microsoft.com/en-us/library/windows/desktop/ms682583%28v=vs.85%29.aspx
Am 08.10.14 07:14, schrieb Andrew Piskorski:
On Tue, Oct 07, 2014 at 12:09:41AM +0200, Gustaf Neumann wrote:
Please something more to try: to provide some potential evidence for
my too early for tcl calls hypothesis, i've deactivated for the
time being the mutex time monitoring for windows, since the earliest
calls are from mutex calls. Can you check when possible, whether
Tcl_GetTime() can be used now inside Ns_GetTime()?
Gustaf, I think you were on to something there.
In my fork I turned the Windows mutex timings back on, because they
work now. However, they ONLY work correctly when Ns_GetTime() is
using the old-style AOLserver code with EPOCH_BIAS, etc.
If I change Ns_GetTime() back to using the Ns_GetTimeFromTcl()
platform-independent code, then the server immediately hangs on
startup in TclpGetDate(), just like in my original email on 10/3.
To summarize, there seem to be two combinations that work ok (so far):
1. Ns_GetTime() uses EPOCH_BIAS calculation, normal mutex timings in
nsthread/mutex.c turned on.
2. Ns_GetTime() uses Ns_GetTimeFromTcl(), mutex timings in
nsthread/mutex.c DISABLED.
With either of those combinations, Naviserver will start up and then
shut down, which is really all I've tested.
--
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Wed, Oct 08, 2014 at 08:05:42AM +0200, Maurizio Martignano wrote: I do agree with your remarks in your mail, I had a look at the Visual Studio Project Files you use, the only relevant differences I can see are the I do not use any Visual Studio Project Files for Naviserver. The ones in Mercurial are old and I have no idea if they work or not. The Naviserver Windows build I've been working on is controlled by the makefiles, a combination of the two Windows-only Makefile.win32 for nmake, and the individual module-specific makefiles which work in both Gnu make and Microsoft nmake. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Wed, Oct 08, 2014 at 06:50:35PM +0200, Maurizio Martignano wrote: Dear Andrew and Gustaf, In due time, when you both are confident on the status of Naviserver on Windows, I would try it inside my Windows-OpenACS distribution. That sounds excellent to me. My own use of Naviserver on Windows will be in a rather narrow application (not OpenACS; we use Linux for that), so it would be great to see it tested on a wider range of workloads. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
yes, it might be a good time. as wrote in the other thread, i was able to compile an start nsd with very little windows experience. Be aware, that runnning nsd.exe -c does not mean that all of nsd is really running. -g PS: my own time budget will be reduced significantly for the next weeks. Am 08.10.14 18:50, schrieb Maurizio Martignano: Dear Andrew and Gustaf, In due time, when you both are confident on the status of Naviserver on Windows, I would try it inside my Windows-OpenACS distribution. First of all I would try it in few installations running on the machines I have in my offices, then once I see it they are running fine and everything I have now still behave the same, I would make the new version of the distribution available. Is that OK with you? Maurizio -Original Message- From: Andrew Piskorski [mailto:a...@piskorski.com] Sent: 08 October 2014 18:39 To: naviserver-devel@lists.sourceforge.net Subject: Re: [naviserver-devel] Naviserver hangs on Windows On Wed, Oct 08, 2014 at 09:32:56AM +0200, Gustaf Neumann wrote: the advantage of approach 2 (disable mutex timings, use ) is that this works for 32bit and 64 bit, while (1) works most likely only for 64bit (if i look at the constant). No, I'm confident that the approach using EPOCH_BIAS works just fine on 32-bit Windows, because in AOLserver 4.0.7 and 4.0.8 that is the ONLY implementation of Ns_GetTime on Windows, and I ran those versions of AOLserver for years on Windows XP 32-bit. Calls to Ns_GetTime() are very common, so AFAICT, for those AOLservers to work at all, that EPOCH_BIAS code had to be working fine on 32-bit Windows XP. The only other alternative is that AOLserver was somehow magically calling an entirely different function instead of Ns_GetTime(), which I judge as very unlikely. The mutex timings is an additional feature in naviserver, so at least for the time being, one can life without that and with the deeper knowledge, we might be able to activate this later again. Ah, I didn't realize that. I should turn them back off for Windows in my fork then? The problem seems to be: DllMain() calls certain functions before the normal entry point main() is called, which initializes tcl etc. The Windows man page [1] says: Interesting. And of course DllMain() only exists on Windows. But, why do we need DllMain() at all? I assume we DO need it of course, I just don't understand the design differences between Naviserver's pthread and winthread support. Both Posix and Windows use Nsthreads_LibInit(), but DllMain() seems to be an extra layer not present for Posix. changing the Ns_Time.sec (in nsthread.h) to be time_t rather than long fixes nearly everything. The reason, naviserver uses 2x long in Ns_Time is probably due to Tcl, using: AFAICT, using long is just wrong, and Tcl and Naviserver both should have been using time_t for many years now. I'm not sure when time_t was invented and standardized, but since then it's been what Unix and Windows both use. So what am I missing? Why did the Tcl maintainers stick with long for Tcl_Time.sec? It could just be a holdover from before time_t existed or really mattered, but perhaps they had a better for keeping it as a long. So at least, when using Tcl_GetTime(), which returns the time as Tcl_Time, everything should be fine. Note, that in your native time implementation, you assign to timePtr-sec with (time_t), which might be part of the problem you are experiencing. Could be. I will try the no-mutex-timings and Ns_GetTimeFromTcl() approach again, this time with Ns_Time.sec changed back to type long rather than time_t. Probably, i should get a windows installation running, but the setup costs are quite high in terms of time and lack on windows-knowlege on my side. Yeah. Maybe the Docker stuff Stephen showed us would make it easier? Definitely see the notes I sent a while back on exactly what Microsoft software to install. All the main stuff you need is available for free but figuring out WHAT you're supposed to install is not easy. Btw, if you do go that route, be aware that I am using nmake -f Makefile.win32 for building and cleaning, but NOT for installing Naviserver. I have not debugged the install commands of those two *.win32 makefiles at all, because I have my own old Tcl script (win32-install-nsd.tcl) for intalling on Windows, which I was able to quickly adapt to Naviserver. If you want that install script let me know. I haven't added it to Mercurial yet because it is in CVS, and I'd like to nicely retain its full history it, but haven't looked up how to best do that yet. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0
Re: [naviserver-devel] Naviserver hangs on Windows
Stephen, the Docker image is cool stuff! Many thanks! This eases mingw builds significantly. It compiles now fine the first bunch of files in nsthread fine, but stops then with x86_64-w64-mingw32-gcc -shared -L../nsthread -L../nsd -L../nsdb -o libnsthread.dll error.o master.o memory.o mutex.o cslock.o rwlock.o reentrant.o sema.o thread.o tls.o time.o pthread.o fork.o signal.o winthread.o -L/usr/local/tcl8.6.2-mingw/lib -ltcl86 -lnetapi32 -lkernel32 -luser32 -ladvapi32 -lws2_32 -L/usr/local/tcl8.6.2-mingw/lib /usr/lib64/gcc/x86_64-w64-mingw32/4.8.3/../../../../x86_64-w64-mingw32/bin/ld: cannot find -ltcl86 collect2: error: ld returned 1 exit status make[1]: *** [libnsthread.dll] Error 1 Looks like the installation of tcl is missing... -g On 07.10.14 00:33, Stephen wrote: Here are some build errors while cross-compiling with mingw using this docker image: https://bitbucket.org/groks/build-naviserver-mingw/src/tip/naviserver/ -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Fri, Oct 03, 2014 at 04:16:41PM +0200, Maurizio Martignano wrote: Dear Andrew, These are the basic options I would use for a Windows 64 bit, AMD64: COPTS = /EHsc /W3 /nologo /c DEFS = $(DEFS) /D _WINDOWS /D TCL_THREADS=1 /D WIN32 /D _WIN32 \ /D FD_SETSIZE=128 /D NO_CONST=1 /D _MBCS # /D _USE_32BIT_TIME_T LOPTS = /NOLOGO /SUBSYSTEM:CONSOLE /OPT:NOREF /OPT:NOICF Did you comment out your _USE_32BIT_TIME_T there? What does /D _WIN32 do? What C run-time library are you using? I've been able to find Microsoft docs on the various compiler and linker flags, but NOT anything clear on what all the magic /D preprocessor definitions actually do. The nmake files I got from Ibrahim (at Archiware) use the /MTd compiler flag, which should give me the debug version of the libcmtd.lib CRT, as explained here: http://msdn.microsoft.com/en-us/library/abx4dbyh%28v=vs.100%29.aspx http://msdn.microsoft.com/en-us/library/2kzt1wy3%28v=vs.100%29.aspx The other main option is /MDd for msvcrtd.lib, but I don't know which is the default when you pass no flags at all. Most of our other build options are the same. However, besides the ones I mentioned above, I am also using these extra flags that you do not have: In COPTS: /Zi /RTC1 In DEFS: /D _CRT_SECURE_NO_WARNINGS /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _DEBUG In LOPTS: /DEBUG AFAICT those all just enable additional debugging and/or run-time checks, and so should be pretty safe. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Mon, Oct 06, 2014 at 05:18:08PM -0400, Andrew Piskorski wrote: On Sun, Oct 05, 2014 at 09:37:39PM +0200, Maurizio Martignano wrote: Did you use the define _USE_32BIT_TIME_T yes or not? I haven't tried using _USE_32BIT_TIME_T yet, but I think using it would be INCORRECT on Windows-64. If I do pass the /D _USE_32BIT_TIME_T flag, I get this: C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\INCLUDE\crtdefs.h(463) : fatal error C1189: #error : You cannot use 32-bit time_t (_USE_32BIT_TIME_T) with _WIN64 So that seems pretty clear, I must NOT set that flag. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Tue, Oct 07, 2014 at 04:09:02PM -0400, Andrew Piskorski wrote:
C:\ C:\web\nsd-atp\bin\nsd -c
[384.e14][-main-] Notice: nsmain: NaviServer/4.99 starting
[384.e14][-main-] Fatal: nsthreads: localtime_s failed in ns_localtime: win32
err: 22
In ns_localtime(), I changed this call:
errNum = localtime_s(tlsPtr-ltbuf, clock);
to instead do this:
struct tm foo;
errNum = localtime_s(foo, clock);
And yet it still fails in exactly the same way, with EINVAL = 22.
This makes me think there is nothing wrong with the tlsPtr code or
data, rather something is seriously broken in my build. (But I have
no idea what.)
Possibly of interest, is that _localtime64_s() fails in exactly the
same way. _localtime32_s() apparently does not trigger EINVAL, but
instead fails on the same debug assertion, where the tm_mday value is
out of range (and most likely -1 just like before, although I did not
confirm that).
I increased my compiler warning level from /W1 to /W3, and now I see
LOTS of type warnings, e.g.:
log.c(689) : warning C4133: 'function' : incompatible types - from 'long *'
to 'const time_t *'
nssock.c(377) : warning C4133: 'function' : incompatible types - from 'int *'
to 'const char *'
binder.c(324) : warning C4244: 'function' : conversion from 'SOCKET' to
'int', possible loss of data
driver.c(3888) : warning C4244: '=' : conversion from 'SOCKET' to 'int',
possible loss of data
driver.c(2356) : warning C4244: 'function' : conversion from 'ssize_t' to
'unsigned int', possible loss of data
driver.c(3475) : warning C4244: '=' : conversion from 'ssize_t' to 'off_t',
possible loss of data
driver.c(3351) : warning C4244: 'function' : conversion from 'Tcl_WideInt' to
'long', possible loss of data
driver.c(2346) : warning C4267: 'function' : conversion from 'size_t' to
'int', possible loss of data
driver.c(3354) : warning C4267: 'function' : conversion from 'size_t' to
'unsigned int', possible loss of data
tclhttp.c(371) : warning C4244: '=' : conversion from '__int64' to 'int',
possible loss of data
That first one in log.c looks mildly suspicious. But, on a 64-bit
machine like this long and time_t are both 64-bit integers, so I
don't see the problem. Why are they incompatible?
http://msdn.microsoft.com/en-us/library/323b6b3k%28v=vs.100%29.aspx
However, in include/nsthread.h, why is Ns_Time defined like this?:
typedef struct Ns_Time {
longsec;
longusec;
} Ns_Time;
Shouldn't the type of sec be time_t rather than long?
--
Andrew Piskorski a...@piskorski.com
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Am 06.10.14 06:46, schrieb Jeff Rogers: This struck me as an interesting optimization question, so I wrote a quick program to test it (attached). This is not an interesting optimization question, since Tcl itself uses gettimeofday() (plus jumping around which might have a bad cache and locality influence, and some more copying). One cannot expect a big difference. For better reliability, i've increased the count. Below are 4 runs on openacs.org. The native call is from 8% to 20% faster (the latter one most probably due to external influences). The machine is a bare metal. Numbers will vary depending on the OS. The faster the system function gettimeofday() is, the bigger is the percentage difference. -g gustafn@openacs:~$ ./tt count: 1 Tcl_GetTime:3858686 usec 0.04 per Ns_GetTime:3811593 usec 0.04 per gettimeofday:3561367 usec 0.04 per gustafn@openacs:~$ ./tt count: 1 Tcl_GetTime:3858657 usec 0.04 per Ns_GetTime:3824398 usec 0.04 per gettimeofday:3563398 usec 0.04 per gustafn@openacs:~$ ./tt count: 1 Tcl_GetTime:4351765 usec 0.04 per Ns_GetTime:4024717 usec 0.04 per gettimeofday:3594736 usec 0.04 per gustafn@openacs:~$ ./tt count: 1 Tcl_GetTime:3864511 usec 0.04 per Ns_GetTime:3831866 usec 0.04 per gettimeofday:3541932 usec 0.04 per -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Gustaf Neumann wrote: Am 06.10.14 06:46, schrieb Jeff Rogers: This struck me as an interesting optimization question, so I wrote a quick program to test it (attached). This is not an interesting optimization question, since Tcl itself uses gettimeofday() (plus jumping around which might have a bad cache and locality influence, and some more copying). I think my underlying point was missed in the measurements. If there is not significant performance advantage to the platform-specific optimized version of Ns_GetTime over the platform-independent Tcl_GetTime, is it worth it to keep the optimized version? The penalty to be paid is in problems like the current one, where the windows build is for some reason having problems. On windows the function is delegating to Tcl_GetTime, but the system-specific bits confuse the issue. Your measurements are consistent and show the relative results I expected. Such is the benefit of real hardware :) They show a small advantage to Ns_GetTime over Tcl_GetTime, about 1.2%. This *is* a very frequently called function, but it is also pretty fast in any case. That's 1.2% of perhaps 5% (?? totally random guess) of overall request processing time. To me, that adds up to not a lot. A different takeaway from this benchmark is that if you are trying to wring every last bit of performance out of this call, it might be advantageous to define it as a macro (or use gcc/c99 inline declarations so it can be inlined and avoid the function call overhead and get to within a few instructions of the raw gettimeofday call (presumably you'll still need to copy the elements to the Ns_Time struct). -J One cannot expect a big difference. For better reliability, i've increased the count. Below are 4 runs on openacs.org. The native call is from 8% to 20% faster (the latter one most probably due to external influences). The machine is a bare metal. Numbers will vary depending on the OS. The faster the system function gettimeofday() is, the bigger is the percentage difference. -g gustafn@openacs:~$ ./tt count: 1 Tcl_GetTime:3858686 usec 0.04 per Ns_GetTime:3811593 usec 0.04 per gettimeofday:3561367 usec 0.04 per gustafn@openacs:~$ ./tt count: 1 Tcl_GetTime:3858657 usec 0.04 per Ns_GetTime:3824398 usec 0.04 per gettimeofday:3563398 usec 0.04 per gustafn@openacs:~$ ./tt count: 1 Tcl_GetTime:4351765 usec 0.04 per Ns_GetTime:4024717 usec 0.04 per gettimeofday:3594736 usec 0.04 per gustafn@openacs:~$ ./tt count: 1 Tcl_GetTime:3864511 usec 0.04 per Ns_GetTime:3831866 usec 0.04 per gettimeofday:3541932 usec 0.04 per -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Sun, Oct 05, 2014 at 09:31:31PM +0200, Gustaf Neumann wrote: On busy systems, Ns_GetTime() is one of the most frequent calls, used e.g. for mutex timings, so this makes a difference. Yes, so it's better for performance reasons, like I said. You may have written that small paragraph of code from scratch, but it is still true that inside the ifdef you added, the code calling gettimeofday() is essentially identical to what AOLserver used for many years. This is all fine, I see no problems with that code. What i did was to extend configure to look, if there is gettimeofday() Actually, and strangely, my Naviserver on Windows appeared to NEVER call gettimeofday() there, regardless of whether HAVE_GETTIMEOFDAY was true or false! I tried it both ways. I can't explain that; perhaps I was merely confused. But what is much more important to me, is that running the final portable Tcl-dependent code at the end definitely triggered a serious bug, making Naviserver unusable. b) When Tcl_GetTime() is not working on your system than you have a broken Tcl installation. Does this happen with your self-compiled tcl version? I do not know whether Tcl_GetTime() is working correctly in ActiveTcl 8.5 on Windows 7 64-bit or not. I will try something to verify that. Right now all I can say for sure is that funny indirect way that Naviserver calls Tcl_GetTime() in Ns_GetTime() fails horribly on my Windows build. So far I've never compiled Tcl on Windows 7 myself. I may try that soon to help with the debugging, but so far I've been exclusing using ActiveTcl 8.5. I strongly suspect that ActiveTcl is compiled correctly, but perhaps I am somehow linking against it incorrectly, or using incompatible build options, or something like that. http://msdn.microsoft.com/en-us/library/a442x3ye.aspx The *tp and *clock values look like a correct time_t count of seconds since the Unix epoch. So perhaps something is wrong with the only other argument ther to localtime_s(): tlsPtr-ltbuf The page you are citing contains the error conditions, when EINVAL is returned. The first argument can't be NULL, so clock must be invalid according to this information. No, the clock value is not invalid, I already checked that and it is a correct count of seconds since the Unix epoch. The problem must lie either in the other argument, or in something entirely separate and out of band (e.g., build options). on this issue. Have you tried the suggestion of Maurizio concerning USE_32BIT_TIME_T? I have not tried using it yet, but maybe I should. Previously, I wasn't sure if Maurizio was recommending it or warning against it. (I see now that he recommends using it.) My previous experience with AOLserver 4.0.x on Windows was with 32-bit Windows, which would have natively used 32-bit time_t. So perhaps forcing my 64-bit build to use 32-bit time_t would be useful. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Mon, Oct 06, 2014 at 10:16:31AM -0700, Jeff Rogers wrote: If there is not significant performance advantage to the platform-specific optimized version of Ns_GetTime over the platform-independent Tcl_GetTime, is it worth it to keep the optimized version? It is very much worth it to me, at least for now, because: One, I don't understand how the invocation of the platform-independent Tcl_GetTime even works. There is never any explicit call to Tcl_GetTime nor any other Tcl function, and Tcl_GetTime does not appear on the call stack. The so-callled optimized code is easier to understand. Two, the current platform-independent approach locks up in my Windows build, while the old Windows-specific code that AOLserver used for many years does not. I do not know why this is, but I certainly will use the old Windows-specific code until I figure it out. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Sun, Oct 05, 2014 at 09:37:39PM +0200, Maurizio Martignano wrote:
Did you use the define _USE_32BIT_TIME_T yes or not?
I believe you should use it.
I haven't tried using _USE_32BIT_TIME_T yet, but I think using it
would be INCORRECT on Windows-64.
Tcl 8.5.16 does this in win/tclWinPort.h:
#ifndef _WIN64
/* See [Bug 3354324]: file mtime sets wrong time */
# define _USE_32BIT_TIME_T
#endif
So it does NOT set _USE_32BIT_TIME_T when _WIN64 is defined.
I assume _WIN64 is always defined for 64-bit Windows.
Relevant discussion appears to be here:
http://sourceforge.net/p/tcl/bugs/4845/
http://sourceforge.net/p/tcl/bugs/5115/
http://sourceforge.net/p/mingw/bugs/1973/
It looks to me like the horrible confusion about what should be done
is only for ** 32-bit ** Windows systems, especially when trying to
support multiple old and new versions of 32-bit Windows.
In contrast, 64-bit Windows seems straightforward. And I am on 64-bit
Windows now! So I think I should simply NEVER set _USE_32BIT_TIME_T.
Does that sound right?
What does Tcl actually do, in practice, on 64-bit Linux and Window?
Here's a simple test, which shows that Tcl 8.5.x does indeed use a
64-bit time_t in both those cases, and that the integer values are
exactly the same on Windows and Linux, as they should be. That
corresponds to my understanding of the code above, that Tcl NEVER sets
_USE_32BIT_TIME_T on 64-bit Windows.
# On Windows 7 64-bit, running ActiveTcl 8.5:
% info patchlevel
8.5.15
% info nameofexecutable
C:/P/Tcl85/bin/tclsh85.exe
% info sharedlibextension
.dll
% set in_l [list {1970-01-01 00:00:00 GMT} {2038-12-01 00:00:00 GMT}
{-12-01 00:00:00 GMT}]
{1970-01-01 00:00:00 GMT} {2038-12-01 00:00:00 GMT} {-12-01 00:00:00 GMT}
% foreach ss $in_l { set tt [clock scan $ss] ; puts $tt - [clock format $tt
-gmt 1] }
0 - Thu Jan 01 00:00:00 GMT 1970
2174774400 - Wed Dec 01 00:00:00 GMT 2038
253399622400 - Wed Dec 01 00:00:00 GMT
% exit
# On Linux 64-bit, Ubuntu 12.04.3 LTS:
$ /usr/bin/tclsh8.5
% info patchlevel
8.5.11
% info nameofexecutable
/usr/bin/tclsh8.5
% info sharedlibextension
.so
% set in_l [list {1970-01-01 00:00:00 GMT} {2038-12-01 00:00:00 GMT}
{-12-01 00:00:00 GMT}]
{1970-01-01 00:00:00 GMT} {2038-12-01 00:00:00 GMT} {-12-01 00:00:00 GMT}
% foreach ss $in_l { set tt [clock scan $ss] ; puts $tt - [clock format $tt
-gmt 1] }
0 - Thu Jan 01 00:00:00 GMT 1970
2174774400 - Wed Dec 01 00:00:00 GMT 2038
253399622400 - Wed Dec 01 00:00:00 GMT
% exit
--
Andrew Piskorski a...@piskorski.com
--
Slashdot TV. Videos for Nerds. Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On 06.10.14 20:15, Andrew Piskorski wrote:. Actually, and strangely, my Naviserver on Windows appeared to NEVER call gettimeofday() there, regardless of whether HAVE_GETTIMEOFDAY was true or false! I tried it both ways. I can't explain that; perhaps I was merely confused. Actually, the acronym coming out of configure is #ifdef HAVE_GETTIMEOFDAY ... #endif Note that the expression is true, no matter whether HAVE_GETTIMEOFDAY is set to 1 or 0 ... was that maybe the problem? But what is much more important to me, is that running the final portable Tcl-dependent code at the end definitely triggered a serious bug, making Naviserver unusable. The word unusable is a understatement. The problem here is, when the tcl library does not work, most likely other calls to the tcl library will probably fail as well. Maybe the problem is that functions of the tcl library are called to early, before it is initialized. I've merged in your changes to the main branch. In order to keep your change history, i merged the changes rather than copying the files, but had at the end the problem to get rid the branch clean, since mercurial likes to keep branches around. when for the clock value for localtime_s() is valid, then the only problem might be the storage area, which is coming from the thread-local storage. Maybe localtime_s is not happy with the provided address. Switching the localtime() might work, but is only a fix for the symptoms. -g -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Mon, Oct 06, 2014 at 11:31:27PM +0200, Gustaf Neumann wrote: I've merged in your changes to the main branch. In order to keep your change history, i merged the changes rather than copying the files, but had at the end the problem to get rid the branch clean, since mercurial likes to keep branches around. Is my clean branch helpful to you, or should I get rid of it? I created that clean branch with that idea that you'd prefer to merge from it rather than from my default branch. But I'm new to Mercurial and not entirely sure how useful that really is. What I wanted is to have my clean branch be almost exactly the same as my default, but with any Andy-specific hacks removed. So I branched clean, and then removed my rpath hack. (As we've previously discussed, I need that rpath hack to work on Linux, but it's clearly the wrong fix. I added it only because I don't understand where the true bug is coming from.) In Mercurial, it doesn't seem that easy to keep two branches SLIGHTLY different from each other. Apparently the recommended way to cherry pick changes is with hg graft, but I did not try that, as it didn't seem like what I really wanted in this case. I'll see how it goes as I do more merges across my two branches, assuming I keep the clean branch around. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On 06.10.14 23:54, Andrew Piskorski wrote: Is my clean branch helpful to you, or should I get rid of it? on the clone used for the pull request, it is easier, if no new branches are introduced (at least for me). Please something more to try: to provide some potential evidence for my too early for tcl calls hypothesis, i've deactivated for the time being the mutex time monitoring for windows, since the earliest calls are from mutex calls. Can you check when possible, whether Tcl_GetTime() can be used now inside Ns_GetTime()? -g -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
On Mon, Oct 06, 2014 at 11:33:25PM +0100, Stephen wrote: Here are some build errors while cross-compiling with mingw using this docker image: https://bitbucket.org/groks/build-naviserver-mingw/src/tip/naviserver/ The docker image sounds useful. However, building with Mingw on Windows would use the Unix makefiles. I've never tested that build combination and doubt that anyone else has done so recently either. Right now I'm focused on just getting Windows builds to work with nmake and MSVC. Mingw can come later, especially since it (at least currently) and MSVC are tied to different build systems. It would be nice to eventually adopt a single build system that would work well with all combinations of platforms and compilers, but discussing that seems somewhat premature. Btw, the pages below on Tcl-based replacements for make and autoconf are interesting, especially in that it should be feasible to have **Tcl itself** self-host on such tools, while avoiding the chicken-and-egg problem: http://wiki.tcl.tk/27561 http://wiki.tcl.tk/27197 http://wiki.tcl.tk/12593 But I'd want get the opinion of the Tcl core team and other experts before going very far down such a path, either for Naviserver or in my own code. -- Andrew Piskorski a...@piskorski.com -- Slashdot TV. Videos for Nerds. Stuff that Matters. http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Am 03.10.14 16:49, schrieb Andrew Piskorski: I found two relevant commits in Mercurial, from 2009 and 2012, below. From those logs, it looks like what happened is that in 2009, Zoran removed the platform specific C code and instead made Naviserver call Tcl_GetTime (somehow). Then in 2012, Gustaf added back the old Unix-specific gettimeofday() implementation for performance reasons. no, i did NOT add back the old Unix-specific gettimeofday() implementation. What i did was to extend configure to look, if there is gettimeofday() available on the system. If it is, it bypasses the call to Tcl_GetTime() to get the timestamp via system call. On busy systems, Ns_GetTime() is one of the most frequent calls, used e.g. for mutex timings, so this makes a difference. The advantage of using Tcl_GetTime() is to delegate system dependencies to Tcl. What does this tell us for your situation: a) you have most probably not HAVE_GETTIMEOFDAY defined in your installation, otherwise you have an unresolved external. So, you were not at all affected by this change of mine! b) When Tcl_GetTime() is not working on your system than you have a broken Tcl installation. Does this happen with your self-compiled tcl version? In some later mail, you wrote: ./include/nsconfig.h:276:#define SIZEOF_TIME_T 8 That nsconfig.h was created on Linux, but my Windows build uses it too. Perhaps that is the problem? But how the heck can I generate a correct version of nsconfig.h for my Windows 7 system? I have not checked the details of your changes, but as a background: If one compiles naviserver with -DHAVE_CONFIG_H (the normal case under unix-like systems), then ./include/nsconfig.h is used. For MSVC is a set minimal set of hard-coded replacement defines included, which might or might not be incomplete or requires win7 or 64bit changes. If you are including ./include/nsconfig.h (generated under linux) under windows, then you are doomed. So in LogTime(), I added some simple printf's just before calling strftime(). ptm-tm_mday and all the other struct members have a value of -1. That's because in ns_localtime(), localtime_s() returns an EINVAL = 22 Invalid argument error code, and then sets all the members to -1. http://msdn.microsoft.com/en-us/library/a442x3ye.aspx The *tp and *clock values look like a correct time_t count of seconds since the Unix epoch. So perhaps something is wrong with the only other argument ther to localtime_s(): tlsPtr-ltbuf The page you are citing contains the error conditions, when EINVAL is returned. The first argument can't be NULL, so clock must be invalid according to this information. you might check for testing localtime() as a replacement, but read as well in the manpage aboutUSE_32BIT_TIME_T http://msdn.microsoft.com/en-us/library/bf12f0hc.aspx I am not familiar with the TIME_T mess under windows, check as well the endless discussions on http://sourceforge.net/p/mingw/bugs/1973/?page=0 on this issue. Have you tried the suggestion of Maurizio concerning USE_32BIT_TIME_T? -g -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
I believe you should use it. Thank you, Maurizio -Original Message- From: Andrew Piskorski [mailto:a...@piskorski.com] Sent: 05 October 2014 19:47 To: naviserver-devel@lists.sourceforge.net Cc: Maurizio Martignano Subject: Re: [naviserver-devel] Naviserver hangs on Windows On Sat, Oct 04, 2014 at 04:07:27PM +0200, Maurizio Martignano wrote: What is your target? Windows 32 or Windows 64? Windows 7, 64-bit. Did you use the define _USE_32BIT_TIME_T yes or not? No I did not. -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Gustaf Neumann wrote:
What i did was to extend configure to look, if there is gettimeofday()
available
on the system. If it is, it bypasses the call to Tcl_GetTime() to get
the timestamp
via system call. On busy systems, Ns_GetTime() is one of the most
frequent calls, used e.g. for mutex timings, so this makes a difference.
The advantage of using Tcl_GetTime() is to delegate system dependencies
to Tcl.
This struck me as an interesting optimization question, so I wrote a
quick program to test it (attached).
The only test environment at the moment I have is a linux VPS, so
there's bound to be random influences from virtualization and whatever
else happens to be running on the host at the time, so the noise is
high. Still, I was not able to see any clear difference between
gettimeofday(), Ns_GetTime, and Tcl_GetTime - on any given run any of
the three was equally likely to be fastest.
time-int the program reports typically 2:1 system time:user time.
What this suggests to me is that - at least in my environment - the
probable few hundred cycles difference in the implementations is
completely lost to syscall and timeslice overhead or possibly cache line
flushes, and it's not worth spending too much time worrying about it.
I'd be interested to hear anyone else's results and interpretations, or
suggestions about how to better measure the differences in these.
-J
/*
* test timing
* timing of raw gettimeofday vs Ns_GetTime and Tcl_GetTime
* compile with:
* cc -O2 -I include -I /usr/include/tcl8.5 -o tt tt.c -ltcl8.5 -L lib -lnsd -lnsthread -lpthread
*/
#include ns.h
#include tcl.h
#include stdio.h
#include sys/time.h
void showdiff(char *fn, int count, struct timeval s, struct timeval e) {
int diff=(e.tv_sec-s.tv_sec)*100 + (e.tv_usec-s.tv_usec);
fprintf(stderr,%s:\t%d usec %4.2f per\n,fn,diff,(double)diff/count);
}
int main(int argc, char** argv) {
struct timeval s,e;
struct timeval d;
int diff,c;
int diff_s,diff_us;
int count=1000;
Tcl_Time timePtr;
Ns_Time ntimePtr;
if (argc == 2) {
count=strtol(argv[1],NULL,10);
}
fprintf(stderr,count: %d\n,count);
/* warmup */
for (c=0;c100;c++) {
gettimeofday(d,NULL);
Tcl_GetTime(timePtr);
Ns_GetTime(ntimePtr);
}
gettimeofday(s,NULL);
for(c=0;ccount;c++) {
Tcl_GetTime(timePtr);
}
gettimeofday(e,NULL);
showdiff(Tcl_GetTime,count,s,e);
gettimeofday(s,NULL);
for(c=0;ccount;c++) {
Ns_GetTime(ntimePtr);
}
gettimeofday(e,NULL);
showdiff(Ns_GetTime,count,s,e);
gettimeofday(s,NULL);
for(c=0;ccount;c++) {
gettimeofday(d,NULL);
}
gettimeofday(e,NULL);
showdiff(gettimeofday,count,s,e);
}
--
Slashdot TV. Videos for Nerds. Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Dear Andrew, Did you happen to use this define _USE_32BIT_TIME_T? Hope it helps, Maurizio -Original Message- From: Andrew Piskorski [mailto:a...@piskorski.com] Sent: 03 October 2014 14:49 To: naviserver-devel@lists.sourceforge.net Subject: [naviserver-devel] Naviserver hangs on Windows With my changes here, the core Naviserver compiles now on my Windows 7 64-bit machine: https://bitbucket.org/naviserver/naviserver/pull-requests https://bitbucket.org/apiskors/naviserver/commits/all However, it doesn't work at all. If I simply run nsd -h, it hangs indefinitely, and locks up the Command Prompt window where I ran nsd. Control-c does not abort, I have to close the window. Note that I built Naviserver using ActiveTcl 8.5, I did not compile Tcl myself. I just started learning to use the WinDbg debugger. Below is a stack trace. This one is from attaching WinDbg to an already-running nsd.exe -h process, but it looks the same if I intead tell WinDbg to run nsd.exe itself (with no command line arguments) . From the stack trace, Naviserver seems to be getting stuck when Ns_GetTime (line 69 of time.c) calls TclpGetDate. But, line 69 does not explicitly call TclpGetDate, it justs manipulates a Tcl_Time tbuf structure like so: timePtr-sec = tbuf.sec; In Tcl 8.5.x, TclpGetDate is in win/tclWinTime.c, but the string TclpGetDate does not occur anywhere in the Naviserver code, so I'm not sure how it manages to call it. A function pointer somewhere? Any ideas what might be wrong here, or what else I should do to debug this further? Here the initial WinDbg output when attaching to a running nsd.exe -h process, and then the stack trace (the k command): *** wait with pending attach Symbol search path is: C:\web\nsd-atp\lib;http://msdl.microsoft.com/download/symbols Executable search path is: ModLoad: 0001`3f08 0001`3f0fe000 C:\web\nsd-atp\bin\nsd.exe ModLoad: `775e `77789000 C:\Windows\SYSTEM32\ntdll.dll ModLoad: `773c `774df000 C:\Windows\system32\kernel32.dll ModLoad: 07fe`fd43 07fe`fd49c000 C:\Windows\system32\KERNELBASE.dll ModLoad: 07fe`f068 07fe`f0828000 C:\web\nsd-atp\bin\nsd.dll ModLoad: 07fe`faab 07fe`fab4e000 C:\web\nsd-atp\bin\nsthread.dll ModLoad: `1000 `100dc000 C:\P\Tcl85\bin\tcl85.dll ModLoad: `774e `775da000 C:\Windows\system32\USER32.dll ModLoad: 07fe`fdd4 07fe`fdda7000 C:\Windows\system32\GDI32.dll ModLoad: 07fe`ff8e 07fe`ff8ee000 C:\Windows\system32\LPK.dll ModLoad: 07fe`ff47 07fe`ff539000 C:\Windows\system32\USP10.dll ModLoad: 07fe`ff2e 07fe`ff37f000 C:\Windows\system32\msvcrt.dll ModLoad: 07fe`ff38 07fe`ff45b000 C:\Windows\system32\ADVAPI32.dll ModLoad: 07fe`fdad 07fe`fdaef000 C:\Windows\SYSTEM32\sechost.dll ModLoad: 07fe`ff78 07fe`ff8ad000 C:\Windows\system32\RPCRT4.dll ModLoad: 07fe`fee0 07fe`fee4d000 C:\Windows\system32\WS2_32.dll ModLoad: 07fe`ff46 07fe`ff468000 C:\Windows\system32\NSI.dll ModLoad: 07fe`ff8b 07fe`ff8de000 C:\Windows\system32\IMM32.DLL ModLoad: 07fe`fd9c 07fe`fdac9000 C:\Windows\system32\MSCTF.dll Break-in sent, waiting 30 seconds... WARNING: Break-in timed out, suspending. This is usually caused by another thread holding the loader lock (e98.12c): Wake debugger - code 8007 (first chance) ntdll!ZwWaitForSingleObject+0xa: `776312fa c3 ret 0:000 kc Call Site ntdll!ZwWaitForSingleObject *** ERROR: Symbol file could not be found. Defaulted to export symbols for C:\P\Tcl85\bin\tcl85.dll - KERNELBASE!WaitForSingleObjectEx *** WARNING: Unable to verify checksum for C:\web\nsd-atp\bin\nsthread.dll tcl85!TclpGetDate [...] 0:000 k Child-SP RetAddr Call Site `0027eeb8 07fe`fd4310dc ntdll!ZwWaitForSingleObject+0xa `0027eec0 `100a4766 KERNELBASE!WaitForSingleObjectEx+0x79 `0027ef60 07fe`faab4139 tcl85!TclpGetDate+0x5c2 `0027efc0 07fe`faab1bf6 nsthread!Ns_GetTime+0x29 [z:\web\ns-fork\naviserver\nsthread\time.c @ 69] `0027f010 07fe`faab25f9 nsthread!Ns_MutexLock+0x76 [z:\web\ns-fork\naviserver\nsthread\mutex.c @ 210] `0027f0d0 07fe`faab144a nsthread!Ns_CsEnter+0x69 [z:\web\ns-fork\naviserver\nsthread\cslock.c @ 172] `0027f110 07fe`faab3db8 nsthread!Ns_MasterLock+0x2a [z:\web\ns-fork\naviserver\nsthread\master.c @ 71] `0027f140 07fe`faab2c58 nsthread!Ns_TlsAlloc+0x28 [z:\web\ns-fork\naviserver\nsthread\tls.c @ 77] `0027f180 07fe`faab33d2 nsthread!NsInitReentrant+0x28 [z:\web\ns-fork\naviserver\nsthread\reentrant.c @ 65] `0027f1b0 07fe`faab4514 nsthread!NsInitThreads+0x32 [z:\web\ns-fork\naviserver\nsthread\thread.c @ 105] `0027f1e0
Re: [naviserver-devel] Naviserver hangs on Windows
On Fri, Oct 03, 2014 at 03:21:02PM +0200, Maurizio Martignano wrote: Dear Andrew, Did you happen to use this define _USE_32BIT_TIME_T? No, I did not. Should I? Does Tcl use that? Grepping my source files, I see this: ./include/nsconfig.h:276:#define SIZEOF_TIME_T 8 That nsconfig.h was created on Linux, but my Windows build uses it too. Perhaps that is the problem? But how the heck can I generate a correct version of nsconfig.h for my Windows 7 system? -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Well I do not know... What is your target? W32 or W64? What is your target machine? E.g. X64? or AMD? How do you link the DLL? Maurizio -Original Message- From: Andrew Piskorski [mailto:a...@piskorski.com] Sent: 03 October 2014 15:58 To: naviserver-devel@lists.sourceforge.net Subject: Re: [naviserver-devel] Naviserver hangs on Windows On Fri, Oct 03, 2014 at 03:21:02PM +0200, Maurizio Martignano wrote: Dear Andrew, Did you happen to use this define _USE_32BIT_TIME_T? No, I did not. Should I? Does Tcl use that? Grepping my source files, I see this: ./include/nsconfig.h:276:#define SIZEOF_TIME_T 8 That nsconfig.h was created on Linux, but my Windows build uses it too. Perhaps that is the problem? But how the heck can I generate a correct version of nsconfig.h for my Windows 7 system? -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Dear Andrew, These are the basic options I would use for a Windows 64 bit, AMD64: COPTS = /EHsc /W3 /nologo /c DEFS= $(DEFS) /D _WINDOWS /D TCL_THREADS=1 /D WIN32 /D _WIN32 \ /D FD_SETSIZE=128 /D NO_CONST=1 /D _MBCS # /D _USE_32BIT_TIME_T LOPTS = /NOLOGO /SUBSYSTEM:CONSOLE /OPT:NOREF /OPT:NOICF Hope it helps, Maurizio -Original Message- From: Andrew Piskorski [mailto:a...@piskorski.com] Sent: 03 October 2014 15:58 To: naviserver-devel@lists.sourceforge.net Subject: Re: [naviserver-devel] Naviserver hangs on Windows On Fri, Oct 03, 2014 at 03:21:02PM +0200, Maurizio Martignano wrote: Dear Andrew, Did you happen to use this define _USE_32BIT_TIME_T? No, I did not. Should I? Does Tcl use that? Grepping my source files, I see this: ./include/nsconfig.h:276:#define SIZEOF_TIME_T 8 That nsconfig.h was created on Linux, but my Windows build uses it too. Perhaps that is the problem? But how the heck can I generate a correct version of nsconfig.h for my Windows 7 system? -- Andrew Piskorski a...@piskorski.com -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
Hm, the old AOLserver 4.0.7 version of Ns_GetTime() had a Windows
ifdef which is entirely missing from Naviserver! (See below.)
I found two relevant commits in Mercurial, from 2009 and 2012, below.
From those logs, it looks like what happened is that in 2009, Zoran
removed the platform specific C code and instead made Naviserver call
Tcl_GetTime (somehow). Then in 2012, Gustaf added back the old
Unix-specific gettimeofday() implementation for performance reasons.
So should we just add back the old Windows-specific C implementation
too? Or is it better to use Zoran's way of invoking Tcl_GetTime? Do
we even know for sure if that method really is platform-independent
as Zoran thought it was? E.g., did Zoran or someone else at Archiware
test it on Windows?
Part of Ns_GetTime() from ancient AOLserver 4.0.7 sources:
#ifdef _WIN32
/*
* Number of 100 nanosecond units from 1/1/1601 to 1/1/1970
*/
#define EPOCH_BIAS 1164447360i64
union {
unsigned __int64i;
FILETIMEs;
} ft;
GetSystemTimeAsFileTime(ft.s);
timePtr-sec = (time_t)((ft.i - EPOCH_BIAS) / 1000i64);
timePtr-usec =(long)((ft.i / 10i64) % 100i64);
#else
#endif
changeset: 2435:16508ab4b3de
user:Gustaf Neumann neum...@wu-wien.ac.at
date:Fri Nov 02 01:54:38 2012 +0100
files: configure.in include/nsconfig.h.in nsthread/time.c
description:
use gettimeofday() if available instead of the rount trip to tcl
changeset: 2009:4bf1ee6ebf84
user:Zoran Vasiljevic z...@archiware.com
date:Sat Sep 29 14:14:03 2007 +0100
files: ChangeLog nsproxy/nsproxylib.c nsthread/time.c
description:
Ns_GetTime now relies on Tcl_GetTime to be platform-independent
--
Andrew Piskorski a...@piskorski.com
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Naviserver hangs on Windows
The Windows-OpenACS distribution which I make available here
(http://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/) is based
on AOLServer 4.5.2, contains the sources, is compiled with Visual Studio
2013 and runs on Windows 64.
So if I where you I would give a look at that distribution and see how all
those time related functions have been handled.
That could save you some time.
Maurizio
-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com]
Sent: 03 October 2014 16:49
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] Naviserver hangs on Windows
Hm, the old AOLserver 4.0.7 version of Ns_GetTime() had a Windows ifdef
which is entirely missing from Naviserver! (See below.)
I found two relevant commits in Mercurial, from 2009 and 2012, below.
From those logs, it looks like what happened is that in 2009, Zoran
removed the platform specific C code and instead made Naviserver call
Tcl_GetTime (somehow). Then in 2012, Gustaf added back the old
Unix-specific gettimeofday() implementation for performance reasons.
So should we just add back the old Windows-specific C implementation too?
Or is it better to use Zoran's way of invoking Tcl_GetTime? Do we even know
for sure if that method really is platform-independent
as Zoran thought it was? E.g., did Zoran or someone else at Archiware test
it on Windows?
Part of Ns_GetTime() from ancient AOLserver 4.0.7 sources:
#ifdef _WIN32
/*
* Number of 100 nanosecond units from 1/1/1601 to 1/1/1970 */ #define
EPOCH_BIAS 1164447360i64
union {
unsigned __int64i;
FILETIMEs;
} ft;
GetSystemTimeAsFileTime(ft.s);
timePtr-sec = (time_t)((ft.i - EPOCH_BIAS) / 1000i64);
timePtr-usec =(long)((ft.i / 10i64) % 100i64); #else #endif
changeset: 2435:16508ab4b3de
user:Gustaf Neumann neum...@wu-wien.ac.at
date:Fri Nov 02 01:54:38 2012 +0100
files: configure.in include/nsconfig.h.in nsthread/time.c
description:
use gettimeofday() if available instead of the rount trip to tcl
changeset: 2009:4bf1ee6ebf84
user:Zoran Vasiljevic z...@archiware.com
date:Sat Sep 29 14:14:03 2007 +0100
files: ChangeLog nsproxy/nsproxylib.c nsthread/time.c
description:
Ns_GetTime now relies on Tcl_GetTime to be platform-independent
--
Andrew Piskorski a...@piskorski.com
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI
DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you
Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI
DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel
