Re: [naviserver-devel] ns_proxy hang

2017-05-13 Thread Maurizio Martignano
Dear all,

 

CBMC (http://www.cprover.org/cbmc/) symbolic execution of “nsproxy” reports an 
error in RecvBuff. It could be a false positive.

 

Hope it helps,

Maurizio

 

 

[SendBuf.assertion.1] assertion (slavePtr != ((void *)0)): SUCCESS

[SendBuf.assertion.2] assertion (dsPtr != ((void *)0)): SUCCESS

[SendBuf.overflow.1] arithmetic overflow on signed * in (ms % 1000) * 1000: 
SUCCESS

[SendBuf.pointer_dereference.1] dereference failure: pointer NULL in *dsPtr: 
SUCCESS

[SendBuf.pointer_dereference.2] dereference failure: pointer invalid in *dsPtr: 
SUCCESS

[SendBuf.pointer_dereference.3] dereference failure: deallocated dynamic object 
in *dsPtr: SUCCESS

[SendBuf.pointer_dereference.4] dereference failure: dead object in *dsPtr: 
SUCCESS

[SendBuf.pointer_dereference.5] dereference failure: pointer outside dynamic 
object bounds in *dsPtr: SUCCESS

[SendBuf.pointer_dereference.6] dereference failure: pointer outside object 
bounds in *dsPtr: SUCCESS

[SendBuf.pointer_dereference.7] dereference failure: pointer NULL in *dsPtr: 
SUCCESS

[SendBuf.pointer_dereference.8] dereference failure: pointer invalid in *dsPtr: 
SUCCESS

[SendBuf.pointer_dereference.9] dereference failure: deallocated dynamic object 
in *dsPtr: SUCCESS

[SendBuf.pointer_dereference.10] dereference failure: dead object in *dsPtr: 
SUCCESS

[SendBuf.pointer_dereference.11] dereference failure: pointer outside dynamic 
object bounds in *dsPtr: SUCCESS

[SendBuf.pointer_dereference.12] dereference failure: pointer outside object 
bounds in *dsPtr: SUCCESS

[SendBuf.pointer_dereference.13] dereference failure: pointer outside dynamic 
object bounds in *dsPtr: SUCCESS

[SendBuf.pointer_dereference.14] dereference failure: pointer outside object 
bounds in *dsPtr: SUCCESS

[SendBuf.overflow.2] arithmetic overflow on unsigned + in iov[(signed long long 
int)0].iov_len + iov[(signed long long int)1].iov_len: SUCCESS

[SendBuf.pointer_dereference.15] dereference failure: pointer NULL in 
*slavePtr: SUCCESS

[SendBuf.pointer_dereference.16] dereference failure: pointer invalid in 
*slavePtr: SUCCESS

[SendBuf.pointer_dereference.17] dereference failure: deallocated dynamic 
object in *slavePtr: SUCCESS

[SendBuf.pointer_dereference.18] dereference failure: dead object in *slavePtr: 
SUCCESS

[SendBuf.pointer_dereference.19] dereference failure: pointer outside dynamic 
object bounds in *slavePtr: SUCCESS

[SendBuf.pointer_dereference.20] dereference failure: pointer outside object 
bounds in *slavePtr: SUCCESS

[SendBuf.pointer_dereference.21] dereference failure: pointer NULL in 
*return_value__errno$1: SUCCESS

[SendBuf.pointer_dereference.22] dereference failure: pointer invalid in 
*return_value__errno$1: SUCCESS

[SendBuf.pointer_dereference.23] dereference failure: deallocated dynamic 
object in *return_value__errno$1: SUCCESS

[SendBuf.pointer_dereference.24] dereference failure: dead object in 
*return_value__errno$1: SUCCESS

[SendBuf.pointer_dereference.25] dereference failure: pointer outside dynamic 
object bounds in *return_value__errno$1: SUCCESS

[SendBuf.pointer_dereference.26] dereference failure: pointer outside object 
bounds in *return_value__errno$1: SUCCESS

[SendBuf.pointer_dereference.27] dereference failure: pointer NULL in 
*return_value__errno$2: SUCCESS

[SendBuf.pointer_dereference.28] dereference failure: pointer invalid in 
*return_value__errno$2: SUCCESS

[SendBuf.pointer_dereference.29] dereference failure: deallocated dynamic 
object in *return_value__errno$2: SUCCESS

[SendBuf.pointer_dereference.30] dereference failure: dead object in 
*return_value__errno$2: SUCCESS

[SendBuf.pointer_dereference.31] dereference failure: pointer outside dynamic 
object bounds in *return_value__errno$2: SUCCESS

[SendBuf.pointer_dereference.32] dereference failure: pointer outside object 
bounds in *return_value__errno$2: SUCCESS

[SendBuf.pointer_dereference.33] dereference failure: pointer NULL in 
*return_value__errno$3: SUCCESS

[SendBuf.pointer_dereference.34] dereference failure: pointer invalid in 
*return_value__errno$3: SUCCESS

[SendBuf.pointer_dereference.35] dereference failure: deallocated dynamic 
object in *return_value__errno$3: SUCCESS

[SendBuf.pointer_dereference.36] dereference failure: dead object in 
*return_value__errno$3: SUCCESS

[SendBuf.pointer_dereference.37] dereference failure: pointer outside dynamic 
object bounds in *return_value__errno$3: SUCCESS

[SendBuf.pointer_dereference.38] dereference failure: pointer outside object 
bounds in *return_value__errno$3: SUCCESS

[SendBuf.pointer_dereference.39] dereference failure: pointer NULL in 
*slavePtr: SUCCESS

[SendBuf.pointer_dereference.40] dereference failure: pointer invalid in 
*slavePtr: SUCCESS

[SendBuf.pointer_dereference.41] dereference failure: deallocated dynamic 
object in *slavePtr: SUCCESS

[SendBuf.pointer_dereference.42] dereference failure: dead object in *slavePtr: 
SUCCESS

[SendBuf.pointer_dereference.43] dereference 

Re: [naviserver-devel] ns_urlencode on Windows Naviserver

2017-05-12 Thread Maurizio Martignano
Dear Brian,

 

   a short look into the code, at the two different
implementations (and how much they differ) would have  avoided this wrong
assumption.

 

On top of that, as Gustaf suggested, using the "-part path" option you can
obtain the full URL properly converted.

 

The main point, the main difference with the old implementation, is about
how "reserved characters", like "/", ":",  have to be treated.

 

The old implementation percent-encode reserved characters in any situation.

The new, and I guess correct, implementation percent-encode these character
only when they are not used as delimiters.

 

This is the relevant RFC 3986 text:
"

Reserved Characters

 

   URIs include components and subcomponents that are delimited by

   characters in the "reserved" set.  These characters are called

   "reserved" because they may (or may not) be defined as delimiters by

   the generic syntax, by each scheme-specific syntax, or by the

   implementation-specific syntax of a URI's dereferencing algorithm.

   If data for a URI component would conflict with a reserved

   character's purpose as a delimiter, then the conflicting data must be

   percent-encoded before the URI is formed.

"

 

Hope it helps,

Maurizio

 

 

From: Brian Fenton [mailto:brian.fen...@quest.ie] 
Sent: 12 May 2017 16:18
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver

 

Thanks Gustaf

 

An issue arose during migration testing which was breaking some of our
existing AOLserver code, then when I saw the documentation issue, I thought
best to check with the list that maybe there was an issue with the Windows
version.
I'll dig into our code and see what the problem is exactly.

 

thanks

Brian

 

From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 12 May 2017 14:52
To: naviserver-devel@lists.sourceforge.net
 
Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver

 

Am 12.05.17 um 13:41 schrieb Brian Fenton:

Thank you David

 

So, assuming the documentation is incorrect
https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html  it
seems that the API has been changed from AOLserver.

 

Brian

Brian, are you worried about the literal comparison in a test case, or is
there a deeper background?

Some more background: ns_urlencode in AOLserver was always incorrect in
respect to the RFCs, which define different encodings for the "path" and the
"query" part of an URL. Therefore many years ago, NaviServer added flags
"-part" to specify the encoding (implementing an extension of RFC1738
(1994)). 

In the (unreleased) tip version of NaviServer on bitbucket, i've updated the
implementation to be in sync with valid RFCs (RFC 3986 for URIs and  RFC
6265, cookie encodings). So if one compares the result of a tip version of
NaviServer with the documentation of the released version, the results
differ (btw., the example on the NaviServer man page is bad). See about the
changes in the tip see  e.g. [1] and the following postings.

-g

[1] https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50
 =201703=22

-- 
Univ.Prof. Dr. Gustaf Neumann
WU Vienna
Institute of Information Systems and New Media
Welthandelsplatz 1, A-1020 Vienna, Austria
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] ns_urlencode on Windows Naviserver

2017-05-12 Thread Maurizio Martignano
Dear Brian,

   Is this really a "Windows Naviserver" issue or is it a matter
of a new implementation of the function, different from what was previously
done in Aolserver? 

 

The two implementations seem to me quite different.

 

Thanks a lot,

Maurizio

 

 

From: Brian Fenton [mailto:brian.fen...@quest.ie] 
Sent: 12 May 2017 11:15
To: 'naviserver-devel@lists.sourceforge.net'

Subject: [naviserver-devel] ns_urlencode on Windows Naviserver

 

I think there might be a problem with ns_urlencode on Windows Naviserver.

 

I just ran this example (taken directly from the documentation):

ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com

=1,2,3

 

And the result was unexpected:
http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2
c

 

The correct result is
http%3a%2f%2fwww%2eaolserver%2ecom%2fredirect%2eadp%3furl%3dhttp%3a%2f%2fwww
%2eaol%2ecom%26t%3d1%2c2%2c3

 

This is breaking some of our existing AOLserver code during migration
testing.

 

Any suggestions?

 



Brian Fenton
Senior Developer

Phone: +353 1 679 9933
  grantmanagementsoftware.com

  


 



  

  

Quest Computing Ltd. Ushers Court | 31-33 Ushers Quay | Dublin 8, Ireland


 


 


This e-mail transmission may contain confidential information that is
intended for the individual or entity named on the e-mail address. If you
are not the intended recipient, please reply to the sender so that Quest
Computing Ltd can arrange for the proper delivery and then please delete the
message from your inbox.

If you have received this e-mail in error, you are hereby notified that any
disclosure, copying, distribution, or reliance upon the contents of this
e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, 31-33
Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in
Ireland No. 146435.

 

 

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] IPv6 for NaviServer

2016-02-25 Thread Maurizio Martignano
Dear all,

   To complement my previous message, I have observed that the
system compiled with HAVE_IPV4 works.

On the contrary, if compiled with HAVE_IPV6, still presents some few
problems.

So the Windows-OpenACS distribution will have at the moment Naviserver
compiled with HAVE_IPV4.

In the meantime and with no rush I will try to investigate the problems of
the HAVE_IPV6 version.

 

Thank you,

Maurizio

 

 

From: Maurizio Martignano [mailto:maurizio.martign...@spazioit.com] 
Sent: 25 February 2016 22:20
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Dear all,

   This is a comment to my previous message.

 

Unfortunately, the compiled system is still not working properly. I need to
investigate more in details what the problems are.

 

In the mean time I put back in the Windows-OpenACS, the Naviserver commit of
the 24th of February, the one without IPV6 Support.

 

Thank you,

Maurizio

 

 

From: Maurizio Martignano [mailto:maurizio.martign...@spazioit.com] 
Sent: 25 February 2016 18:05
To: naviserver-devel@lists.sourceforge.net
<mailto:naviserver-devel@lists.sourceforge.net> 
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Dear Gustaf,

   Thank you for your inputs.

I implemented all your changes but the system was still not working.

I had a careful look at it and eventually I found the cause (at least in my
configuration):



The problem in caused by the function inet_ntop in nswin32.c. After removing
it, everything works fine.

The system seems now OK to me.

 

Thank you,

Maurizio

 

 

 

 

From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 25 February 2016 12:33
To: naviserver-devel@lists.sourceforge.net
<mailto:naviserver-devel@lists.sourceforge.net> 
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Hi Maurizio,

thanks for the fixes. as such, they could not work, but they point
out things, where one has to watch out. I have fixed these
things in a way it could work under windows.

By looking at the differences, it seems as if you compiled just a 
IPv4 version (no IPv6 enabled). i would have expected this
to go more smoothly. In case you tried the sample 
nsd-config.tcl script, that could not work with IPv4, since
IPv6 addresses were hard-coded.

all the best
-g

Am 24.02.16 um 18:00 schrieb Maurizio Martignano:

Dear Gustaf,
  Thank you as always for all your work.
 
About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
Visual Studio 2015 with 64 bit as target.
With this configuration I tried to modify the file sockaddr.c (to make it
compile), but I'm not sure about the changes I made.
I also had to modify the file nswin32.c as I could not find anywhere the
function "NsSockGetPort".
Well the resulting system compiles... but nothing works.
I need much more time to do a proper testing/tuning and I can't do that at
the moment.
As soon as I discover more, I'll let you know.
 
Back to your delivery of the 16th of February.
 
Once again thank you,
Maurizio
 
 
-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 24 February 2016 14:01
To: Navidevel  <mailto:naviserver-devel@lists.sourceforge.net>
<naviserver-devel@lists.sourceforge.net>
Subject: [naviserver-devel] IPv6 for NaviServer
 
Dear all,
 
over the last week, i've worked on IPv6 support for NaviServer.
For this task, i took the following path:
1) build a version of NaviServer for IPv6
2) build a single code source for compiling either for IPv4 or IPv6
3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
4) Run regression test suite under IPv4 and/IPv6
5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
(requires some interface changes to handle generic sockaddrs)
 
Everything seems to work fine, one can work with the same config files as
before, and everything works as expected, but one can as well listen on IPv6
addresses (e.g.
the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
kernel allows to deliver as well
IPv4 traffic to the IPv6 interface, then one can talk also multiple
protocols. The mileage may vary depending on the OS and kernel.
 
It is as well possible to deactivate IPv6 support at compile time via
--disable-ipv6.
 
Here is a summary of the necessary changes:
- Added a amall Abstraction API for IP-version agnostic calls:
   Ns_LogSockaddr(), Ns_SockaddrGetPort(),
   Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
   Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
 
- Replaced in interface more places where "sockaddr_in" (the IPv4
   version of the sockaddr) with the generic (version agnostic)
   sockaddr (also in IPv4 code to obtain API consistency when no IPv6
   is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
   Ns_SockBind(), Ns_SockBindUdp()
 
- The Ns_Sock structure contained sockaddr_in, whic

Re: [naviserver-devel] IPv6 for NaviServer

2016-02-25 Thread Maurizio Martignano
Dear all,

   This is a comment to my previous message.

 

Unfortunately, the compiled system is still not working properly. I need to
investigate more in details what the problems are.

 

In the mean time I put back in the Windows-OpenACS, the Naviserver commit of
the 24th of February, the one without IPV6 Support.

 

Thank you,

Maurizio

 

 

From: Maurizio Martignano [mailto:maurizio.martign...@spazioit.com] 
Sent: 25 February 2016 18:05
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Dear Gustaf,

   Thank you for your inputs.

I implemented all your changes but the system was still not working.

I had a careful look at it and eventually I found the cause (at least in my
configuration):



The problem in caused by the function inet_ntop in nswin32.c. After removing
it, everything works fine.

The system seems now OK to me.

 

Thank you,

Maurizio

 

 

 

 

From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 25 February 2016 12:33
To: naviserver-devel@lists.sourceforge.net
<mailto:naviserver-devel@lists.sourceforge.net> 
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Hi Maurizio,

thanks for the fixes. as such, they could not work, but they point
out things, where one has to watch out. I have fixed these
things in a way it could work under windows.

By looking at the differences, it seems as if you compiled just a 
IPv4 version (no IPv6 enabled). i would have expected this
to go more smoothly. In case you tried the sample 
nsd-config.tcl script, that could not work with IPv4, since
IPv6 addresses were hard-coded.

all the best
-g

Am 24.02.16 um 18:00 schrieb Maurizio Martignano:

Dear Gustaf,
  Thank you as always for all your work.
 
About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
Visual Studio 2015 with 64 bit as target.
With this configuration I tried to modify the file sockaddr.c (to make it
compile), but I'm not sure about the changes I made.
I also had to modify the file nswin32.c as I could not find anywhere the
function "NsSockGetPort".
Well the resulting system compiles... but nothing works.
I need much more time to do a proper testing/tuning and I can't do that at
the moment.
As soon as I discover more, I'll let you know.
 
Back to your delivery of the 16th of February.
 
Once again thank you,
Maurizio
 
 
-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 24 February 2016 14:01
To: Navidevel  <mailto:naviserver-devel@lists.sourceforge.net>
<naviserver-devel@lists.sourceforge.net>
Subject: [naviserver-devel] IPv6 for NaviServer
 
Dear all,
 
over the last week, i've worked on IPv6 support for NaviServer.
For this task, i took the following path:
1) build a version of NaviServer for IPv6
2) build a single code source for compiling either for IPv4 or IPv6
3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
4) Run regression test suite under IPv4 and/IPv6
5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
(requires some interface changes to handle generic sockaddrs)
 
Everything seems to work fine, one can work with the same config files as
before, and everything works as expected, but one can as well listen on IPv6
addresses (e.g.
the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
kernel allows to deliver as well
IPv4 traffic to the IPv6 interface, then one can talk also multiple
protocols. The mileage may vary depending on the OS and kernel.
 
It is as well possible to deactivate IPv6 support at compile time via
--disable-ipv6.
 
Here is a summary of the necessary changes:
- Added a amall Abstraction API for IP-version agnostic calls:
   Ns_LogSockaddr(), Ns_SockaddrGetPort(),
   Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
   Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
 
- Replaced in interface more places where "sockaddr_in" (the IPv4
   version of the sockaddr) with the generic (version agnostic)
   sockaddr (also in IPv4 code to obtain API consistency when no IPv6
   is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
   Ns_SockBind(), Ns_SockBindUdp()
 
- The Ns_Sock structure contained sockaddr_in, which is not sufficient
   to store IPv6 addresses. Therefore it was necessary to change
   that to "sockaddr_storage" which guarantees to be large enough
   to keep all kind of addresses. This affects as well driver modules
   which communicate via the NaviServer managed sockets. Old drivers
   might crash with the new structure. Therefore, I've bumped
   NS_DRIVER_VERSION to 3 such that driver modules (also binaries)
   can communicate that they support IPv6. In IPv6 mode, NaviServer
   refuses to work with versions <3.
 
- Replaced all usages of ns_inet_ntoa() by ns_intet_ntop()
   (except for legacy an minimal library support)
 
- Added log file warnings on various

Re: [naviserver-devel] IPv6 for NaviServer

2016-02-25 Thread Maurizio Martignano
Dear Gustaf,

   Thank you for your inputs.

I implemented all your changes but the system was still not working.

I had a careful look at it and eventually I found the cause (at least in my
configuration):



The problem in caused by the function inet_ntop in nswin32.c. After removing
it, everything works fine.

The system seems now OK to me.

 

Thank you,

Maurizio

 

 

 

 

From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 25 February 2016 12:33
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] IPv6 for NaviServer

 

Hi Maurizio,

thanks for the fixes. as such, they could not work, but they point
out things, where one has to watch out. I have fixed these
things in a way it could work under windows.

By looking at the differences, it seems as if you compiled just a 
IPv4 version (no IPv6 enabled). i would have expected this
to go more smoothly. In case you tried the sample 
nsd-config.tcl script, that could not work with IPv4, since
IPv6 addresses were hard-coded.

all the best
-g

Am 24.02.16 um 18:00 schrieb Maurizio Martignano:

Dear Gustaf,
  Thank you as always for all your work.
 
About Windows I cannot really talk about MinGW nor Cygwin. I'm just using
Visual Studio 2015 with 64 bit as target.
With this configuration I tried to modify the file sockaddr.c (to make it
compile), but I'm not sure about the changes I made.
I also had to modify the file nswin32.c as I could not find anywhere the
function "NsSockGetPort".
Well the resulting system compiles... but nothing works.
I need much more time to do a proper testing/tuning and I can't do that at
the moment.
As soon as I discover more, I'll let you know.
 
Back to your delivery of the 16th of February.
 
Once again thank you,
Maurizio
 
 
-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 24 February 2016 14:01
To: Navidevel  <mailto:naviserver-devel@lists.sourceforge.net>
<naviserver-devel@lists.sourceforge.net>
Subject: [naviserver-devel] IPv6 for NaviServer
 
Dear all,
 
over the last week, i've worked on IPv6 support for NaviServer.
For this task, i took the following path:
1) build a version of NaviServer for IPv6
2) build a single code source for compiling either for IPv4 or IPv6
3) let the IPv6 version run the setup-files for IPv4 in the IPv6 version
4) Run regression test suite under IPv4 and/IPv6
5) Allow mixed mode (handle IPv4 and IPv6 by the same binary)
(requires some interface changes to handle generic sockaddrs)
 
Everything seems to work fine, one can work with the same config files as
before, and everything works as expected, but one can as well listen on IPv6
addresses (e.g.
the IPv6 unspecified address "::"). IPv4->6 mapping is activated, or the
kernel allows to deliver as well
IPv4 traffic to the IPv6 interface, then one can talk also multiple
protocols. The mileage may vary depending on the OS and kernel.
 
It is as well possible to deactivate IPv6 support at compile time via
--disable-ipv6.
 
Here is a summary of the necessary changes:
- Added a amall Abstraction API for IP-version agnostic calls:
   Ns_LogSockaddr(), Ns_SockaddrGetPort(),
   Ns_SockaddrGetSockLen(), Ns_SockaddrMask(), Ns_SockaddrMaskBits(),
   Ns_SockaddrSetPort(), ns_inet_ntop(), ns_inet_pton()
 
- Replaced in interface more places where "sockaddr_in" (the IPv4
   version of the sockaddr) with the generic (version agnostic)
   sockaddr (also in IPv4 code to obtain API consistency when no IPv6
   is available): Ns_BindSock(), Ns_ConnSetPeer(), Ns_GetSockAddr(),
   Ns_SockBind(), Ns_SockBindUdp()
 
- The Ns_Sock structure contained sockaddr_in, which is not sufficient
   to store IPv6 addresses. Therefore it was necessary to change
   that to "sockaddr_storage" which guarantees to be large enough
   to keep all kind of addresses. This affects as well driver modules
   which communicate via the NaviServer managed sockets. Old drivers
   might crash with the new structure. Therefore, I've bumped
   NS_DRIVER_VERSION to 3 such that driver modules (also binaries)
   can communicate that they support IPv6. In IPv6 mode, NaviServer
   refuses to work with versions <3.
 
- Replaced all usages of ns_inet_ntoa() by ns_intet_ntop()
   (except for legacy an minimal library support)
 
- Added log file warnings on various places, where NaviServer
   was ignoring error states in socket communications silently.
 
- Transformed hash table for open listen ports to string keys
   (the old code used 32bit IPv4-addresses as keys in hash tables)
- added test for ns_listencallback
 
- Ns_HttpParseHost: new abstraction for parsing host and port number.
   Reason: IPv6 literal notation contains many colons, parsing
   IP-literal notation according to RFC 3986 section 3.2.2 is necessary
   (removed places where code was searching for ':' to expect the port)
 
- Added [ns_info ipv6] to obtain information whether the binary
   supports IPv6 or not
 

Re: [naviserver-devel] more logging problems on Windows

2014-11-14 Thread Maurizio Martignano
Dear Andrew,
I would like to draw your attention to the availability of Visual
Studio 2013 Community Edition.

http://msdn.microsoft.com/en-us/visual-studio-community-vs.aspx

Hope it helps,
Maurizio


-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 07 November 2014 14:00
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] more logging problems on Windows

On Thu, Nov 06, 2014 at 03:03:18PM +0100, Gustaf Neumann wrote:

 The seemingly same write() operation, which refuses to write to the 
 access.log in driver.c works fine when called directly in nslog.c.

 https://bitbucket.org/naviserver/naviserver/commits/4cb76f20a8464fde98
 ac3147c6184a90442a2808

Wow, weird.  I tried turning your new fix on/off, and confirm the behavior
you see.  Yet as you said, the new code you added to
NsAsyncWrite() is the same as the old/normal code in NsAsyncWrite()!

So what could the difference possibly be?  Since we're not actually using
the async writer thread feature at all, it's even the same thread calling
_write() in both cases.  LogTrace() is in nslog.dll while NsAsyncWrite() is
in libnsd.dll, but I don't see how/why that could possibly matter.

Microsoft's docs for _write() and _open() don't mention anything
suspicious:

  http://msdn.microsoft.com/en-us/library/1570wh78%28v=vs.100%29.aspx
  http://msdn.microsoft.com/en-us/library/z0kc8e3z%28v=vs.100%29.aspx

WinDbg thinks the source file used for Microsoft's _write() was:
  f:\dd\vctools\crt_bld\self_x86\crt\src\write.c 
Naturally I don't have that exact path, but I do have this, which seems to
be the same thing:
  c:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\crt\src\write.c 

For binary files, _write() is a reasonably simple wrapper around
WriteFile(), but it's surprisingly complicated for text-mode files.

My access log seems to always be file descriptor 3, so in WinDbg I set a
breakpoint like this:

  bu NsAsyncWrite .if (fd = 0n3) {} .else {gc} 

Naviserver opens its access log in text mode; that's the default, and by
stepping through in WinDbg I do see it enter some of the text-mode sections
in _write_nolock().

As a little experiment, in LogOpen() I added a _O_BINARY flag, so it opened
the access log like this:

fd = open(logPtr-file, O_APPEND|O_WRONLY|O_CREAT|_O_BINARY, 0644); 

That made no difference to the behavior.  (I did NOT run that under WinDbg
to confirm, but presumably it was in binary mode like I asked.) So this lost
writes bug doesn't seem to depend on text vs. binary mode, it's the same for
both.

So, I have no idea.  The only ways I can imagine to try further to track
this down would be:

1. See if the bug is replicable in a standalone, non-Naviserver C program.
Maybe with two different DLLs involved.

2. Or, in NsAsyncWrite, do not use _write(), instead call WriteFile()
directly and see what happens.

That sounds like a lot of work though, much more than this really warrants.
I'm really glad you found a work-around, Gustaf!  I suggest we just leave it
as is, but add some comments about why it's there.

--
Andrew Piskorski a...@piskorski.com


--
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] Code Quality

2014-11-13 Thread Maurizio Martignano
Dear Ibrahim,
In a way, behind the scenes, I have been helping Gustaf in improving
the quality of Naviserver.

My company, Spazio IT, is active providing Verification and Validation
activities to the European Space Agency and AIRBUS Helicopters.
http://www.spazioit.com/pages_en/sol_inf_en/code_quality_en/
For them I have developed a code quality platform based on the open source
web application SonarQube (http://www.sonarqube.org). In particular I have
modified and customized the Ada and C/C++ plugins for the special needs of
my customers.

To make a long story short I was also interested in the quality and
stability of Naviserver so I made available a demo of the above mentioned
quality platform to Gustaf and I run analyses on Naviserver on a daily basis
the system on a daily basis, spending on it something in between 30minutes -
1 hour every day.
I have been doing this since the 28th of September.
The demo system, with information on Naviserver, is available here.
http://sonarsrv.spazioit.com:9000/ 
Basically most of the improvements Gustaf has been implementing have been
triggered by the analyses results made available on this platform.


In terms of research activity I'm currently integrating into the platform as
improvements to the C/C++ static analysis it provides CBMC
(http://www.cprover.org/cbmc/) and Frama-C (http://frama-c.com/).

The results of this research  will be presented at the European Space Agency
in December. After the event I will made available the presentation and
speech also to this list.

For the time being please have a look at the following presentation.
http://www.spazioit.com/pages_en/sol_inf_en/code_quality_en/code-inspection_
en/ 

Thanks and kind regards,
Maurizio Martignano 
Tel +39 0376 1434259 - Fax +39 0376 1434187
Mob +39 349 7284927
--- 
Spazio IT - Soluzioni Informatiche s.a.s. 
P.IVA: IT02178630204 / REA: MN - 231014 

Registered Office: via Manzoni 40
Operating Office: via Don Bosco 10  
46030 - San Giorgio di Mantova 
ITALY 
www.spazioit.com 









-Original Message-
From: Ibrahim Tannir [mailto:itan...@archiware.com] 
Sent: 13 November 2014 13:00
To: naviserver-devel@lists.sourceforge.net
Subject: [naviserver-devel] Code Quality

Hello everybody,

First of all, I'd like to thank Gustaf and Andrew who are currently
investing a lot of effort into cleaning up the code.

Since Gustaf seems to have some time that he is dedicating into our cause
right now, an idea struck me, that could bring us and the code up to the
next level:

I have been monitoring Coverity and their development for years
http://www.coverity.com/products/coverity-save/
Even tough they target high end and high profile software and companies,
they have been quite supportive of the open community. 
So I thought we might also ask them to allow us to run Naviserver and Tcl
through their static code analysis suite.

To be able to do so, someone must invest the time and the effort. 
Both Zoran and I are up to our necks in work. However, if Gustaf would be
willing to do so, given the time he is already investing, that would be
great. It might even speed up and simplify the work he is currently doing.

@Gustav, what do you think?

Cheers,
Ibrahim



--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] Naviserver hangs on Windows

2014-10-08 Thread Maurizio Martignano
Dear Andrew,
Sorry for the late reply, but have been away of my computer.

1. _USE_32BIT_TIME_T
Indeed, you are absolutely write. My Windows distribution had since 2011 to
2013 both Windows 32 and Windows 64 binaries.
I was using more or less the same make and configuration files to build it.
The define _USE_32BIT_TIME_T was used for the Windows 32 build but not for
the Windows 64. The current version of my distribution only contains Windows
64 binaries and in fact the define is commented out in the makefile.
Sorry for the time I made you waste on this point.

2. About the configuration
I am using the tools made available by Visual Studio 2013 Professional
Edition in the VS2013 X64 Native Tools Command Prompt. That is:
CL: Microsoft (R) C/C++ Optimizing Compiler Version 18.00.21005.1 for x64
LINK: Microsoft (R) Incremental Linker Version 12.00.21005.1
This is what I use and your questions below:
COPTS   = /EHsc /W3 /nologo /c
DEFS= $(DEFS) /D _WINDOWS /D TCL_THREADS=1 /D WIN32 /D _WIN32 \
  /D FD_SETSIZE=128 /D NO_CONST=1 /D _MBCS # /D
_USE_32BIT_TIME_T
LOPTS   = /NOLOGO /SUBSYSTEM:CONSOLE /OPT:NOREF /OPT:NOICF

Did you comment out your _USE_32BIT_TIME_T there?
[MM] Absolutely, I already explained what I did in point 1 and once again
apologies.
What does /D _WIN32 do?
[MM] This should be set automatically by the compiler, so in theory it is
redundant... but one never knows...
What C run-time library are you using?
[MM] Visual C++ Redistributable Packages for Visual Studio 2013 (Release, No
Debug)
http://www.microsoft.com/en-us/download/details.aspx?id=40784

I do agree with your remarks in your mail, I had a look at the Visual Studio
Project Files you use, the only relevant differences I can see are the
following:
1. you are targeting as Machine_X64, I am targeting AMD64
2. you use Active TCL, I did compile from scratch also TCL, using the same
compiler options (I believe this is very important as we have no control on
how Active TCL is built)

Hope this helps,
Maurizio




-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 07 October 2014 22:36
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] Naviserver hangs on Windows

On Mon, Oct 06, 2014 at 05:18:08PM -0400, Andrew Piskorski wrote:
 On Sun, Oct 05, 2014 at 09:37:39PM +0200, Maurizio Martignano wrote:
   Did you use the define _USE_32BIT_TIME_T yes or not?

 I haven't tried using _USE_32BIT_TIME_T yet, but I think using it 
 would be INCORRECT on Windows-64.

If I do pass the /D _USE_32BIT_TIME_T flag, I get this:

C:\Program Files (x86)\Microsoft Visual Studio
10.0\VC\INCLUDE\crtdefs.h(463) : fatal error C1189: #error :  You cannot use
32-bit time_t (_USE_32BIT_TIME_T) with _WIN64 

So that seems pretty clear, I must NOT set that flag.

--
Andrew Piskorski a...@piskorski.com


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI
DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you
Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI
DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] warning, shift count undefined behavior

2014-10-08 Thread Maurizio Martignano
SonarQube (http://www.sonarqube.org) is just a web application displaying
the results of some analyses performed by other tools.

I am currently working on an Ada plugin and C/C++ plugin that I use for some
Verification and Validation activities.

For each language there might be a plugin. For each plugin there might be
the so called sensors (i.e. interfaces to external tools).

In the installation you see the C plugin is using two sensors: one for
CppCheck and one for PClint/FLEXELint (configured to check compliance with
MISRA C 2004).

Sometimes the warnings may be kind of misleading and false positive, most
of the times they are common sense and it is worthwhile to check them.

I will keep the data in sync with Naviserver evolution.

Maurizio


-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 08 October 2014 17:34
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] warning, shift count undefined behavior

On Wed, Oct 08, 2014 at 08:14:14AM +0200, Maurizio Martignano wrote:
 Dear Andrew,
   Yes, it is a potential issue.
 http://sonarsrv.spazioit.com:9000/dashboard/index?id=my%3Anaviserver%3
 Ansd%2

Neat!  Hm, it uses SonarQube, which I'd never heard of before.  What static
analyzer is ultimately being used to generate that info?

I couldn't seem to save a direct link to the shift/cast issue in
nsd/sockfile.c, but it is pretty easy to browse to it.

--
Andrew Piskorski a...@piskorski.com


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI
DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you
Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI
DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] Naviserver hangs on Windows

2014-10-05 Thread Maurizio Martignano
I believe you should use it.

Thank you,
Maurizio


-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 05 October 2014 19:47
To: naviserver-devel@lists.sourceforge.net
Cc: Maurizio Martignano
Subject: Re: [naviserver-devel] Naviserver hangs on Windows

On Sat, Oct 04, 2014 at 04:07:27PM +0200, Maurizio Martignano wrote:
   What is your target? Windows 32 or Windows 64?

Windows 7, 64-bit.

 Did you use the define _USE_32BIT_TIME_T yes or not?

No I did not.

-- 
Andrew Piskorski a...@piskorski.com


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] Naviserver hangs on Windows

2014-10-03 Thread Maurizio Martignano
Dear Andrew,
Did you happen to use this define _USE_32BIT_TIME_T?

Hope it helps,
Maurizio


-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 03 October 2014 14:49
To: naviserver-devel@lists.sourceforge.net
Subject: [naviserver-devel] Naviserver hangs on Windows

With my changes here, the core Naviserver compiles now on my Windows 7
64-bit machine:

  https://bitbucket.org/naviserver/naviserver/pull-requests
  https://bitbucket.org/apiskors/naviserver/commits/all

However, it doesn't work at all.  If I simply run nsd -h, it hangs
indefinitely, and locks up the Command Prompt window where I ran nsd.
Control-c does not abort, I have to close the window.

Note that I built Naviserver using ActiveTcl 8.5, I did not compile Tcl
myself.

I just started learning to use the WinDbg debugger.  Below is a stack trace.
This one is from attaching WinDbg to an already-running nsd.exe -h
process, but it looks the same if I intead tell WinDbg to run nsd.exe itself
(with no command line arguments) .

From the stack trace, Naviserver seems to be getting stuck when
Ns_GetTime (line 69 of time.c) calls TclpGetDate.  But, line 69 does not
explicitly call TclpGetDate, it justs manipulates a Tcl_Time tbuf structure
like so:

timePtr-sec = tbuf.sec; 

In Tcl 8.5.x, TclpGetDate is in win/tclWinTime.c, but the string
TclpGetDate does not occur anywhere in the Naviserver code, so I'm not
sure how it manages to call it.  A function pointer somewhere?

Any ideas what might be wrong here, or what else I should do to debug this
further?


Here the initial WinDbg output when attaching to a running nsd.exe -h
process, and then the stack trace (the k command):


*** wait with pending attach
Symbol search path is:
C:\web\nsd-atp\lib;http://msdl.microsoft.com/download/symbols
Executable search path is: 
ModLoad: 0001`3f08 0001`3f0fe000   C:\web\nsd-atp\bin\nsd.exe
ModLoad: `775e `77789000   C:\Windows\SYSTEM32\ntdll.dll
ModLoad: `773c `774df000
C:\Windows\system32\kernel32.dll
ModLoad: 07fe`fd43 07fe`fd49c000
C:\Windows\system32\KERNELBASE.dll
ModLoad: 07fe`f068 07fe`f0828000   C:\web\nsd-atp\bin\nsd.dll
ModLoad: 07fe`faab 07fe`fab4e000
C:\web\nsd-atp\bin\nsthread.dll
ModLoad: `1000 `100dc000   C:\P\Tcl85\bin\tcl85.dll
ModLoad: `774e `775da000
C:\Windows\system32\USER32.dll
ModLoad: 07fe`fdd4 07fe`fdda7000   C:\Windows\system32\GDI32.dll
ModLoad: 07fe`ff8e 07fe`ff8ee000   C:\Windows\system32\LPK.dll
ModLoad: 07fe`ff47 07fe`ff539000   C:\Windows\system32\USP10.dll
ModLoad: 07fe`ff2e 07fe`ff37f000
C:\Windows\system32\msvcrt.dll
ModLoad: 07fe`ff38 07fe`ff45b000
C:\Windows\system32\ADVAPI32.dll
ModLoad: 07fe`fdad 07fe`fdaef000
C:\Windows\SYSTEM32\sechost.dll
ModLoad: 07fe`ff78 07fe`ff8ad000
C:\Windows\system32\RPCRT4.dll
ModLoad: 07fe`fee0 07fe`fee4d000
C:\Windows\system32\WS2_32.dll
ModLoad: 07fe`ff46 07fe`ff468000   C:\Windows\system32\NSI.dll
ModLoad: 07fe`ff8b 07fe`ff8de000   C:\Windows\system32\IMM32.DLL
ModLoad: 07fe`fd9c 07fe`fdac9000   C:\Windows\system32\MSCTF.dll
Break-in sent, waiting 30 seconds...
WARNING: Break-in timed out, suspending.
 This is usually caused by another thread holding the loader lock
(e98.12c): Wake debugger - code 8007 (first chance)
ntdll!ZwWaitForSingleObject+0xa:
`776312fa c3  ret

0:000 kc
Call Site
ntdll!ZwWaitForSingleObject
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for
C:\P\Tcl85\bin\tcl85.dll - KERNELBASE!WaitForSingleObjectEx
*** WARNING: Unable to verify checksum for C:\web\nsd-atp\bin\nsthread.dll
tcl85!TclpGetDate [...]

0:000 k
Child-SP  RetAddr   Call Site
`0027eeb8 07fe`fd4310dc ntdll!ZwWaitForSingleObject+0xa
`0027eec0 `100a4766 KERNELBASE!WaitForSingleObjectEx+0x79
`0027ef60 07fe`faab4139 tcl85!TclpGetDate+0x5c2
`0027efc0 07fe`faab1bf6 nsthread!Ns_GetTime+0x29
[z:\web\ns-fork\naviserver\nsthread\time.c @ 69]
`0027f010 07fe`faab25f9 nsthread!Ns_MutexLock+0x76
[z:\web\ns-fork\naviserver\nsthread\mutex.c @ 210]
`0027f0d0 07fe`faab144a nsthread!Ns_CsEnter+0x69
[z:\web\ns-fork\naviserver\nsthread\cslock.c @ 172]
`0027f110 07fe`faab3db8 nsthread!Ns_MasterLock+0x2a
[z:\web\ns-fork\naviserver\nsthread\master.c @ 71]
`0027f140 07fe`faab2c58 nsthread!Ns_TlsAlloc+0x28
[z:\web\ns-fork\naviserver\nsthread\tls.c @ 77]
`0027f180 07fe`faab33d2 nsthread!NsInitReentrant+0x28
[z:\web\ns-fork\naviserver\nsthread\reentrant.c @ 65]
`0027f1b0 07fe`faab4514 nsthread!NsInitThreads+0x32
[z:\web\ns-fork\naviserver\nsthread\thread.c @ 105]
`0027f1e0 

Re: [naviserver-devel] Naviserver hangs on Windows

2014-10-03 Thread Maurizio Martignano
Well
I do not know...
What is your target? W32 or W64? What is your target machine? E.g. X64? or
AMD?
How do you link the DLL?

Maurizio


-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 03 October 2014 15:58
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] Naviserver hangs on Windows

On Fri, Oct 03, 2014 at 03:21:02PM +0200, Maurizio Martignano wrote:
 Dear Andrew,
   Did you happen to use this define _USE_32BIT_TIME_T?

No, I did not.  Should I?  Does Tcl use that?

Grepping my source files, I see this:

  ./include/nsconfig.h:276:#define SIZEOF_TIME_T 8

That nsconfig.h was created on Linux, but my Windows build uses it too.
Perhaps that is the problem?  But how the heck can I generate a correct
version of nsconfig.h for my Windows 7 system?

--
Andrew Piskorski a...@piskorski.com


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI
DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you
Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI
DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] Naviserver hangs on Windows

2014-10-03 Thread Maurizio Martignano
Dear Andrew,
These are the basic options I would use for a Windows 64 bit, AMD64:

COPTS   = /EHsc /W3 /nologo /c
DEFS= $(DEFS) /D _WINDOWS /D TCL_THREADS=1 /D WIN32 /D _WIN32 \
  /D FD_SETSIZE=128 /D NO_CONST=1 /D _MBCS # /D
_USE_32BIT_TIME_T 
LOPTS   = /NOLOGO /SUBSYSTEM:CONSOLE /OPT:NOREF /OPT:NOICF



Hope it helps,
Maurizio


-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 03 October 2014 15:58
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] Naviserver hangs on Windows

On Fri, Oct 03, 2014 at 03:21:02PM +0200, Maurizio Martignano wrote:
 Dear Andrew,
   Did you happen to use this define _USE_32BIT_TIME_T?

No, I did not.  Should I?  Does Tcl use that?

Grepping my source files, I see this:

  ./include/nsconfig.h:276:#define SIZEOF_TIME_T 8

That nsconfig.h was created on Linux, but my Windows build uses it too.
Perhaps that is the problem?  But how the heck can I generate a correct
version of nsconfig.h for my Windows 7 system?

--
Andrew Piskorski a...@piskorski.com


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI
DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you
Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI
DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] Naviserver hangs on Windows

2014-10-03 Thread Maurizio Martignano
The Windows-OpenACS distribution which I make available here
(http://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/) is based
on AOLServer 4.5.2, contains the sources, is compiled with Visual Studio
2013 and runs on Windows 64.
So if I where you I would give a look at that distribution and see how all
those time related functions have been handled.
That could save you some time.

Maurizio



-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 03 October 2014 16:49
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] Naviserver hangs on Windows

Hm, the old AOLserver 4.0.7 version of Ns_GetTime() had a Windows ifdef
which is entirely missing from Naviserver!  (See below.)

I found two relevant commits in Mercurial, from 2009 and 2012, below.
From those logs, it looks like what happened is that in 2009, Zoran
removed the platform specific C code and instead made Naviserver call
Tcl_GetTime (somehow).  Then in 2012, Gustaf added back the old
Unix-specific gettimeofday() implementation for performance reasons.

So should we just add back the old Windows-specific C implementation too?
Or is it better to use Zoran's way of invoking Tcl_GetTime?  Do we even know
for sure if that method really is platform-independent
as Zoran thought it was?  E.g., did Zoran or someone else at Archiware test
it on Windows?


Part of Ns_GetTime() from ancient AOLserver 4.0.7 sources:

#ifdef _WIN32
/*
 * Number of 100 nanosecond units from 1/1/1601 to 1/1/1970  */ #define
EPOCH_BIAS  1164447360i64
union {
unsigned __int64i;
FILETIMEs;
} ft;

GetSystemTimeAsFileTime(ft.s);
timePtr-sec = (time_t)((ft.i - EPOCH_BIAS) / 1000i64);
timePtr-usec =(long)((ft.i / 10i64) % 100i64); #else #endif


changeset:   2435:16508ab4b3de
user:Gustaf Neumann neum...@wu-wien.ac.at
date:Fri Nov 02 01:54:38 2012 +0100
files:   configure.in include/nsconfig.h.in nsthread/time.c
description:
use gettimeofday() if available instead of the rount trip to tcl

changeset:   2009:4bf1ee6ebf84
user:Zoran Vasiljevic z...@archiware.com
date:Sat Sep 29 14:14:03 2007 +0100
files:   ChangeLog nsproxy/nsproxylib.c nsthread/time.c
description:
Ns_GetTime now relies on Tcl_GetTime to be platform-independent

--
Andrew Piskorski a...@piskorski.com


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI
DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you
Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI
DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] no mkstemp() on Windows, use TclpCreateTempFile()?

2014-09-28 Thread Maurizio Martignano
Perfect,

 

   You eventually landed on using _mktemp_s which is what
described in the first link I provided you with and is what I was hoping for
Windows.

 

Thanks a lot,

Maurizio

 

 

From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 28 September 2014 16:56
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] no mkstemp() on Windows, use
TclpCreateTempFile()?

 

mktemp() is useful, but is in in the unix world general strongly 
discouraged in favor of mkstemp(). The problem with mktemp()  + file-create 
is, that in the time between the pathname is constructed and the file is
created,
another process might have created a file with the same name. mkstemp() is 
actually guaranteed to create a unique file. 

For the time being, i've commited a small implementation of mkstemp() for
windows,
that opens files with O_EXCL, which should be a first approximation.
However,
it's not tested, please test it and don't be surprised, if it needs more
tweaks
(maybe my flag settings are too restrictive).

all the best
-g

Am 27.09.14 16:11, schrieb Andrew Piskorski:

On Fri, Sep 26, 2014 at 08:55:37PM +0200, Maurizio Martignano wrote:

Dear Andrew and Gustaf,
   I believe this is the function you are looking for:
 
http://msdn.microsoft.com/en-us/library/ms235413.aspx

 
No, it is not.  mktemp and friends merely generate a file name, while
mkstemp also opens the file and returns a file handle.  So yes, you
could use mktemp as part of an implementation of mkstemp, but they do
are not replacements for each other.
 

 

--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] no mkstemp() on Windows, use TclpCreateTempFile()?

2014-09-26 Thread Maurizio Martignano
http://msdn.microsoft.com/en-us/library/ms235413.aspx

Maurizio


-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 26 September 2014 14:13
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] no mkstemp() on Windows, use
TclpCreateTempFile()?

Dear Andrew,

We have to be careful on licence mixes. The tcl8.5 version of
TclpCreateTempFile uses already mkstemp, but the windows implemetation looks
like a slim approach for a win32 implementation (aside of the TclFile
handling)  using
CreateFile* and the appropriate flags to make the file a temp file...

-g

On 26.09.14 02:51, Andrew Piskorski wrote:
 mkstemp() does not exist on Windows, and Naviserver uses it in three
 places:

$ find . -name *.[ch] -print | xargs grep -in mkstemp
./nsd/adpeval.c:1183:fd = mkstemp(debugfile);
./nsd/tclhttp.c:426:fd =
mkstemp(httpPtr-spoolFileName);
./nsd/driver.c:2301:sockPtr-tfd = mkstemp(sockPtr-tfile);

 We could stick in our own mkstemp() implementation.  Some starting 
 points for that may be:


http://stackoverflow.com/questions/6036227/mkstemp-implementation-for-win32
http://sourceforge.net/p/mingw/bugs/2003/

 But Tcl 8.4 and later already have TclpCreateTempFile() (two separate 
 implementations for Unix and Windows), which seems to solve the same 
 problem.  It returns a TclFile object, not a file descriptor.

 Can/should Naviserver's existing users of mkstemp() be modified to 
 instead work with TclpCreateTempFile()?  Or will that not quite work, 
 and we should borrow some of the Windows version of 
 TclpCreateTempFile() for direct use in Naviserver?




--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI
DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you
Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI
DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] no mkstemp() on Windows, use TclpCreateTempFile()?

2014-09-26 Thread Maurizio Martignano
Dear Andrew and Gustaf,
I believe this is the function you are looking for:

http://msdn.microsoft.com/en-us/library/ms235413.aspx

Hope it helps,
Maurizio


-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 26 September 2014 14:13
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] no mkstemp() on Windows, use
TclpCreateTempFile()?

Dear Andrew,

We have to be careful on licence mixes. The tcl8.5 version of
TclpCreateTempFile uses already mkstemp, but the windows implemetation looks
like a slim approach for a win32 implementation (aside of the TclFile
handling)  using
CreateFile* and the appropriate flags to make the file a temp file...

-g

On 26.09.14 02:51, Andrew Piskorski wrote:
 mkstemp() does not exist on Windows, and Naviserver uses it in three
 places:

$ find . -name *.[ch] -print | xargs grep -in mkstemp
./nsd/adpeval.c:1183:fd = mkstemp(debugfile);
./nsd/tclhttp.c:426:fd =
mkstemp(httpPtr-spoolFileName);
./nsd/driver.c:2301:sockPtr-tfd = mkstemp(sockPtr-tfile);

 We could stick in our own mkstemp() implementation.  Some starting 
 points for that may be:


http://stackoverflow.com/questions/6036227/mkstemp-implementation-for-win32
http://sourceforge.net/p/mingw/bugs/2003/

 But Tcl 8.4 and later already have TclpCreateTempFile() (two separate 
 implementations for Unix and Windows), which seems to solve the same 
 problem.  It returns a TclFile object, not a file descriptor.

 Can/should Naviserver's existing users of mkstemp() be modified to 
 instead work with TclpCreateTempFile()?  Or will that not quite work, 
 and we should borrow some of the Windows version of 
 TclpCreateTempFile() for direct use in Naviserver?




--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI
DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you
Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI
DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] pid_t missing on Windows

2014-09-18 Thread Maurizio Martignano
Dear Gustav,
This is ok.

I still do not quite understand/explain why the nsthread module is so
different between Aolserver and Naviserver. And again as the two pieces of
software are implemented at the moment, the Aolserver version seems to be
more portable.

Thank you,
Maurizio

-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 18 September 2014 13:51
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] pid_t missing on Windows


historically, unix used int for pids. the open group / ieee standard
defines pid_t  [1],  which is used e.g. for fork(), getpid(),
kill() etc.
on linux, mac os x etc. Therefore, the use of pid_t instead of int
is a good thing, especially, since pids are passed around as arguments and
returned by naviserver functions.

for keeping the code portable, the best thing is to define in naviserver
pid_t  as well for windows, but using HANDLE.

I've changed right now various places in naviserver, where pids are assumed
as int, and added a definition for pid_t for win as well. everything
compiles still nicely under unix, i have not testes with windows, but things
should have become there better as well. Please get an update from
bitbucket.

-g

[1] http://pubs.opengroup.org/onlinepubs/007904975/toc.htm

Am 18.09.14 01:26, schrieb Andrew Piskorski:
 On Windows with the VC 2012 compiler I get this error, which seems to 
 then break a lot of other important stuff downstream:

thread.c(55) : error C2061: syntax error : identifier 'pid_t'

 Is pid_t defined anywhere for Windows?  AFAICT it is not.  Adding this 
 in an appropriate spot in nsthread.h makes the error go away:

#ifndef pid_t
typedef int pid_t;
#endif

 But I don't know whether that's actually a correct fix for the various 
 places that use pid_t.  Interestingly, in nsd/exec.c Ns_WaitForProcess 
 uses pid_t ONLY on Unix, and has a completely different implementation 
 for Windows, which uses a HANDLE type instead of pid_t.  And 
 winthread.c definitely uses HANDLE.  So it looks like use of the 
 Unix-only pid_t crept into a few of the nominally cross-platform files 
 over time.  But what's the correct fix?



--
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce Perforce version control.
Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Slashdot TV.  Video for Nerds.  Stuff that Matters.
http://pubads.g.doubleclick.net/gampad/clk?id=160591471iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] pid_t missing on Windows

2014-09-17 Thread Maurizio Martignano
Dear Andrew,

Handles in windows are void pointers. 32 bit void pointers in Windows 32 and
64 bit void pointers in Windows 64 (obviously). However in WIN64 only the
less significant 32 bits of a handle are used for compatibility and
interoperability reasons.
So having
typedef int pid_t;
or
typedef HANDLE pid_t;
it is not quite the same.

Frankly what puzzles me is the huge code difference between the Aolserver
and Naviserver versions of this file (thread.c), and in general of all the
nsthread module.

I also have the feeling that the the Aolserver version is less OS dependent
and more portable.

Well yes, Naviserver has been ported on Windows but with MinGW and quite
likely only in Win32. I suspect that many other problems will show up in
Windows 64 and I sincerely hope you are concentrating your efforts on this
platform (Windows 64).

Hope this helps and thank you for your work,
Maurizio
 

-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 18 September 2014 01:26
To: naviserver-devel@lists.sourceforge.net
Subject: [naviserver-devel] pid_t missing on Windows

On Windows with the VC 2012 compiler I get this error, which seems to then
break a lot of other important stuff downstream:

  thread.c(55) : error C2061: syntax error : identifier 'pid_t'

Is pid_t defined anywhere for Windows?  AFAICT it is not.  Adding this in an
appropriate spot in nsthread.h makes the error go away:

  #ifndef pid_t
  typedef int pid_t;
  #endif 

But I don't know whether that's actually a correct fix for the various
places that use pid_t.  Interestingly, in nsd/exec.c Ns_WaitForProcess uses
pid_t ONLY on Unix, and has a completely different implementation for
Windows, which uses a HANDLE type instead of pid_t.  And winthread.c
definitely uses HANDLE.  So it looks like use of the Unix-only pid_t crept
into a few of the nominally cross-platform files over time.  But what's the
correct fix?

--
Andrew Piskorski a...@piskorski.com


--
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce Perforce version control.
Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191iu=/4140/ostg.clktrk
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] Windows 7 recommended compiler and build tools?

2014-08-25 Thread Maurizio Martignano
Going 32 bit nowadays is a no go. Full stop.

Wish you luck.

Maurizio


-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 25 August 2014 19:46
To: naviserver-devel@lists.sourceforge.net
Cc: Maurizio Martignano
Subject: Re: [naviserver-devel] Windows 7 recommended compiler and build
tools?

On Sun, Aug 10, 2014 at 05:18:57PM +0200, Maurizio Martignano wrote:

 As build environment I would select your option #2, that is  
 Microsoft's compiler, using nmake from the command-line.

So, just how are you folks actually making that work?

I imagine the build process should be something roughly along these lines:

  C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd /Debug /x64
/win7
  ./configure --enable-symbols --with-tcl=C:/P/Tcl85/lib
--prefix=C:/web/nsd499-dtk
  nmake -f Makefile all 

But Naviserver's configure is of course a Bourne shell script and cannot
run from the Windows command prompt.  So what should I do instead?  Install
Msys just to run configure?  Or perhaps skip configure and just pass various
options to nmake?  Is the Makefile that ships with Naviserver even known to
work on Windows with nmake and the MS SDK compiler?

I do NOT have 64-bit zlib installed yet.  Hopefully I can build without it,
and just turn the zlib features in Naviserver?

Btw, I have ActiveTcl (which is threaded) installed in C:\P\Tcl85\, so I
figured I'd try linking against that first before compiling my own Tcl.

The stock Tcl win/README file says that they build with Visual C++
6.0 and the TEA Makefile, but VC++ 6.0 is 32-bit only, so the ActiveState
folks must be doing something slightly different for their 64-bit builds.
Anybody know offhand what they really use or where their 64-bit Windows
build scripts are located?

--
Andrew Piskorski a...@piskorski.com


--
Slashdot TV.  
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] Windows 7 recommended compiler and build tools?

2014-08-10 Thread Maurizio Martignano
Dear all,
Using different C/C++ compilers for different modules of the same
binary/executable is always a bad idea and not only in Windows.

Different compilers can be used to create different binaries, not really
integrated with each other's. In the Windows-OpenACS distribution
(http://www.spazioit.com/pages_en/sol_inf_en/windows-openacs_en/) the core
system (i.e. Aolserver and its modules, Tcl/Tk, xotcl, Thread,  are all
built with one single compiler (Microsoft Visual Studio 2013 Professional
Editions). All other tools, utilities are built with what is more
convenient, PostgreSQL 8.3.23 binaries with MinGW and PostgreSQL 9.3.1 with
MinGW64.

As build environment I would select your option #2, that is  Microsoft's
compiler, using nmake from the command-line.

In terms of target I would use at the moment Windows Server 2012 R2. Windows
7 is rather peculiar, but I do not know the actual user requirements.

Hope this helps,
Maurizio Martignano

-Original Message-
From: Andrew Piskorski [mailto:a...@piskorski.com] 
Sent: 10 August 2014 12:46
To: naviserver-devel@lists.sourceforge.net
Subject: [naviserver-devel] Windows 7 recommended compiler and build tools?

I just started looking into running NaviServer 4.99.6 (the latest
release) on Windows 7 (64-bit), and am trying to figure out the proper
approach to building it (plus my own custom code) from source.

From prior experience with ancient versions of Tcl and AOLserver, I
know of at least three build approaches on Microsoft Windows:

1. Microsoft Visual Studio, using (brittle, ugly) GUI-based project
   files to control the build.
2. Microsoft's compiler, using nmake from the command-line.
3. Msys + MinGW, using Gnu make.

Which of those do you recommend for NaviServer?  Also, do I need to use the
same compiler for everything, or is it feasible to say build NaviServer with
MinGW, but Tcl or my own custom code with Microsoft's
2013 compiler?

In NaviServer 4.99.6, the README gives a download link for msys_mingw8.zip,
which dates from c. 2004.  Is that old link just an oversight, or is there
some reason I should use such an ancient version of MinGW rather than a
current one?

In the win and win32 subdirectories, I see *.sln and *.dsw files for use
with Visual Studio 12.0.  Do those work better/worse than MinGW?
Also, is the command-line make-based build infrastructure set up to work
with Microsoft's compilers, or is it only for use with MinGW?

Does MinGW still only support 32-bit, and for 64-bit I need to instead use
the MinGW-w64 fork?  Since MinGW-w64 handles both 32 and 64 bit, is there
any reason to prefer the original MinGW over it?

Is there any reason I should stick with 32-bit NaviServer, Tcl, etc. on
Windows 7, rather than moving to 64-bit?

Thanks in advance for your advice and tips!

Btw, Microsoft's current compilers are apparently available free of charge,
but just figuring out what you actually need to download and install is
non-trivial.  I found this advice from Jan. 2012 very
helpful:

  http://www.willus.com/ccomp.shtml?p11

And here's the Microsoft stuff I actually downloaded and installed:

- Windows SDK 7.1 for Windows 7 and .NET Framework 4:
  Rather than using the online installer (although it would work fine):
http://www.microsoft.com/en-us/download/details.aspx?id=8279
  I downloaded the GRMSDKX_EN_DVD.iso x86-64:
http://www.microsoft.com/download/en/details.aspx?id=8442
 
http://download.microsoft.com/download/F/1/0/F10113F5-B750-4969-A255-274341A
C6BCE/GRMSDKX_EN_DVD.iso
  As far as I could tell that's still the newest version of the SDK.
  From the Release Notes:
http://go.microsoft.com/fwlink/?LinkID=186826
  it's clear that it includes both the Visual C++ 2010 compilers for both 32
and 64 bit
  (the same compilers that ship with Visual Studio 2010),
  the MSBuild 4.0 command line build environment, etc.

- The SDK does not ship with the necessary .NET Framework:
http://msdn.microsoft.com/en-us/vstudio/aa496123
  So I used the web installer to install .Net Framework 4.5.1 from
2013-10-12:
http://www.microsoft.com/en-us/download/details.aspx?id=40773
  NOT the older .Net Framework 4 from 2011-02-21:
http://www.microsoft.com/download/en/details.aspx?id=17851

- VS Express 2013 for Windows Desktop:
  To get the GUI, I also installed the Express for Desktop version of
  Visual Studio, via the VS2013_RTM_DskExp_ENU.iso file:
 
http://www.visualstudio.com/downloads/download-visual-studio-vs#d-express-wi
ndows-desktop
http://www.visualstudio.com/en-US/products/visual-studio-express-vs

--
Andrew Piskorski a...@piskorski.com


--
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] naviserver with connection thread queue

2012-11-29 Thread Maurizio Martignano
This is good engineering work. I rise my hat to that.

Congrats,

Maurizio

 

 

From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 29 November 2012 19:51
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] naviserver with connection thread queue

 

Dear all,

one more update. Here is now the data of a full day with the Async-Writer
thread. The avg queueing time dropped from 0,0547 to 0,0009, the standard
deviation from 0,7481 to 0,0211. This is a noticable improvement.

accept  queue   run total
avg 0,  0,0009  0,1304  0,1313
stdev   0,  0,0211  0,9022  0,9027
min 0,  0,  0,0032  0,0033
max 0,  0,9772  28,1699 28,1700
median  0,  0,0001  0,0208  0,0209
 

But still, the sometime large values are worth to investigate. It turned out
that the large queueing time came from requests from taipeh, which contained
several 404 errors. The size of the 404 request is 727 bytes, and therefore
under the writersize, which was configured as 1000. The delivery of an error
message takes to this site more than a second. Funny enough, the delivery of
the error message blocked the connection thread longer than the delivery of
the image when it is above the writersize.

I will reduce the writersize further, but still a slow delivery can even
slow down the delivery of the headers, which happens still in the connection
thread. Most likely, i won't address this now.  however, i will check the
effects of fastpath cache, which was deactivated so far...

best regards
-gustaf neumann

Am 28.11.12 11:38, schrieb Gustaf Neumann:

Dear all,

here is a short update of the findings and changes since last week. 

One can now activate for nslog logpartialtimes, which adds an entry to the
access log containing the partial request times (accept time, queuing time,
and run time). The sample site runs now with minthreads == 5, maxthreads 10.

If one analyzes the data of one day of our sample site, one can see that in
this case, the accept time is always 0 (this is caused by nsssl, which is a
sync driver, header parsing happens solely in the connection thread), the
queue time is on average 54 ms (milliseconds), which is huge, while the
median is 0.1 ms. The reason is that we can see a maximum queue time of 21
secs! As a consequence, the standard deviation is as well huge. When one
looks more closely to the log file, one can see that at times, when the
queuing time is huge, as well the runtime is often huge, leading to
cascading effects as described below. cause and reason are hard to
determine, since we saw large runtimes even on delivery of rather small
times

  accept  queue   run total
avg  0,  0,0547  0,2656  0,3203
stdev0,  0,7481  0,9740  1,4681
min  0,  0,  0,0134  0,0135
max  0,  21,9594 16,6905 29,1668
median   0,  0,0001  0,0329  0,0330
 

The causes for the random slow cases are at least from two sources: slow
delivery (some clients are slow in retrieving data, especially some bots
seems to be bad) and system specific latencies (e.g. backup running, other
servers becoming active, etc.). To address this problem, i moved more file
deliveries to the writer thread (reducing writersize) and activated
TCP_DEFER_ACCEPT. One sees significant improvements, the average queuing
time is 4.6 ms, but still the standard deviation is huge, caused by some
erratic some large times.

  accept  queue   run total
avg  0,  0,0046  0,1538  0,1583
stdev0,  0,0934  0,6868  0,6967
min  0,  0,  0,0137  0,0138
max  0,  4,6041  22,4101 22,4102
median   0,  0,0001  0,0216  0,0217
 

There are still some huge values, and the queuing time was lucky not to be
influenced by the still large runtimes of requests. Aside of the top values,
one sees sometimes unexplained delays of a few seconds. which seem to block
everything.

A large source for latencies is the file system. On our production system we
already invested some efforts to tune the file system in the past, such we
don't see effects as large as on this sample site. Since NaviServer should
work well also on non-tuned file systems, i implemented an async writer
thread, that decouples writing to files (access.log and error.log) from the
connection threads. The interface is the same as unix write(), but does not
block.

I have this running since yesterday evening on the sample site, and do far,
it seems to help to reduce the random latencies significantly. i'll wait
with posting data until i have a similar time range to compare, maybe the
good times i am seeing are accidentally.

I have not yet looked into the details for handling the async writer withing
logrotate and shutdown.

On another front, i was experimenting with jemalloc and tcmalloc.
Previously we had the following figures, using tcl as distributed with zippy
malloc

  with minthreads=2, maxthreads=2
  requests 10307 spools 1280 queued 2704 connthreads 11 rss 425

When changing 

Re: [naviserver-devel] code cleanup

2012-10-26 Thread Maurizio Martignano
Dear all,
I do apologize for my inactivity but some health problem has
distracted me from the computer. Hopefully I should be able to come back
fully operative pretty soon.

First of all I would like to thank Gustav for his work.

Secondly, it seems to me there are two major activities going on at the
moment, I do not know how to call them:

1. call cleanup / consolidation  - I would call this baseline
2. discussion/work on new features- I would call this new developments

Perhaps we should extend the baseline activity to all modules that could be
possibly needed by an OpenACS application, e.g. on top the standard ones
also modules like nsoracle, nsopenssl, nsldap and so on what do you
think? 

Wouldn't also make sense to work first on this consolidated and extended
baseline, before actually concentrating/working on new developments?

Thanks a lot,
Maurizio



-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 26 October 2012 21:12
To: Navidevel
Subject: [naviserver-devel] code cleanup

Dear friends,

Over the last days i did some cleanup that i have just committed with one
changeset to bitbucket. With these changes, naviserver still compiles with
clang -Wall -pedantic, and passes all regression tests. Furthermore, it
compiles but as well with Microsoft Visual Studio 11 without warning on most
files (still things to do, such as bind for udp sockets, pread emulation,
read-operations in the spooler, ...; just did nsd 
+ nsthread). Most likely the visual studio port needs more
still more work, i did not manage to link with whatever
tcl8.5.12 produces (it looks for a tcl84t.lib, which is not produced per
default from tcl, most likely, one needs more setup-magic from Maurizio). I
have used Maurizio's project files, and added essentially the flags
_CRT_SECURE_NO_WARNINGS and _CRT_NONSTDC_NO_DEPRECATE.

The huge number of casts from Maurizio's first attempt are not needed (e.g.
assigning void* to a typed pointer, such as the results of the *alloc*
functions. However, it complained about many of the size_t assignments,
which convinced me to proceed this way.

The changes are:

- use size_t, ssize_t, off_t wherever appropriate (sometimes dangerous, when
negative values are used for sizes, or  when e.g. Tcl_GetWideIntFromObj() is
used; i tried to be conservative in such cases)

- push casts to int as far as possible down the code (e.g. 
down to Tcl_* functions accepting sizes, but defined with int, such as
Tcl_WriteChars, Tcl_NewStringObj,
Tcl_NewByteArrayObj)

- be more careful on redefining macros (MSC seems a moving
target)

- provide base definition for inttypes for MSC WIN32 and WIN64

- use NS_SOCKET instead of SOCKET (which is defined under Windows). We
should stick with macros in our own namespace whenever possible.

- added the project files and the solution file for visual studio (without
the SQL Server file containing the configuration state, which is 35MB)

As far i can tell, everything works fine under Linux and Mac OS X, but i
certainly can't exclude some sign mismatches on seldomly used paths.

-gustaf neumann



--
The Windows 8 Center
In partnership with Sourceforge
Your idea - your app - 30 days. Get started!
http://windows8center.sourceforge.net/
what-html-developers-need-to-know-about-coding-windows-8-metro-style-apps/
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
WINDOWS 8 is here. 
Millions of people.  Your app in 30 days.
Visit The Windows 8 Center at Sourceforge for all your go to resources.
http://windows8center.sourceforge.net/
join-generation-app-and-make-money-coding-fast/
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] [AOLSERVER] Naviserver Win-64 Sources

2012-10-17 Thread Maurizio Martignano
OK

Simple reason: Visual Studio complains about that stuff and it is annoying.
I changed that everywhere. It doesn't do any arm, and for you it is free.
Why you just do not accept it?

And the same goes for the keywords new and delete. On the contrary the
keyword bool as I already mentioned interferes with macro definition on
bool in the C201X STD.

The KR STD function style was not only in the file I mentioned, but as I
mentioned is present also one or two more places in the codebase, so feel
free to check my archive and do a diff.

If all this C/C++ compatibility is so crazy and non-sense how come
there's an option in gcc checking for a common subset? 

I do not know if you are aware of the MISRA-C rules (Guidelines for the use
of C language in critical systems - http://en.wikipedia.org/wiki/MISRA_C); I
do cause I work on this stuff - my bread and butter.
Well in the 1998 version of this standard there was this rule:
Rule 44 - Redundant explicit casts should be not used.
But in the 2004 version of the standard this rule has been rescinded, not
by me, but by the authors of the standard, because for some people it was
controversial and for some others it did not make any sense.

Anyhow. All of this in not really important. I do not want to be annoyed by
Visual Studio. I will use my version of the codebase. Want to use it? Please
do, you are very welcome. Want to keep the original version, it is ok too.

Hope it helps,
Maurizio


-Original Message-
From: Jeff Rogers [mailto:dv...@diphi.com] 
Sent: 17 October 2012 07:26
To: Maurizio Martignano
Cc: naviserver-devel@lists.sourceforge.net
Subject: Re: [AOLSERVER] Naviserver Win-64 Sources

Hi Maurizio,

I appreciate your efforts on the windows portability and build front; but on
the issue of c++ acceptability, I think you are trying to do something that
really doesn't need doing.

naviserver/aolserver is written in C.  C is not C++.  C++ is not new C 
or a better C - it is C++.  Making the code look a little more palatable
to programmers who know C++ but not C is asking for more trouble.  I would
expect an enthusiastic young C++ programmer to see all those casts from
ns_malloc and ask why is this using old-fashioned 
casts and malloc, when it should be using type-safe new?   And the 
answer is that C is not C++; in ansi C casting from a void * to another data
pointer type is explicitly a safe operation.  The cast is unnecessary noise.
Many C++ programmers will tell you that casts are a sign of poor design.  In
C++ that may be true. C is not C++.

Similarly, aolserver/naviserver is not hello.c.  I don't mean to turn away
new programmers - on the contrary, I think the code is very clean and
understandable, but it is still complicated;  if a programmer is going to
get confused by seeing new as a variable name instead of a reserved word
then they're going to be utterly baffled trying to understand race
conditions in rapid thread creation or the lifecycle of conn threads.

So what this really amounts to is a suggestion to change the coding
standards and build/warning flags for the project.  That is a valid topic of
discussion.  I personally don't think they need changing, but I speak for me
and not for anyone else involved, and as I'm sure you're well aware coding
standards are things that holy wars are made of so there's likely to be no
shortage of opinion on it.  If you wish to change the coding standards, it
should be brought up as such, and not under the guise of windows
portability, which it has nothing to do with.

-J

PS: re: tclxkeylist.c - ok, perhaps not so surprising there.  I suspect that
code was imported verbatim around 1998 and updated very little since then.
I wonder how much this functionality is still used, or if it would make
sense to move it into a separate loadable module (could users just use a
complete tclx, or would that interact badly with signal handling or other
bits?), since I would expect most new code to use dict instead.


Maurizio Martignano wrote:
 Dear Jeff,
   Thank you very much for your remarks. I would like to share what I 
 think.

 2.a
 Of course we are talking about C files and not C++ files, so any 
 compiler will accept the sources as they are (at least till now). But 
 the point here I believe is making the codebase easy to read and 
 maintain. Nowadays programmers (unlike  us I am afraid - I am 52 years 
 old) are used to C++ more than C; so trying to use the common subset 
 between ISO C e C++ standard make sense. If I take gcc and I compile 
 the code with the option -Wc++-compat I get a warning for every ISO 
 C construct that is outside of the common subset of ISO C and ISO C++, 
 e.g. request for implicit conversion from void * to a pointer to non-void
type.
 So that is an effort in making the code easily readable by programmers 
 (not
 compilers) with C++ background.

 2.b
 Shocking? Have a look at the file tclXkeylist.c... The same 
 construct is used  here and there in other places (look at the diffs

Re: [naviserver-devel] [AOLSERVER] Naviserver Win-64 Sources

2012-10-17 Thread Maurizio Martignano
Dear Gustav,
Thank you for your mail message.

My answer here below.

Maurizio


-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 17 October 2012 11:47
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] [AOLSERVER] Naviserver Win-64 Sources

Maurizio,

the changes you have done are textually quite large, the patch is 6000 lines
+.
Please, don't expect, we can apply this and forget about it, we need some
time to digest this. Please, don't be impatient, we all have limited time.

For some problems, it seems for me to be better to follow the tcl
conventions, since it is not unlikely, that who ever works on the details of
naviserver will have to deal with the tcl sources as well.

What it makes this also more time consuming is that some of the changes
should be probably done differently, like the cases described below that i
have picked randomly. This list is most like not complete.

all the best
-gustaf neumann

+#ifdef _WIN64
+Ns_DStringPrintf(dsPtr, %d %d %d %l64d %l64d %l64d
%l64d,
+#else
  Ns_DStringPrintf(dsPtr, %d %d %d % PRId64  % 
PRId64  % PRId64  % PRId64,
+#endif


question arise: is this just _WIN64 or as well _WIN32?
Why not define PRId64 correctly for _WIN64? (and _WIN32?)

[MM] Because the PRId64 thing works with gcc and not with Visual Studio (or
to be correct I did not manage to make it work). I believe it works if you
use MinGW, so this is why I used only _WIN64


Or, why have you commented out certain code (a) for win64 and (b) for all
platforms which is certain useful:

  #ifdef NS_NOCOMPAT
-#  error No compatibility macros at present
+// #  error No compatibility macros at present
  #endif

This is my error, sorry, should be under _WIN64

...

  #ifdef HAVE_TCL_GETMEMORYINFO
-Tcl_GetMemoryInfo(ds);
-Tcl_DStringResult(interp, ds);
+// Tcl_GetMemoryInfo(ds);
+// Tcl_DStringResult(interp, ds);
  #endif
  return TCL_OK;

same as above under _WIN64

another problem is that there are some fixes included, which have nothing to
do with
_WIN64 and which are fixed already in the tip version on bitbucket.

--- /usr/local/src/naviserver-4.99.4/nsd/compress.c
2010-03-07 17:08:38.0 +0100
+++ ./nsd/compress.c2012-10-13 10:03:54.0 +0200
@@ -299,7 +299,11 @@
  return;
  }

+#ifdef _WIN64
+int
+#else
  char *
+#endif
  Ns_CompressBufsGzip(Ns_CompressStream *stream, struct iovec *bufs, int
nbufs, int flags,
  Ns_DString *dsPtr)

[MM]
That looked to me as an error, but did not dare to correct it. Just correct
the fix and take away the _WIN64 stuff if you believe it is right


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for
free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


Re: [naviserver-devel] [AOLSERVER] Naviserver Win-64 Sources

2012-10-17 Thread Maurizio Martignano
1.Macros are defined.
2. What is not accepted is the starting with multiple format strings, e.g.
%d %d %d % - 1st string PRId64 - 2nd string  % --- and so on




-Original Message-
From: Gustaf Neumann [mailto:neum...@wu.ac.at] 
Sent: 17 October 2012 14:06
To: naviserver-devel@lists.sourceforge.net
Subject: Re: [naviserver-devel] [AOLSERVER] Naviserver Win-64 Sources

On 17.10.12 12:35, Maurizio Martignano wrote:

 +#ifdef _WIN64
 +Ns_DStringPrintf(dsPtr, %d %d %d %l64d %l64d %l64d
 %l64d,
 +#else
Ns_DStringPrintf(dsPtr, %d %d %d % PRId64  %
 PRId64  % PRId64  % PRId64,
 +#endif


 question arise: is this just _WIN64 or as well _WIN32?
 Why not define PRId64 correctly for _WIN64? (and _WIN32?)

 [MM] Because the PRId64 thing works with gcc and not with Visual 
 Studio (or to be correct I did not manage to make it work). I believe 
 it works if you use MinGW, so this is why I used only _WIN64
PRId64 has nothing to do with gcc, it is supposed to be defined by
inttypes.h. From googling around it seems that these macros are often a
problem with visual c, not only a problem for _WIN64.

 From your changes, i deduce, that PRId64 and PRIuMAX are not defined  for
your platform, but PRIu64 seems to be there (at least, you did not change
this).  Please check, if the following change fixes the issues with the PRI*
macros under your platform
https://bitbucket.org/naviserver/naviserver/changeset/92877ff5b9625aac023c91
1dcd862894cfd441ce

 Or, why have you commented out certain code (a) for win64 and (b) for 
 all platforms which is certain useful:

#ifdef NS_NOCOMPAT
 -#  error No compatibility macros at present
 +// #  error No compatibility macros at present
#endif

 This is my error, sorry, should be under _WIN64

 ...

Is anybody aware about NS_NOCOMPAT ?
This seem to stem from the win32 project files.
Can this be removed?


#ifdef HAVE_TCL_GETMEMORYINFO
 -Tcl_GetMemoryInfo(ds);
 -Tcl_DStringResult(interp, ds);
 +// Tcl_GetMemoryInfo(ds);
 +// Tcl_DStringResult(interp, ds);
#endif
return TCL_OK;

 same as above under _WIN64
Why is this a problem with _WIN64?

-gustaf neumann


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics Download AppDynamics Lite for
free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
naviserver-devel mailing list
naviserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/naviserver-devel