Unfortunately, these things don't show up on the mac log.
Robert.
On Wednesday, January 29, 2003, at 04:16 AM, Web Dude wrote:
Not sure about your log files, but in Windows if you pay attention to
the threads size on the tafs that are running, you may find the
problem. I have found corrupt tafs this way. Threads in the hundreds
of thousands for an execution will indicate a loop or a file in a
corrupt state. Also look for any KILLS as in:
KILLED E:blahblah/whatever/searchcustomers.taf
Jeff,
You might get a clue on the last TAF served by corresponding the web
server log file with the Witango log file. That will always keep
logging, even when the daemon dies. If you can look at a few
instances you might build a case for common culprits to find
specific files that could be causing the crash, as Robert mentioned.
Jason
On Tuesday, January 28, 2003, at 10:24 PM, Robert Garcia wrote:
Before you troubleshoot further, you have to determine if you have a
specific method/taf that is causing the problem. Unfortunately the
OS X logging doesn't work as well as on the windows version.
Robert
On Tuesday, January 28, 2003, at 06:17 PM, Jeffrey Bohmer wrote:
The load is real traffic. The witangoevents.log does not specify a
taf,
rather it says things like:
[20794] 2003-01-28 10:23:02 RUNTIME INFO Started accepting
user
requests
[20794] 2003-01-28 10:50:08 RUNTIME FATAL Caught fatal signal
SIGSEGV; thread id = 3801008
[21067] 2003-01-28 10:51:00 START INFO Witango daemon
started
The Witango.log does not contain anything about the crash. It also
never
has a CRLF on the last line written, so the startup message does
not appear
on it's own line.
The server actually crashes and the witangod process dies. A
watcher
script is called by cron every minute to check on the server and
restart it
if not running.
I've set THREADPOOLSIZE=10 and LOGGINGLEVEL=1. When I get a chance
to
restart the server, I'll watch it and see what happens.
BTW, thanks for all your help!,
- Jeff
At 05:36 PM 1/28/03 -0800, you wrote:
What are using to simulate the load? Are you hitting the same taf
over
and over? Are you monitoring real traffic? If you are monitoring
realtraffic, does the witangoevents.log tell you which taf the app
server crashed on? When it crashes, is it actually crashing,
producing
a crash log, or just hanging indefinitely?
Robert.
On Tuesday, January 28, 2003, at 12:57 PM, Jeffrey Bohmer wrote:
I've been monitoring the system with top -u for a while.
witangod's
threads are always 3 above THREADPOOLSIZE. CPU usage for
witangod is
usually 90 - 110% when the site is getting hits. When no hits are
coming in, it drops out of the list. (I assume CPU usage can go
above
100% because this is an SMP machine.)
I will try out OpenLink's iODBC manager, if necessary. But I'd
like
to try what I can with my current setup first.
- Jeff
The threadpoolsize acts different than you think it would. I have
done alot of testing, and the more you add, the slower, and more
buggy the server gets. It is worse on the mac than windows, but
occurs on both. Try setting the threadpoolsize to 10. Also,
before
you test, open the terminal, and enter:
top -u 5
Then leave this running while under load. Monitor the CPU usage
of
the witango process, and also check the number of threads. It
should
be 2 or 3 above the threadpool size. Let me know what you see.
Robert.
On Tuesday, January 28, 2003, at 11:58 AM, Jeffrey Bohmer wrote:
For a few minutes before the crashes, the app server gets about
35-45 requests per minute. Other hardware/config info:
Dual 1.25GHz G4
1.75 GB memory
PostgreSQL has it's own drives for data & logs.
THREADPOOLSIZE=25
LOGGINGLEVEL=3
All caches/buffers are quite large and no swapping occurs.
Is THREADPOOLSIZE too low for this traffic?
Also, I have logging turned up to see if a particular piece of
code
is being executed before the crash (or some kind of pattern).
But
if the last part of the log is lost on a crash, then I won't
get
that info and could turn logging back down to 1.
- Jeff
How heavy a load? I have tested it under heavy loads, and It
did
well, where it used to crash. There is still a point where it
crashes when absolutely hammered, but it seems to be beyond
the
point of reasonable expectation. If the cache is off, it will
crash under a lighter load. Looking at your setup, I would
try
the iODBC from openlink. You can get it from my sight if you
wist
at http://www.theradmac.com/
Robert Garcia
On Tuesday, January 28, 2003, at 11:23 AM, Jeffrey Bohmer
wrote:
TRUE for both. (The CACHESIZE is about 20MB.)
- Jeff
What are the:
CACHE=
CACHEINCLUDEFILES=
variables set to in the witango.ini file?
Robert Garcia
On Tuesday, January 28, 2003, at 08:49 AM, Jeffrey Bohmer
wrote:
The latest production app server on OS X crashes every so
often,
when traffic is fairly high. The app server's FATAL
signal is
either a bus error or segmentation violation, usually the
latter.
If anyone has any ideas as to why this might happen, or
things
to try to prevent it, I would be happy to receive your
insight.
I wonder if some code is causing the crash. But I believe
the
way the Witango.log is written to disk makes this
difficult
to
determine. It seems the app server delays writes to
Witango.log.
I bet the last bit of log info is lost when a crash occurs.
Therefore, I don't know what was happening just before a
crash.
Am I correct in my assumptions here?
FYI ... in October, I added a bug report about this at
developer.witango.com. The beta server had this problem
more
often
that the production server, but it's still a problem. Bug
Track
shows an empty status for this report (and another bug
report I
added).
Witango App Server 5.0.1.054
PostgreSQL 7.3.1
OS X Server 10.2.3
stock iODBC 2.1.6
stock apache 1.3.27
Thank you,
- Jeff
-- Jeff Bohmer
VisionLink, Inc.
_________________________________
303.402.0170
www.visionlink.org
_________________________________
People. Tools. Change. Community.
____________________________________________________________
__ ___
__ __ __ _
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message
body
--
Robert Garcia
BigHead Technology
2781 N Carlmont Pl
Simi Valley, CA 93065
Phone 805.522.8577
http://www.bighead.net/
http://www.theradmac.com/
[EMAIL PROTECTED]
_____________________________________________________________
__ ___
__ __ __
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
Jeff Bohmer
VisionLink, Inc.
_________________________________
303.402.0170
www.visionlink.org
_________________________________
People. Tools. Change. Community.
______________________________________________________________
__ ___
__ __ _
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message
body
--
Robert Garcia
BigHead Technology
2781 N Carlmont Pl
Simi Valley, CA 93065
Phone 805.522.8577
http://www.bighead.net/
http://www.theradmac.com/
[EMAIL PROTECTED]
_______________________________________________________________
__ ___
__ __
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
Jeff Bohmer
VisionLink, Inc.
_________________________________
303.402.0170
www.visionlink.org
_________________________________
People. Tools. Change. Community.
________________________________________________________________
__ ___
__ _
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
Robert Garcia
BigHead Technology
2781 N Carlmont Pl
Simi Valley, CA 93065
Phone 805.522.8577
http://www.bighead.net/
http://www.theradmac.com/
[EMAIL PROTECTED]
_________________________________________________________________
__ ___
__
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
Jeff Bohmer
VisionLink, Inc.
_________________________________
303.402.0170
www.visionlink.org
_________________________________
People. Tools. Change. Community.
__________________________________________________________________
__ ___
_
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
Robert Garcia
BigHead Technology
2781 N Carlmont Pl
Simi Valley, CA 93065
Phone 805.522.8577
http://www.bighead.net/
http://www.theradmac.com/
[EMAIL PROTECTED]
___________________________________________________________________
__ ___
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--------------------------
bohmer
Mobility Productions
http://mobility303.com
phone: 303.516.9544
--------------------------
____________________________________________________________________
__ __
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
Robert Garcia
BigHead Technology
2781 N Carlmont Pl
Simi Valley, CA 93065
Phone 805.522.8577
http://www.bighead.net/
http://www.theradmac.com/
[EMAIL PROTECTED]
_____________________________________________________________________
__ _
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
____________________________________________________________________
Jason Pamental, President
[EMAIL PROTECTED]
Bathysphere Digital Media Services, Inc.
http://bathyspheredms.com
____________________________________________________________________
Tel: 401.490.6830 Fax: 401.490.6831
________________________________________
A North American Distributor for Witango (http://www.witango.us)
Rapid Web Application Development - XML Execution Engine
______________________________________________________________________
__
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
_______________________________________________________________________
_
TO UNSUBSCRIBE: send a plain text/US ASCII email to
[EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
--
Robert Garcia
BigHead Technology
2781 N Carlmont Pl
Simi Valley, CA 93065
Phone 805.522.8577
http://www.bighead.net/
http://www.theradmac.com/
[EMAIL PROTECTED]
________________________________________________________________________
TO UNSUBSCRIBE: send a plain text/US ASCII email to [EMAIL PROTECTED]
with unsubscribe witango-talk in the message body
