Witango-Talk: Very puzzling...
I have a situation here that I find very puzzling. So I am going back to the experts for advice. We have a few applications that, by their nature, perform thousands of requests against a SQL database and return thousands of records. I am quite sure that the coding of the application isn't as good as it could be and we will look into rewriting it. Here's our issue: This program, from start to the time it displays the returning page will take about 3 minutes. (stop groaning). We are testing this same application on a new box containing 8 dual core cpu's. I would expect one of those cpus to be pegged while Witango rips through these thousands of requests. Just the opposite. The cpus indicate hardly anything happening on the box. The same is true of SQL server. Even though I am literally throwing thousands of requests at SQL 2005 I'm seeing virtually no cpu activity on that box either. It looks almost like Witango is waiting for something, processes a request, waits a bit more, processes the next request, waits again, etc. The environment is Windows Server 2003. One Box is running Witango 5.5 with 8 dual core processors and the other box is Windows Server 2003 running SQL Server 2005. Yes, this program is coded and does some looping but I would still expect it to run one heck of a lot faster than it is. Any suggestions? Gene Wolf Supervisor, Business Systems DRS Sensors Targeting Systems-Optronics 2330 Commerce Park Drive NE Palm Bay, Florida 32905 Phone: 321-309-0685 321-309-0202 (fax) Dictionary.com Word of the Day http://dictionary.reference.com/wordoftheday/ This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message. This (document/presentation) may contain technical data as defined in the International Traffic In Arms Regulations (ITAR) 22 CFR 120.10. Export of this material is restricted by the Arms Export Control Act (22 U.S.C. 2751 et seq.) and may not be exported to foreign persons without prior written approval from the U.S. Department of State. TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf
RE: Witango-Talk: Very puzzling...
Hi Gene, You have at least 2 options to help find your bottleneck. 1. Use the debug output from the TAF to check the timing between Actions (the numbers inside the square brackets [001]). The numbers may not be entirely accurate, but they are great at pointing at which database or logic Actions are more time consuming than others. 2. Try running SQL Profiler on the database server. This gives you a view of all the queries executing against a given database (you can even filter the flood of activity). Turn Profiler on and off before and after running one of your long queries, then analyze the statistics to see which queries are slower than others - and the space of time between each query. You can also copy and paste the complete SQL text from Profiler into SQL Management Studio and do further analyse of the queries. Hope this helps. Scott, -Original Message- From: Wolf, Gene [EMAIL PROTECTED] Sent: Thu, June 28, 2007 6:50 am To: witango-talk@witango.com Subject: Witango-Talk: Very puzzling... I have a situation here that I find very puzzling. So I am going back to the experts for advice. We have a few applications that, by their nature, perform thousands of requests against a SQL database and return thousands of records. I am quite sure that the coding of the application isn't as good as it could be and we will look into rewriting it. Here's our issue: This program, from start to the time it displays the returning page will take about 3 minutes. (stop groaning). We are testing this same application on a new box containing 8 dual core cpu's. I would expect one of those cpus to be pegged while Witango rips through these thousands of requests. Just the opposite. The cpus indicate hardly anything happening on the box. The same is true of SQL server. Even though I am literally throwing thousands of requests at SQL 2005 I'm seeing virtually no cpu activity on that box either. It looks almost like Witango is waiting for something, processes a request, waits a bit more, processes the next request, waits again, etc. The environment is Windows Server 2003. One Box is running Witango 5.5 with 8 dual core processors and the other box is Windows Server 2003 running SQL Server 2005. Yes, this program is coded and does some looping but I would still expect it to run one heck of a lot faster than it is. Any suggestions? Gene Wolf Supervisor, Business Systems DRS Sensors Targeting Systems-Optronics 2330 Commerce Park Drive NE Palm Bay, Florida 32905 Phone: 321-309-0685 321-309-0202 (fax) Dictionary.com Word of the Day http://dictionary.reference.com/wordoftheday/ This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message. This (document/presentation) may contain technical data as defined in the International Traffic In Arms Regulations (ITAR) 22 CFR 120.10. Export of this material is restricted by the Arms Export Control Act (22 U.S.C. 2751 et seq.) and may not be exported to foreign persons without prior written approval from the U.S. Department of State. TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf
Re: Witango-Talk: Very puzzling...
Are you doing ANY processing in those loops? if you are doing @rows to just display, it should be fairly fast, but if you are are doing any kind of processing or calculations in those loops, witango will just choke. Look in the past some tests I did with randomizing large arrays in witango. I do not use SQL 2005, but I can say that using SQL 2000 I did also see the database seem to not be working as hard, or really taking advantage of the cpu. Moving to mysql and linux totally solved that issue. Much faster, much more efficient IMHO. One of my biggest clients who is still on witango, php port not done yet, does about 1 million dollars a day in ecommerce through our application. We have had to bleed every bit of efficiency from the app and DB. The DBs were setup by MSSQL DBA that were paid a lot of money. They didn't believe me for a long time, but end of last year, they finally let me take them to Linux MYSQL, and it was a HUGE boost in performance. NOTE: I am very good at tuning MYSQL dbs, and I am sure this could not be as big of a boost if the mysql db was not tuned properly. Also, the mysql db is completely innodb, not myisam, in case you are wondering. To solve large returnsets with processing, the only way to do it is NOT to use witango. Write an external action in a compiled format, I use realbasic, to do this. Don't use javabean, because witango also has a huge bottleneck of reading in lots of data through the javabean interface. HTH. -- Robert Garcia President - BigHead Technology VP Application Development - eventpix.com 13653 West Park Dr Magalia, Ca 95954 ph: 530.645.4040 x222 fax: 530.645.4040 [EMAIL PROTECTED] - [EMAIL PROTECTED] http://bighead.net/ - http://eventpix.com/ On Jun 28, 2007, at 5:50 AM, Wolf, Gene wrote: I have a situation here that I find very puzzling. So I am going back to the experts for advice. We have a few applications that, by their nature, perform thousands of requests against a SQL database and return thousands of records. I am quite sure that the coding of the application isn't as good as it could be and we will look into rewriting it. Here's our issue: This program, from start to the time it displays the returning page will take about 3 minutes. (stop groaning). We are testing this same application on a new box containing 8 dual core cpu's. I would expect one of those cpus to be pegged while Witango rips through these thousands of requests. Just the opposite. The cpus indicate hardly anything happening on the box. The same is true of SQL server. Even though I am literally throwing thousands of requests at SQL 2005 I'm seeing virtually no cpu activity on that box either. It looks almost like Witango is waiting for something, processes a request, waits a bit more, processes the next request, waits again, etc. The environment is Windows Server 2003. One Box is running Witango 5.5 with 8 dual core processors and the other box is Windows Server 2003 running SQL Server 2005. Yes, this program is coded and does some looping but I would still expect it to run one heck of a lot faster than it is. Any suggestions? Gene Wolf Supervisor, Business Systems DRS Sensors Targeting Systems-Optronics 2330 Commerce Park Drive NE Palm Bay, Florida 32905 Phone: 321-309-0685 321-309-0202 (fax) Dictionary.com Word of the Day http://dictionary.reference.com/wordoftheday/ This e-mail, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient. Any review, use, distribution, or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive information for the intended recipient), please contact the sender by reply e-mail and delete all copies of this message. This (document/presentation) may contain technical data as defined in the International Traffic In Arms Regulations (ITAR) 22 CFR 120.10. Export of this material is restricted by the Arms Export Control Act (22 U.S.C. 2751 et seq.) and may not be exported to foreign persons without prior written approval from the U.S. Department of State. __ __ TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf
RE: Witango-Talk: Very puzzling...
Hi Gene
To get started on creating the stored procedures you can right click on the
Search Action that you are using and then slect SQL Query.
Copy and paste that into your Direct to Database Action or into a SQL query
window to test the query.
Also when I want to use a Direct to Database Action rather than a stored
procedure I assign the SQL to a vaiable:
@ASSIGN NAME=sql_contactDD VALUE='SELECT
v1.id_de300orgdep,v1.de300name,v1.id_de450role,v1.de450name,v1.id_ct100cont,
v1.ct100nmlast,v1.ct100nmfirst,v1.id_ct125role,v1.ct100dob,v1.id_re100rq
FROM physio.v_ct100_de400 v1
WHERE (v1.id_de100org=@VAR v_Dept_de100 SCOPE=Request AND
v1.id_ct100cont=@VAR contactID SCOPE=Request)
ORDER BY 1 ASC,6 ASC' Scope=Request
___or a more comples sample__
@ASSIGN NAME=sql_contactDD VALUE='SELECT DISTINCT
v1.id_de300orgdep,v1.id_de450role,v1.id_ct100cont,v1.ct100nmlast,v1.ct100nmf
irst,v1.ct100dob,v1.de450name
FROM physio.v_ct100_de400 v1,physio.ct100 c2
WHERE (c2.id_ct100cont=v1.id_ct100cont)
@IF EXPR=(@ARG range=1)@! COMMENT=range = org or dept
AND (v1.id_de100org=@VAR v_Dept_de100 SCOPE=Request)
/@IF
@! COMMENT=if search by ORG then range=1 and is not = to2
BUT if search by dept then range=2 which is = to 2 and gretaer than 1
@IF EXPR=(@ARG range=2)@! COMMENT=range = more narrow by dept
AND (v1.id_de300orgdep=@VAR v_Dept_de300 SCOPE=Request)
/@IF
@! COMMENT=name,contactID,phone
@IF EXPR=('@LENGTH STR=@VAR sql_criteria SCOPE=Request'=1)
AND @VAR sql_criteria SCOPE=Request
/@IF
@! COMMENT=de450 is role
@IF EXPR=('@LENGTH STR=@VAR sql_criteria_de450
SCOPE=Request'=1)AND('@ARG searchde450'=1)
AND @VAR sql_criteria_de450 SCOPE=Request
/@IF
ORDER BY v1.ct100nmlast ASC,v1.ct100nmfirst ASC' Scope=Request
This way I can even LOOP thru results from previous arrays to creat the new
SQL!
@ASSIGN NAME=sql_getRE400_PA_group VALUE=SELECT id_pa100authztn,
MAX(re400startdate) AS re400startdate
FROM physio.re400
WHERE (id_rp350profileid IS NOT NULL) AND (id_ct100cont =
@@Request$v_pracid)
GROUP BY id_pa100authztn
HAVING (MAX(re400startdate) CONVERT(DATETIME, '@VAR
v_datestring_y_m_d SCOPE=Request 00:00:00', 102))
AND(
@ROWS START=1 STEP=1 STOP=@VAR nr_pa100id SCOPE=Request
ARRAY=a_pa100id
(id_pa100authztn = @COL NUM=1)
@IF EXPR='(@CURROW @VAR nr_pa100id SCOPE=Request)' TRUE='OR'
FALSE=''
/@ROWS
)
ORDER BY MAX(DISTINCT id_rp350profileid) DESC, MAX(DISTINCT re400startdate)
DESC SCOPE=Request
Then in the DDB just use the VAR: @VAR sql_contactDD SCOPE=Request
I have a lot of DDB like this on Microsoft SQL and the speeds run about 40
miliseconds to process.
I prefer to use stored procedures but sometimes cant beat a DDB.
Janet Case
_
From: Wolf, Gene [mailto:[EMAIL PROTECTED]
Sent: Thursday, June 28, 2007 8:50 AM
To: witango-talk@witango.com
Subject: Witango-Talk: Very puzzling...
I have a situation here that I find very puzzling. So I am going back to
the experts for advice.
We have a few applications that, by their nature, perform thousands of
requests against a SQL database and return thousands of records. I am quite
sure that the coding of the application isn't as good as it could be and we
will look into rewriting it. Here's our issue: This program, from start to
the time it displays the returning page will take about 3 minutes. (stop
groaning). We are testing this same application on a new box containing 8
dual core cpu's. I would expect one of those cpus to be pegged while Witango
rips through these thousands of requests. Just the opposite. The cpus
indicate hardly anything happening on the box. The same is true of SQL
server. Even though I am literally throwing thousands of requests at SQL
2005 I'm seeing virtually no cpu activity on that box either.
It looks almost like Witango is waiting for something, processes a
request, waits a bit more, processes the next request, waits again, etc. The
environment is Windows Server 2003. One Box is running Witango 5.5 with 8
dual core processors and the other box is Windows Server 2003 running SQL
Server 2005. Yes, this program is coded and does some looping but I would
still expect it to run one heck of a lot faster than it is.
Any suggestions?
Gene Wolf
Supervisor, Business Systems
DRS Sensors Targeting Systems-Optronics
2330 Commerce Park Drive NE
Palm Bay, Florida 32905
Phone: 321-309-0685
321-309-0202 (fax)
Dictionary.com Word of the Day
http://dictionary.reference.com/wordoftheday/
http://dictionary.reference.com/wordoftheday/
This e-mail, including any attached files, may contain confidential and
privileged information for the sole use of the intended recipient. Any
review, use, distribution, or disclosure by others is strictly prohibited.
If you are not the intended recipient (or authorized to receive information
for the intended recipient), please
Witango-Talk: Google site stats...
Hey list, Just wanted to let you guys know that Google is offering free site analytics (stats). http://www.google.com/analytics/ Might be useful for some of you! WebLogoWhite Rick Sanders President 902-401-7689 http://www.webenergy-sw.com www.webenergy-sw.com Shop online for computer systems, accessories, and much, much more! http://shop.webenergy-sw.com/ http://shop.webenergy-sw.com TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.tafattachment: image003.jpg
RE: Witango-Talk: Google site stats...
I've had that going for the past year and a half... _ From: Rick Sanders [mailto:[EMAIL PROTECTED] Sent: Thursday, June 28, 2007 1:39 PM To: witango-talk@witango.com Subject: Witango-Talk: Google site stats... Hey list, Just wanted to let you guys know that Google is offering free site analytics (stats). http://www.google.com/analytics/ Might be useful for some of you! WebLogoWhite Rick Sanders President 902-401-7689 http://www.webenergy-sw.com www.webenergy-sw.com Shop online for computer systems, accessories, and much, much more! http://shop.webenergy-sw.com/ http://shop.webenergy-sw.com TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.tafattachment: image003.jpg
Re: Witango-Talk: Google site stats...
just a caution about this category of service. It relies on a javascript that drops a 3rd party cookie in the visitors' browsers. If those browsers are configured to reject 3rd party cookies, then your data will be catywumpus, as each page visit will look like a new visitor. When using a similar service, it became apparent that a large portion of people have the block on 3rd party cookies, which then required that we look at data through that lens. On Jun 28, 2007, at 11:38 AM, Rick Sanders wrote: Hey list, Just wanted to let you guys know that Google is offering free site analytics (stats). http://www.google.com/analytics/ Might be useful for some of you! image003.jpg Rick Sanders President 902-401-7689 www.webenergy-sw.com Shop online for computer systems, accessories, and much, much more! http://shop.webenergy-sw.com __ __ TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf
Re: Witango-Talk: Google site stats...
You may care or not as to whether Google has access to your client traffic statistics. If your client does not want this - see if you can get yourself a pre-google copy of urchin - which is the software they bought to create Google analytics. Same service -except it runs off your own servers. Roland Dumas wrote: just a caution about this category of service. It relies on a javascript that drops a 3rd party cookie in the visitors' browsers. If those browsers are configured to reject 3rd party cookies, then your data will be catywumpus, as each page visit will look like a new visitor. When using a similar service, it became apparent that a large portion of people have the block on 3rd party cookies, which then required that we look at data through that lens. On Jun 28, 2007, at 11:38 AM, Rick Sanders wrote: Hey list, Just wanted to let you guys know that Google is offering free site analytics (stats). http://www.google.com/analytics/ Might be useful for some of you! * image003.jpg ** **Rick Sanders** *President 902-401-7689 www.webenergy-sw.com http://www.webenergy-sw.com * * *Shop online for computer systems, accessories, and much, much more! http://shop.webenergy-sw.com http://shop.webenergy-sw.com/* TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf
Re: Witango-Talk: Google site stats...
On Jun 28, 2007, at 2:36 PM, Roland Dumas wrote: just a caution about this category of service. It relies on a javascript that drops a 3rd party cookie in the visitors' browsers. Google says in the docs on the analytics services that they issue 1st party cookies for just this reason. Michael Heth ServerSmiths If those browsers are configured to reject 3rd party cookies, then your data will be catywumpus, as each page visit will look like a new visitor. When using a similar service, it became apparent that a large portion of people have the block on 3rd party cookies, which then required that we look at data through that lens. On Jun 28, 2007, at 11:38 AM, Rick Sanders wrote: Hey list, Just wanted to let you guys know that Google is offering free site analytics (stats). http://www.google.com/analytics/ Might be useful for some of you! image003.jpg Rick Sanders President 902-401-7689 www.webenergy-sw.com Shop online for computer systems, accessories, and much, much more! http://shop.webenergy-sw.com _ ___ TO UNSUBSCRIBE: Go to http://www.witango.com/developer/ maillist.taf __ __ TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf
Re: Witango-Talk: Google site stats...
cool. I'm in. On Jun 28, 2007, at 2:54 PM, Michael Heth wrote: On Jun 28, 2007, at 2:36 PM, Roland Dumas wrote: just a caution about this category of service. It relies on a javascript that drops a 3rd party cookie in the visitors' browsers. Google says in the docs on the analytics services that they issue 1st party cookies for just this reason. Michael Heth ServerSmiths If those browsers are configured to reject 3rd party cookies, then your data will be catywumpus, as each page visit will look like a new visitor. When using a similar service, it became apparent that a large portion of people have the block on 3rd party cookies, which then required that we look at data through that lens. On Jun 28, 2007, at 11:38 AM, Rick Sanders wrote: Hey list, Just wanted to let you guys know that Google is offering free site analytics (stats). http://www.google.com/analytics/ Might be useful for some of you! image003.jpg Rick Sanders President 902-401-7689 www.webenergy-sw.com Shop online for computer systems, accessories, and much, much more! http://shop.webenergy-sw.com TO UNSUBSCRIBE: Go to http://www.witango.com/developer/ maillist.taf _ ___ TO UNSUBSCRIBE: Go to http://www.witango.com/developer/ maillist.taf __ __ TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf
