Re: [amibroker] Re: FW: 64 bit Results

[Tomasz Janeczko]

 Hello,

I am wondering what kind of quantum physics calculations you guys are doing 
because it would require
some terribly complex stuff to go into kind of performance issues you are 
describing.

My only guess is that you are not using array operations.

Take this code for example:

|GetPerformanceCounter(1);

*for*( i = 0; i < 1000; i++ )
{
  x = ( *H* + *L* )/2;
}

*Title*=StrFormat("%12.0f", GetPerformanceCounter() );|

When run on i7 920 1000 iterations over 500000 (500K) bars take 100ms.
It means performing 1000 * 500000 additions and divisions in 0.1 sec.

This gives you tremendous 5000000000 additions and divisions (2 ops) per second.
So it is 10 Giga floating point operations per second in AFL on single core.

If your timing is 0.1 second it means that you are doing one billion operations
(if array processing is used) and it means that either you are solving rocket 
science problems
or your formulas are written improperly.

Best regards,
Tomasz Janeczko
amibroker.com

On 2010-08-03 09:40, Rob wrote:

Dennis,

I actually already use Herman's technique of running the AA periodically to 
calculate results that I put into static arrays for access by my other charts. 
However, for my purposes the static arrays only need to be calculated every 
minute. I didn't know you could go down to second resolutions in the AA 
periodic running option.

Very simply the only reason I use this technique is that it allows me to 
calculate much longer static arrays than the rest of my charts need from an 
indicator basis. i.e my drawing code in my charts only requires 40 bars of data 
+ the visible.... however, my AA code uses 6,000 historical bars and creates 
static arrays that my chart code than can access when I scroll backwards on the 
charts.

As you know, one of the limitations of AB is the fact that the maximum number 
of historical bars required by a piece of code is applied to ALL that AFL 
regardless if the rest of the code requires it or not. This is a way of getting 
round that limitation.

If I didn't use this technique my charts would be unusably slow.

--- In amibroker@yahoogroups.com, Dennis Brown<se...@...>  wrote:

Herman,

You bring up some interesting points.  However, if the AA hogs the machine 
during the execution, then I don't see any advantage over doing everything in 
the indicator script.  You still need to calculate new data when something has 
changed that will change the chart in a material way.

A similar thing could be done by having two charts.  One that is open and 
displaying the UI and charted data, and a second one on a top tab, that is 
executing and filling in static arrays and no plots.  The advantage would be if 
multiple charts each ran in their own processor (future feature?), so that a 
true parallel processing was taking place.  The way I do my parameters, any 
chart can have access to any other charts parameters window settings, just by 
knowing the other chart ID.  That way one parameter window can control many 
charts.  Now that I think of it, I don't really have to have my parameters tied 
to a chart ID at all (even though the parameter window is).  Their ID could be 
something completely different and unique that does not change.  Just a side 
effect of how I do things.

However, the important point is to make the script more asynchronous in that 
only data are recalculated when changed, no matter how often the chart is 
refreshed.  For instance many calculations can use the close of the last 
complete bar.  They do not change at every tick, just when a new bar is 
complete.  This will take re-architecting a lot of my code.  Not a small task 
(I have a lot of code), but a potentially rewarding one.

It would take considerable time and effort on my part, so I do not want to 
start in that direction without knowing if I was wasting my time by going in an 
incompatible direction to future AB development.

There is nothing worse than to spend months working to solve a problem that is 
made moot by the next release of AB.  The time could have been spent on 
something more productive.

I know what you mean about conditional code.  I have my Parameters UI set up so 
that the parameters statements are not even executed if the parameters window 
is not open, or a hierarchal section is not visible.  All my parameters are 
saved into static variables and picked up by the code when needed.  I can have 
a lot of code to make the UI very nice, but not cost any execution overhead 
when not needed to interact with the user.  I might have a thousand parameters 
in a single system without the UI becoming cluttered.

I don't write overly long loops.  I just have a lot of them.  I might be able 
to create more array logic in some cases.  I do a lot of array operations, but 
the tricky ones I put in loops.  I'll bet there are several that could be put 
into array logic with some concerted effort, but it is better for me to debug 
them as loops first to get the logic debugged.  I also have debugging tools 
that let me see the internal loop variables at any bar.  Then I have a quick 
check if my array logic is generating the right answer just by switching code 
on the fly to see if anything changes in the output.

BR,
Dennis


On Aug 2, 2010, at 7:11 PM, Herman wrote:

Dennis and Rob,

Dennis, your idea of creating a generic plotting template is very interesting. 
Perhaps it could call the original code normally and substitute static arrays 
when needed. Thanks for the idea. It would be the nearest we can come to 
simulating a FreezeDisplay().

If you use loops be sure that you "break" out of them on mouse click to get an 
immediate mouse response. When I break out of a loop I recall the previous array result. 
hmm, not sure now but it may be that i ran into a problem doing that... not sure now. 
You'll have to try.

I have a lot of conditional sections in my code. If() take almost no time to 
execute, especially in gfx button design this can save time.

Using switch() based state machines you have to careful to design them so that they 
"fall through" in one pass (execution) if sequential conditions are true. If 
you don't you might need several executions (refreshes) to reach the desired state. 
Sometimes omitting the 'break' from a case section can help.

My gfx GUI is extremely fast. I use an auto-hide feature so that buttons only 
show up if I move my cursor to the button area (Top of chart in my case). I use 
a floating menu (Order Bar) that only drops down menus when the cursor is on 
the header button. You can use the header for the most frequent choice and the 
drop-down sub menu for secondary choices. I only save states (persistent) on a 
mouse click, usually after you click the mouse a little more processing time 
doesn't matter. Such techniques get pretty close to using the Param window 
(about 3 times slower).

You may be able to reduce you indicator execution time by letting the AA 
(running at 1 sec intervals) perform complex calculations in the background. 
These calculations could include calculating plots, i.e., in your indicator you 
only plot the static array results from the AA. This no calculations in the 
indicator. The AA seems to run independent of the Indicator formulas. Perhaps 
someone can explain in technical terms how Indicator and AA task are delegated 
to the computer.

zero refresh time in preferences gives you the max refreshes per second - that 
is what I need to get responsive code.

Not sure what you (Rob) refer to with Timed refreshes, perhaps I expressed myself wrong. 
What I meant is timed full-executions so that data is only calculated, say, once a 
second. All other passes would access static results from these calculations. So if a 
chart refreshes 10/times per second, 9 out of ten would use static data and only one 
calculated data. Using the AA 10/10 would use static data. The "average" load 
on your system would be reduced. Partitioning your code more complex ways of 
interlacing/staggering of calculations data may be possible.

I am sorry, I am not a professional programmer :-) there are probably proper 
terms for all these techniques but I don't know them - I just make up terms as 
I go along.

Herman






Herman,

Now you are speaking my language.  I do save the last data or plot for some 
calculations in static arrays and then substitute the plot for the code when I 
am trying to speed up the UI for some operations.

My preferred technique to speed up the UI without doing a lot of extra code and 
static array memory, was to just reduce the number of bars with a 
SetBarsRequired() command at the end of the formula.  This was only partially 
successful though.  All my looping code will reduce the number of bars, but the 
fast AFL array operations do not.  They are stuck with the highest number of 
bars ever requested since formula startup.  I look forward to the day when I 
have control of reducing the number of fast AFL bars on the fly.  That is the 
easy way to to generally speed things up.  Then combined with judicious use of 
saving plots in static arrays can give the rest of the speedup for the type of 
custom UI work that we do.  I even considered if I should create a generic 
template for every module that generates plots to save the plots or restore 
then based on a static variable.

The problem with this is that it would slow down normal operations because of 
all the saving of static arrays.  I use a state machine that only saves the 
static arrays during the first pass of the sequence of a UI operation that is 
multi-pass and time sensitive, like dragging an element around the chart.

Many of my UI functions require multiple AFL passes to complete, and I have 
routinely tried to reduce refresh times, and reduce the number of formulas that 
execute to speed up the chart to make the UI operation faster.  Perhaps, I 
should consider killing everything other than price bar plots and Gfx plots 
when processing any multi-pass UI function.  I have a static variable called 
HurryUp that most of my functions or modules do something with to reduce 
computation time.  I really have to do some major speedup code before I can 
take advantage of faster than one second refreshes.  The 0.1 second refreshes 
will be very useful to me at that point.

BR,
Dennis

On Aug 2, 2010, at 3:48 PM, Herman wrote:



To give you software control the in-line RequestTimedRefresh() must have priority. I have 
never used the setting in Preferences so I am not speaking from experience here, I have 
it set to zero - I am mostly looking for faster execution, not slower/timed/paced 
executions. Remember that these settings determine the MINIMUM refresh interval, if you 
quotes come in more frequently your chart will refresh more frequently. 
Status("redrawaction") is true when a timed refresh takes place, false during 
other (RT quote, click, Param actions, etc)  refreshes.

I just see TJ's comment coming in: "These refreshes are independent. If you specify 
both you will get twice as much." I didn't know that.

I do speed up refreshes on-the-fly, you can do similarly that also between 
panes/charts. For example the code I am looking right now speeds up my chart 
refreshes when I move my mouse over the chart, or when it is over one of my gfx 
buttons:

PrevMX = Nz( StaticVarGet( "LastCursorXPosition" ) );
PrevMY = Nz( StaticVarGet( "LastCursorYPosition" ) );
MouseAction = CM = MX != PrevMX OR MY != PrevMY;
StaticVarSet( "LastCursorXPosition", MX );
StaticVarSet( "LastCursorYPosition", MY );

if( Nz(VarGet( "OnButton" ) ) OR MouseAction ) RefreshTime = 0.1;
else RefreshTime = 1;
RequestTimedRefresh( RefreshTime );

"OnButton" is a variable that is set when my cursor is over a gfx control 
button, regardless where it is on the screen.

To "skip" over panes/formulas while retaining the display is a little more 
difficult, it requires saving display arrays in static variables. There is no graphic 
FreezeDisplay() command. Somewhat slower than a FreezeDisplay() would work, if we had 
one, you can substitute static variables for the calculated arrays when you want to skip 
the formula. I don't remember having actually done this but I am thinking of applying 
this technique in a current application. It should work...

You can use Status("redrawaction") or the GetPerformanceCounter() timer to 
create your delays.

Using the same code in each pane should significantly reduce the work involved 
to try timing schemes, just one formula to change ;-)

btw, what is your formula execution time?

Doing a search in Help for refresh, timing, and such keywords will lead you to 
some distributed bits of information that can prove useful.

Best regards,
herman

At the moment I'm on a basic one second refresh across
all charts (set via preferences>  Intraday)...
All my charts use the same piece of AFL (insert linked
into each chart)... I like the same functionality in each chart.
I will have to think about adjusting the refresh times
across charts. Not sure how I'd work that right now... I'm
still struggling with RequestTimedRefresh() vs the
preferences>intraday setting... you must know Herman....
which one takes precedence...?

------------------------------------
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
Yahoo! Groups Links

------------------------------------

**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.

TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com

TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

Yahoo! Groups Links