What do we need to know to be able to answer "are we fast enough to ship?" and "can we brag about this?"

I was thinking of using measure.lzx (which is built on top of getTimer ()) to report times to instantiate views, time to init an empty canvas, time to add a view to a layout, that sort of thing. Then we'd compare results across multiple builds, varying the code but not the test machine, to give us trend information.

Let's talk about this in person tomorrow.

-ben

On Oct 17, 2006, at 9:13 PM, Jim Grandy wrote:

I'll risk being a wet blanket and note that, interesting as these numbers are from a runtime engineer's POV, they aren't too useful from a are-we-fast-enough-to-ship POV much less a can-we-brag-about- this POV. So they don't directly answer the proximate need, which is for a performance benchmark suite that helps us tell whether we're (a) fast enough, and (b) trending faster or slower.

jim

On Oct 17, 2006, at 7:45 PM, Henry Minsky wrote:

Yes we discovered that function calls were real fast in Firefox, not so fast in IE as I recall, and
slow in Flash 6,7,8

I did some simple tests andI measured a big speedup in Flash 9 (using Adobe's compiler), like
10x to 20x over flash 7.



On 10/17/06, Benjamin Shine <[EMAIL PROTECTED]> wrote: Tell me this isn't the most fascinating data you've seen in a long
time. The executive summary: function calls and array access are
faster in dhtml than in flash, on my particular setup.

Rather than constructing reports with copy and paste, I think I will
bastardize Henry's logger.jsp. Run tests, calculate results, post to
server via a jsp. Then in a separate step we can analyze the results.

http://localhost:8080/legals/test/performance/adding-elements-to-
array.lzx?lzr=swf7
iterations = 200
empty                         :     0.83us ±    0.02 [0.81..0.93]/100
Array.push : 3.51us ± 3.71 [2.50..27.40]/ 100
Array[Array.length] =         :     2.02us ±    0.12 [1.93..3.02]/100
Array[index] =                :     1.63us ±    0.06 [1.54..1.84]/100
Array[index] = (preallocated) :     1.67us ±    0.05 [1.59..1.82]/100
Object[key] = : 1.62us ± 0.88 [1.47..10.36]/ 100

http://localhost:8080/legals/test/performance/adding-elements-to-
array.lzx?lzr=dhtml
iterations = 200
empty : 0.65us ± 1.69 [0.00..5.00]/100
Array.push : 3.45us ± 2.32 [0.00..5.00]/100
Array[Array.length ] = : 2.20us ± 2.49 [0.00..5.00]/100
Array[index] = : 2.40us ± 2.51 [0.00..5.00]/100
Array[index] = (preallocated) : 2.00us ± 2.46 [0.00..5.00]/100
Object[key] = : 1.90us ± 2.44 [0.00..5.00]/100

http://localhost:8080/legals/test/performance/cost-of-a-function-
call.lzx?lzr=swf7
iterations = 200
empty                         :     1.06us ±    0.04 [1.05..1.49]/100
assignment                    :     2.06us ±    0.22 [1.77..2.77]/100
function call : 8.73us ± 5.37 [7.32..35.08]/ 100 function call with 1 param : 8.62us ± 0.48 [8.21..10.29]/ 100 function call with 2 params : 10.45us ± 0.35 [9.93..11.94]/ 100 function call with 3 params : 9.65us ± 0.41 [9.23..11.16]/ 100 function call with 4 params : 14.01us ± 21.89 [9.70..141.19]/100 method call : 11.64us ± 12.68 [8.78..137.09]/100 method call with 1 param : 9.38us ± 0.37 [9.04..12.05]/ 100 proto method call : 10.66us ± 0.26 [10.35..11.35]/100 proto method call with 1 param: 9.60us ± 0.22 [9.29..10.31]/ 100

http://localhost:8080/legals/test/performance/cost-of-a-function-
call.lzx?lzr=dhtml
iterations = 200
empty : 0.65us ± 1.69 [0.00..5.00]/100
assignment : 2.40us ± 2.51 [0.00..5.00]/100
function call : 2.65us ± 2.51 [0.00..5.00]/100
function call with 1 param : 2.45us ± 2.51 [0.00..5.00]/100
function call with 2 params : 3.05us ± 2.45 [0.00..5.00]/100
function call with 3 params : 2.55us ± 2.51 [0.00..5.00]/100
function call with 4 params : 2.70us ± 2.50 [0.00..5.00]/100
method call : 2.75us ± 2.50 [0.00..5.00]/100
method call with 1 param : 2.85us ± 2.49 [0.00..5.00]/100
proto method call : 2.75us ± 2.50 [0.00..5.00]/100
proto method call with 1 param: 3.15us ± 2.43 [0.00..5.00]/100


http://localhost:8080/legals/test/performance/iteration-forms.lzx?
lzr=swf7
iterations = 200
empty                         :     0.10us ±    0.03 [0.07..0.28]/100
for                           :     2.44us ±    0.23 [2.14..3.21]/100
for in : 2.06us ± 2.57 [1.62..25.35]/ 100
while                         :     2.14us ±    0.05 [2.11..2.44]/100

http://localhost:8080/legals/test/performance/iteration-forms.lzx?
lzr=dhtml
iterations = 200
empty : 0.05us ± 0.50 [0.00..5.00]/100
for : 1.95us ± 2.45 [0.00..5.00]/100
for in : 2.80us ± 2.49 [0.00..5.00 ]/100
while : 1.95us ± 2.45 [0.00..5.00]/100




On Oct 17, 2006, at 6:34 PM, Benjamin Shine wrote:

> I just ran across test/performance and components/utils/ performance.
> This
> http://localhost:8080/legals/test/performance/adding-elements-to-
> array.lzx?debug=true
>
> gives
>
> empty : 0.87us ± 0.02 [0.84..0.95]/100 > Array.push : 3.89us ± 1.39 [3.66..17.63]/100 > Array[Array.length] = : 3.47us ± 0.06 [3.40..3.70]/100 > Array[index] = : 2.41us ± 0.04 [2.35..2.53]/100 > Array[index] = (preallocated) : 2.46us ± 0.06 [2.38..2.80]/100 > Object[key] = : 2.83us ± 5.14 [2.26..53.75]/100
>
> Seems to work in dhtml, too. Darn it's nice -- any reason
> components/utils/performance/measure.lzx shouldn't be the basis for
> our updated performance benchmarks?
>
> We need a way to track the results in different versions & runtimes
> of course. Someone's going to say "how about the wiki?" in response
> to which I am going to shoot myself. XML checked into the source
> tree would work better, I think... and if Josh Crowley stays up
> late he could write a charting app to visualize the performance
> results. (Not that Josh Crowley would ever stay up late.)
>
> -ben
>
> On Oct 17, 2006, at 4:06 PM, Max Carlson wrote:
>
>> Yes it does.
>>
>> -Max
>>
>> Henry Minsky wrote:
>>> Does the getTimer() value that is sent in LzIdle events start
>>> from zero when the LFC is loaded?
>>> On 10/16/06, *Henry Minsky* < [EMAIL PROTECTED]
>>> <mailto: [EMAIL PROTECTED]>> wrote:
>>>     At the bug scrub we talked about getting some updated
>>> performance
>>>     benchmarks , are these assigned with some high priorty?
>>>     There's a number of proposed optimizations that are in JIRA
>>> which
>>> would be easier to attack if we had a simple set of metrics we >>> could run. A couple of issues involve app startup time, is that
>>>     adequately covered by the existing inittimer tag ?     --
>>> Henry Minsky
>>>     Software Architect
>>>     [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
>>> --
>>> Henry Minsky
>>> Software Architect
>>> [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
>
> Benjamin Shine
> Software Engineer, Open Laszlo / Laszlo Systems
> [EMAIL PROTECTED]
>
>
>
>

Benjamin Shine
Software Engineer, Open Laszlo / Laszlo Systems
[EMAIL PROTECTED]







--
Henry Minsky
Software Architect
[EMAIL PROTECTED]



Benjamin Shine
Software Engineer, Open Laszlo / Laszlo Systems
[EMAIL PROTECTED]




Reply via email to