Hi All,
I think updating only txt results isn't the best idea now. It can be very very
confusing if we leave broken pngs in the tree. And I can't see any pro to do it
...
Now Qt 4.8, Qt 5.0-WK1, Qt 5.0-WK2 (not QRawWebView, but QQuickWebView based
WTR) have
same results. (except only ~500 tests). (But unfortunately png results are
outdated,
and they changed with this testfont update. We don't run pixel tests, but pngs
are
very useful for gardening. Determining if a new result is correct or not is
impossible
if you can see only the rendertree dump. But it is quite simple if you see png
results.
Additionally we (Szeged guys) always update png results too when we update txt
results.
Updating txt or txt_and_png is only an option, it doesn't need additional
effort. Updating
only txt results needed extra effort, because you have to revert png changes
after you
updated results with rebaseline-server.
I think the new QRawWebView based WTR is an orthogonal problem. I can't see how
can
QRawWebView and QQuickWebView results are different? They should match if there
isn't
bug in one of them. Or in the case if different results is expected/acceptable,
we
have to maintain results for Qt 5.0 WK1 and Qt 5.0 WK2 too. So we disn't have
to rebase
anything, but adding qt-5.0-wk2 specific pngs if we added Qt 5.0 WK1 pngs now.
br,
Ossy
[email protected] írta:
From: [email protected] [[email protected]] on behalf of ext Balazs Kelemen
Are we sure we need pixel results at the moment? For Qt 4.8 it seems to
be unvaluable to me, I don't expect anybody want to do pixel testing
with 4.8. WebKit2 is another problem, because a reliable pixel testing
infrastructure is just under progression. Maybe the new mechanism with
window snapshots will produce slightly different results, in which case
we would have to rebase the results again. I propose not to consider
pixel results as a mandatory part of this rebasing process.
This makes sense to me.
Committing a massive amounts of PNGs should be done after we've decided what we want to do with them.
I propose we finish this rebase with text results only, then finish switching
to the window-snapshot mechanism for the pixel results, and meanwhile figure
out our pixel testing/gardening strategy - e.g. pick one bot (webkit2-linux)
that continuously runs pixel tests and keep its results current, continuously
gardening/skipping whatever gets broken to keep track of pixel regressions.
Once we have the strategy in place, the rebaseline text results from the current effort, and the window-snapshot mechanism, we should then rebaseline the PNGs.
No'am
_______________________________________________
webkit-qt mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo/webkit-qt
