Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
On Tue, 12 Jul 2011 12:11:00 -0300 Gustavo Sverzut Barbieri barbi...@profusion.mobi said: It would be useful to have some references in SVN and also a tool to diff these files, producing a hint on what's different other than pure cmp a b: files differ at byte X. This is done by WebKit, for instance tom's elm_testsuite.sh is a REALLY simple hack :) its not going to do these things. i have a more complete test suit. i actually have the start of a pretty good test suite that has commands to create objects and manipulate them ans then save the current screenshot for later comparison. it actually lets u scripot what objects u will use, where to put them, how to modify them etc so we can test all sorts of combinations of objects and transitions of state. it's lurking locally. i sent tasn a copy a bit back. Another useful idea from WebKit is the virtual dump of the render tree. This walks the canvas, layers and objects (recursively) to produce a textual output, that is then able to be diff'ed. This is simpler to check and diff, also faster to run. Something like: ensure can do this. in theory. though it's languishing. we could roll this right into evas itself though. Evas #123 100x100: Layer #456: Rectangle #1 @0,0 100x100 # clipper=NONE pass_events=TRUE repeat_events=FALSE visible=TRUE Image #2 @10,10 50x50 # clipper=#1 pass_events=TRUE repeat_events=FALSE visible=TRUE visible=TRUE border=2,2,2,2 scale_hint=STATIC file=bla key=NONE Smart #3 @0,0 100x100 # clipper=NONE ... Rectangle #4 ... Text #5... The idea is to create virtual pointers (#1, #2...) so we can reference them and diff the results in different runs. -- Gustavo Sverzut Barbieri http://profusion.mobi embedded systems -- MSN: barbi...@gmail.com Skype: gsbarbieri Mobile: +55 (19) 9225-2202 -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel -- - Codito, ergo sum - I code, therefore I am -- The Rasterman (Carsten Haitzler)ras...@rasterman.com -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
On 27/07/11 14:02, Carsten Haitzler (The Rasterman) wrote: On Tue, 12 Jul 2011 12:11:00 -0300 Gustavo Sverzut Barbieri barbi...@profusion.mobi said: It would be useful to have some references in SVN and also a tool to diff these files, producing a hint on what's different other than pure cmp a b: files differ at byte X. This is done by WebKit, for instance tom's elm_testsuite.sh is a REALLY simple hack :) its not going to do these things. i have a more complete test suit. i actually have the start of a pretty good test suite that has commands to create objects and manipulate them ans then save the current screenshot for later comparison. it actually lets u scripot what objects u will use, where to put them, how to modify them etc so we can test all sorts of combinations of objects and transitions of state. it's lurking locally. i sent tasn a copy a bit back. I'm already using this elm_testsuite.sh full time with textblock. I have this tool I created (it's in svn) called tb_test that I just run through tsuite with various sample format .txts and to easily verify I don't break anything. This hack already saved me a lot of spanking and possibly hours of testing in the past couple of weeks. As for a more complex test suite, Aharon is working on it, it's getting really cool. It allows you to modify everything through code, take screenshots whenever you want, record mouse/keyboard input and etc. It will be really awesome. :) The advantage of this test suite in comparison to your awesome (really awesome) test suite, is that it's intended for higher level testing, i.e widgets and stuff that rely on ecore's main loop (yeah, it works nicely with a mainloop) and not just for canvas objects. -- Tom. -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
On Wed, 27 Jul 2011 14:20:42 +0300 Tom Hacohen tom.haco...@partner.samsung.com said: On 27/07/11 14:02, Carsten Haitzler (The Rasterman) wrote: On Tue, 12 Jul 2011 12:11:00 -0300 Gustavo Sverzut Barbieri barbi...@profusion.mobi said: It would be useful to have some references in SVN and also a tool to diff these files, producing a hint on what's different other than pure cmp a b: files differ at byte X. This is done by WebKit, for instance tom's elm_testsuite.sh is a REALLY simple hack :) its not going to do these things. i have a more complete test suit. i actually have the start of a pretty good test suite that has commands to create objects and manipulate them ans then save the current screenshot for later comparison. it actually lets u scripot what objects u will use, where to put them, how to modify them etc so we can test all sorts of combinations of objects and transitions of state. it's lurking locally. i sent tasn a copy a bit back. I'm already using this elm_testsuite.sh full time with textblock. I have this tool I created (it's in svn) called tb_test that I just run through tsuite with various sample format .txts and to easily verify I don't break anything. This hack already saved me a lot of spanking and possibly hours of testing in the past couple of weeks. As for a more complex test suite, Aharon is working on it, it's getting really cool. It allows you to modify everything through code, take screenshots whenever you want, record mouse/keyboard input and etc. It will be really awesome. :) The advantage of this test suite in comparison to your awesome (really awesome) test suite, is that it's intended for higher level testing, i.e widgets and stuff that rely on ecore's main loop (yeah, it works nicely with a mainloop) and not just for canvas objects. hmm sounds like he's doing something definitely higher up the stack. what i started with was something to test evas itself and test that all states of objects work and render right and that changing states - eg a move, a resize, enable.disable map, clip and unclip, and all sorts of permutations and combinations of these keep working correctly. i guess you could do a similar thing for elm and bind it into the mainloop - eg instead of an explicit save shot/image now you have a shot in 0.2 sec or wait until loop idle for 0.5 sec, then shot so u can wait until anim is done etc. you'd still need lots of command to create widgets and manipulate them of course. -- - Codito, ergo sum - I code, therefore I am -- The Rasterman (Carsten Haitzler)ras...@rasterman.com -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
On Tue, Jul 12, 2011 at 5:23 PM, Tom Hacohen tom.haco...@partner.samsung.com wrote: On 12/07/11 18:11, Gustavo Sverzut Barbieri wrote: It would be useful to have some references in SVN and also a tool to diff these files, producing a hint on what's different other than pure cmp a b: files differ at byte X. This is done by WebKit, for instance Well, it uses image magick to mark the different areas and also provides old and new images. I think it's more than enough to find out exactly what's wrong. Another useful idea from WebKit is the virtual dump of the render tree. This walks the canvas, layers and objects (recursively) to produce a textual output, that is then able to be diff'ed. This is simpler to check and diff, also faster to run. Something like: Evas #123 100x100: Layer #456: Rectangle #1 @0,0 100x100 # clipper=NONE pass_events=TRUE repeat_events=FALSE visible=TRUE Image #2 @10,10 50x50 # clipper=#1 pass_events=TRUE repeat_events=FALSE visible=TRUE visible=TRUE border=2,2,2,2 scale_hint=STATIC file=bla key=NONE Smart #3 @0,0 100x100 # clipper=NONE ... Rectangle #4 ... Text #5... The idea is to create virtual pointers (#1, #2...) so we can reference them and diff the results in different runs. That's a good idea, but a lot more annoying to implement than a simple script, and I'm not sure there's enough benefit to even bother. The test suite is aimed for developers that want to make sure they don't break something when they change it, and also for a testing server, and is intended to run along with all the other unit/regression/etc. tests. For the developers, saying there's a difference and marking it on a picture is usually more than enough, and for the testing server, this will just help us find issues and spot the problematic commits. But we will consider adding those features to the other testing tool, which will not be just a script launching binaries and will give us a lot more control. Maybe using ensure to do this could help to ease the amount of work required by this idea (that I really like to). -- Cedric BAIL -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
On 13/07/11 11:25, Cedric BAIL wrote: Maybe using ensure to do this could help to ease the amount of work required by this idea (that I really like to). Yeah, that's for sure. -- Tom. -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
Compared 111 with 2 diffs. Is it normal? Even there are no diff files. :p Let's run together for the best moment! -Regards, Hermet- -Original Message- From: Tom Hacohenlt;tom.haco...@partner.samsung.comgt; To: Cedric BAILlt;cedric.b...@free.frgt; Cc: Aharon Hillellt;a.hil...@partner.samsung.comgt;; enlightenment-devellt;enlightenment-devel@lists.sourceforge.netgt; Sent: 11-07-13(수) 17:41:07 Subject: Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.On 13/07/11 11:25, Cedric BAIL wrote: Maybe using ensure to do this could help to ease the amount of work required by this idea (that I really like to). Yeah, that's for sure. -- Tom. -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
On 13/07/11 12:10, ChunEon Park wrote: Compared 111 with 2 diffs. Is it normal? Even there are no diff files. :p It printed out the diffs while testing. ls *.comp* to see the files that were different. I guess you get Calendar_2 and Focus_2 different. Focus_2 is an animation, I already removed it from the tests, that's why it doesn't work. As for Calendar_2, it's date based, so you have to run them pretty close to each other in order to make sure you get the same images. Maybe we should add a non-date related calendar test... -- Tom. -- AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on Lean Startup Secrets Revealed. This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
On Tue, Jul 12, 2011 at 10:11 AM, Tom Hacohen tom.haco...@partner.samsung.com wrote: Dear all, I'm glad to introduce elm_tsuite, a pixel comparison test suite for EFL. Thanks should go to Aharon Hillel for writing this one. Commit: http://trac.enlightenment.org/e/changeset/61289 All the info below is also available in the README. This test suite should be used to make sure we don't break rendering of widgets when we do changes. For example, min size calculation breakage of elm_button will likely cause a resize in the test and thus an error will be detected by the test suite. This test suite is by no means limited to Elementary widgets only, more info in the commit log + README. There's currently a sample elementary.txt file there that includes a lot of tests for elementary widgets, just run: ./elm_tsuite.sh elementary.txt o Do changes, and then: ./elm_tsuite.sh elementary.txt to do the comparison. I hope to add some textblock tests soon as well. This is a fast and easy test suite that I hope everyone will start using. Aharon will soon start working on another test suite, that will let you write tests that directly call the screenshot calls and will possibly let you record user interaction. This will hopefully let us do even better testing to elementary widgets (or any other EFL application). If you have any questions, please let Aharon/me know. It would be useful to have some references in SVN and also a tool to diff these files, producing a hint on what's different other than pure cmp a b: files differ at byte X. This is done by WebKit, for instance Another useful idea from WebKit is the virtual dump of the render tree. This walks the canvas, layers and objects (recursively) to produce a textual output, that is then able to be diff'ed. This is simpler to check and diff, also faster to run. Something like: Evas #123 100x100: Layer #456: Rectangle #1 @0,0 100x100 # clipper=NONE pass_events=TRUE repeat_events=FALSE visible=TRUE Image #2 @10,10 50x50 # clipper=#1 pass_events=TRUE repeat_events=FALSE visible=TRUE visible=TRUE border=2,2,2,2 scale_hint=STATIC file=bla key=NONE Smart #3 @0,0 100x100 # clipper=NONE ... Rectangle #4 ... Text #5... The idea is to create virtual pointers (#1, #2...) so we can reference them and diff the results in different runs. -- Gustavo Sverzut Barbieri http://profusion.mobi embedded systems -- MSN: barbi...@gmail.com Skype: gsbarbieri Mobile: +55 (19) 9225-2202 -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
Re: [E-devel] Introducing: elm_tsuite - a pixel comparison test suite for EFL.
On 12/07/11 18:11, Gustavo Sverzut Barbieri wrote: It would be useful to have some references in SVN and also a tool to diff these files, producing a hint on what's different other than pure cmp a b: files differ at byte X. This is done by WebKit, for instance Well, it uses image magick to mark the different areas and also provides old and new images. I think it's more than enough to find out exactly what's wrong. Another useful idea from WebKit is the virtual dump of the render tree. This walks the canvas, layers and objects (recursively) to produce a textual output, that is then able to be diff'ed. This is simpler to check and diff, also faster to run. Something like: Evas #123 100x100: Layer #456: Rectangle #1 @0,0 100x100 # clipper=NONE pass_events=TRUE repeat_events=FALSE visible=TRUE Image #2 @10,10 50x50 # clipper=#1 pass_events=TRUE repeat_events=FALSE visible=TRUE visible=TRUE border=2,2,2,2 scale_hint=STATIC file=bla key=NONE Smart #3 @0,0 100x100 # clipper=NONE ... Rectangle #4 ... Text #5... The idea is to create virtual pointers (#1, #2...) so we can reference them and diff the results in different runs. That's a good idea, but a lot more annoying to implement than a simple script, and I'm not sure there's enough benefit to even bother. The test suite is aimed for developers that want to make sure they don't break something when they change it, and also for a testing server, and is intended to run along with all the other unit/regression/etc. tests. For the developers, saying there's a difference and marking it on a picture is usually more than enough, and for the testing server, this will just help us find issues and spot the problematic commits. But we will consider adding those features to the other testing tool, which will not be just a script launching binaries and will give us a lot more control. -- Tom. -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2 ___ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
