Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 RaiMan posted a new comment: If a webpage is not ready instantly and completed visually overcome time in steps, you might get a found with low similarity. This can either be compensated by adding some waiting time before the search, or by creating a Pattern with the image and a similarity of 0.95, which would internally wait for screen content that satisfies the condition. I prefer the second solution. problematic code: click(button) if (exists(Pattern(image).similar(0.5)) solution 1: click(button) wait(2) if (exists(Pattern(image).similar(0.5)) solution 2: click(button) if (exists(Pattern(image).similar(0.95), 5) this would give the search max 5 seconds time to find with a high score, but the search would succeed already after 1 second if found with the given similarity. So this is more dynamic and leads to faster scripts in the average. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 RaiMan posted a new comment: @Justin I just made some quick tests (with 1.1.0) I always get score values 0.... if you send me the images/screenshots zipped together to my mail rmhdevelop---at---me.com, I can have a look at it. The code you use is not really needed. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Justin Veerman posted a new comment: Thanks for the reply, I understand now. I am having issues where I take a screenshot of the last screen of a test and try to run a new test to see if that screenshot is in my test. I am getting only 60% similarity though. For example, I am filling in login information for GMail. Goto google.com Click gmail Enter an email Click Next I take one screenshot of the middle(including Google, One Account. All of Google and the login form/buttons). I run a new test to see if this screenshot exists, and I get a 60% match. I thought it would be closer to 99%. Any ideas? I can paste code in Java if you'd want it. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Justin Veerman posted a new comment: Boris stated that 'The following two black images, for example, are matched with 0.982491' He also says: 'sometimes exceeding our default level of 0.98'. I am just wondering how Boris knows the exact similarity value if the test outcomes(eg 0.982491). What would the code for seeing the similarity value look like? -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 RaiMan posted a new comment: @Jus Each image comparison / image search results in a so called Match, that contains the top left corner, where the found image is located in the base image or on the screen, besides some other information. On of these values is the score, a value between 0 and 1, denoting the probability, that the match is the searched image. 0 means no similarity at all, whereas 1 means should be the image exactly. might be of interest: http://www.sikulix.com/stories/how-does-sikuli- find-images-on-the-screen in a script saying: m = find(someImage) print m.getScore() would reveal the resulting value of the match. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Justin Veerman posted a new comment: I think my issue may have been that I wasn't adding enough of a time delay in between the actions and the test. I have added a delay and I am getting 100% similarity now. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 RaiMan posted a new comment: @Justin I do not understand you question in this context. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Justin Veerman posted a new comment: How do you display the actual similarity between your two images? -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
[Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
New question #256236 on Sikuli: https://answers.launchpad.net/sikuli/+question/256236 Hello, We are using SikuliX-1.1.0-Beta4 for automated GUI testing and we are very happy with its capabilities and features. We are using a default similarity level of 0.98, which is high enough to detect subtle gui differences. However, we've noticed, that when the compared images are black, there can be very big visual differences (as percent of the image) and still they are matched with very high similarity - sometimes exceeding our default level of 0.98! The following two black images, for example, are matched with 0.982491: http://imgur.com/a/IjsDk The following two gray images are not matched with 0.961927: http://imgur.com/a/9Yd9z Is this expected behaviour? If so, do you have any idea how we can work-around this? -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Status: Open = Answered RaiMan proposed the following answer: Setting the default similarity to 0.98 is high risk. You have to have very good snapshots: as little surrounding even background as possible and hence concentrating on the visual aspects making the difference. Version 1.2 will have a feature to optimize snapshots in this direction. But till then, you have to try to get optimal images yourself. This is surely the problem with your gray image. So with the current implementation a practical value to be used for default similarity is 0.95, when your workflow expects exact matches and you allow your snapshots to be slightly less than optimal. Especially black and white areas as background in the current implementation still may lead to false positives, if the number of background pixels exceed 10 to 20% of the total amount of pixels in the image. Here currently the only possibility is to concentrate on the non-black/non-white areas. in your case the areas where the symbols appear. Version 1.2 will have better support for these situations. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Boris Lazarov posted a new comment: Thanks for the quick response RaiMan - to clarify, that our current issue is not false negatives, as we have an easy mechanism to update our reference imageswhen there is a change, but the false positives, which are much harder to detect - requiring manual analysis of the test logs. Could your be more specific in your remark about version 1.2 - is there going to be a change in the general image comparison algorithm, because if I understand correctly - it is based on OpenCV and is out of the scope of Sikuli development? Our current best solution is to put manually even higher similarity on reference images that have high black (also white it seems) content. Forgot to mention, that we are using the Java API and not the IDE. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 RaiMan proposed the following answer: OpenCV features used in Sikuli: currently only matchTemplate is used (+ some Mat ops to adjust or detect image contents). This wil stay the same of course in SikuliX 1.2, but will be enhanced in various aspects like ignoring transparent areas (or even colored spaces), allow masking, use edge detection for special purposes like buttons … … and optionally try to evaluate the remaining differences in situations with a score less than some max (standard 0. which is taken as exact) The other point is to support optional automatic optimization of screenshots in the above mentioned sense. Again: the more mainly even background the reference image (the image to be searched) has towards the edges, the lower the resulting match score will be (variation about 0.05). example: the arrow in the first sample will match with 1.0 if it has less than 20 % black surrounding pixels. The more black pixels around, the lower the match score (it might drop below 0.95 in some situations). This is simply due to the statistical functions that are internally used in OpenCV's matchTemplate. So currently the strategy should be: 1. have optimal shots, that get match scores above 0.98 2. restrict the region to the area, where the match should be 3. adjust the similarity used for searching to a higher value or even 0.99/exact() by using Patterns (not default similarity) -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Status: Answered = Solved Boris Lazarov confirmed that the question is solved: Thanks RaiMan, that solved my question. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Boris Lazarov posted a new comment: Just to add, that it would be really nice if the Pattern java classs can be extended with more advanced control of the used OpenCV matching method and its parameters. Do you think it is feasible and usefull? -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 RaiMan posted a new comment: --- Do you think it is feasible and useful? Might be. I will find out and add respective features during the next months. Thanks. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 RaiMan posted a new comment: as you might have seen already: made it a request bug -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 Tak Eda posted a new comment: Sikuli has issues with images that have non-white background color Just as you have observed, Sikuli will actually match completely different foreground objects that happen to have the same background, even if after removing the background color, they could be completely different images: http://imgur.com/a/IjsDk @RaiMan, how much work will be required to implement a find() function that implements background removal as in the following article: http://answers.opencv.org/question/24463/how-to-remove-black-background- from-grabcut-output/ Also, if the Sikuli script designed was to pass a sample of the background to the find() function, will it make things easier? @Boris, I have created a new question (https://answers.launchpad.net/sikuli/+question/256255) from yours to better focus on this. Let's take the discussion forward there. This is different from the one you already have here at https://answers.launchpad.net/sikuli/+question/256236 Feel free to refer to this if you create a request bug. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
Re: [Sikuli-driver] [Question #256236]: Differing images matched with very high similarity
Question #256236 on Sikuli changed: https://answers.launchpad.net/sikuli/+question/256236 RaiMan posted a new comment: @TakEda Thanks for the pointer. The basic problems with the current implementation of the image search: - can only be a rectangle - all pixels count (transparency is ignored) saying it other way round: you cannot use images for searching that are somehow masked to a visually non-rectangular shape or have some pixels (usually background to the edges) 100% transparent. This option is on the list for version 1.2 with different solution paths: - somehow take the transparent pixels into account during matching (as an option, makes search last longer) - use edge detection to find a visual object while ignoring some inner parts of the image (e.g. buttons with different content) So feel free to post more ideas on this. I will dive into this area during the next 3 months. I will add this comment to your request answer as well. -- You received this question notification because you are a member of Sikuli Drivers, which is an answer contact for Sikuli. ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
