[Sikuli-driver] [Bug 695616] Re: X 1.0rc1 - Inconsistency in text recognition and matching, especially with integers-as-text!
** Changed in: sikuli Status: New = In Progress ** Changed in: sikuli Importance: Undecided = Low ** Changed in: sikuli Assignee: (unassigned) = RaiMan (raimund-hocke) ** Tags removed: exact match recognition text+integer ** Tags added: fkt-text -- You received this bug notification because you are a member of Sikuli Drivers, which is subscribed to Sikuli. https://bugs.launchpad.net/bugs/695616 Title: X 1.0rc1 - Inconsistency in text recognition and matching, especially with integers-as-text! Status in Sikuli: In Progress Bug description: I have noticed inconsistency in the Sikuli X text recognition and matching, especially when text on the screen/region are integers. Below are the steps to reproduce the problem scenario: Step1: In your web browser open 'TextOnTheScreen.png' from the following web-link (http://qoydkw.bay.livefilestore.com /y1pSr0oDju9ndvMzJJfo_qWD2qEvMdDDwoKquMNWWkUYAxgrsVcXmETY8yW_LqIieOKXgwlTt8yxe8eQDY- SCySCFljo59mSSIC/TextOnTheScreen.png?psid=1) Step2: To the below sikuli script, provide region coordinates of the 'TextOnTheScreen.png' image which is on your web browser. Note! region coordinates must be significantly bigger because you will zoom-in zoom-out of the visible content on your browser in-order to carry-out multiple text recognition tests. r = Region(“TextOnTheScreen”) t1 = 534438 t2 = 534177 t3 = 4438 t = t2 m = r.exists(t,0) if m: if m.text() == t: popup(Exact match) else: popup(m.nearby().text()) else: popup(Nothing found at all!) Step3(TestCase1): Now, run the above sikuli script with t = t2. You might notice one or more of the following: - Integers such as 7 8 are incorrectly recognised - All the integers-as-text are recognised correctly - Only few integers are recognised correctly Questions: - Does the text recognition depend on how much OCR gets trained? - Does the font-type font-size matter? - Should there be a mechanism for validating an exact-match for text recognition? Step4(TestCase2): Now, back to your browser where the 'TextOnTheScreen.png' image is visible. Try to zoom-out (ctrl-) the image and run the above sikuli script with t = t2. You should now notice variation in the text recognition results. Step5(TestCase3): Repeat 'TestCase2' with different zoom-levels in your browser, you might get varied text recognisation results. Step6(TestCase4): Now, run the above sikuli script with t = t3. You might notice that integers-as-text are not recognised at all or with luck it might recognise some integers-as-text? Hope this provide some test-cases for troubleshooting inconsistency in text/integer recognition. Questions: - Should text and integer recognition be independent operations? Should there be a choice for the application developer if she/he wants to combine and/or isolate such operations where-seen-fit? Would such separation lead to reduced-OCR-complexity/accuracy/better-control over the OCR? - How would one reliably obtain an exact string match for e.g. with click(String) operation? To manage notifications about this bug go to: https://bugs.launchpad.net/sikuli/+bug/695616/+subscriptions ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
[Sikuli-driver] [Bug 695616] Re: X 1.0rc1 - Inconsistency in text recognition and matching, especially with integers-as-text!
It seems particularly vulnerable to integers-only, with no spaces e.g 00 1 002003004 is seen as 001 002003004 - almost perfect but 01002003004 is seen as '{ri[?I?)'?1?I?H?!?E! (same app, same font, same color) -- You received this bug notification because you are a member of Sikuli Drivers, which is subscribed to Sikuli. https://bugs.launchpad.net/bugs/695616 Title: X 1.0rc1 - Inconsistency in text recognition and matching, especially with integers-as-text! Status in Sikuli: New Bug description: I have noticed inconsistency in the Sikuli X text recognition and matching, especially when text on the screen/region are integers. Below are the steps to reproduce the problem scenario: Step1: In your web browser open 'TextOnTheScreen.png' from the following web-link (http://qoydkw.bay.livefilestore.com /y1pSr0oDju9ndvMzJJfo_qWD2qEvMdDDwoKquMNWWkUYAxgrsVcXmETY8yW_LqIieOKXgwlTt8yxe8eQDY- SCySCFljo59mSSIC/TextOnTheScreen.png?psid=1) Step2: To the below sikuli script, provide region coordinates of the 'TextOnTheScreen.png' image which is on your web browser. Note! region coordinates must be significantly bigger because you will zoom-in zoom-out of the visible content on your browser in-order to carry-out multiple text recognition tests. r = Region(“TextOnTheScreen”) t1 = 534438 t2 = 534177 t3 = 4438 t = t2 m = r.exists(t,0) if m: if m.text() == t: popup(Exact match) else: popup(m.nearby().text()) else: popup(Nothing found at all!) Step3(TestCase1): Now, run the above sikuli script with t = t2. You might notice one or more of the following: - Integers such as 7 8 are incorrectly recognised - All the integers-as-text are recognised correctly - Only few integers are recognised correctly Questions: - Does the text recognition depend on how much OCR gets trained? - Does the font-type font-size matter? - Should there be a mechanism for validating an exact-match for text recognition? Step4(TestCase2): Now, back to your browser where the 'TextOnTheScreen.png' image is visible. Try to zoom-out (ctrl-) the image and run the above sikuli script with t = t2. You should now notice variation in the text recognition results. Step5(TestCase3): Repeat 'TestCase2' with different zoom-levels in your browser, you might get varied text recognisation results. Step6(TestCase4): Now, run the above sikuli script with t = t3. You might notice that integers-as-text are not recognised at all or with luck it might recognise some integers-as-text? Hope this provide some test-cases for troubleshooting inconsistency in text/integer recognition. Questions: - Should text and integer recognition be independent operations? Should there be a choice for the application developer if she/he wants to combine and/or isolate such operations where-seen-fit? Would such separation lead to reduced-OCR-complexity/accuracy/better-control over the OCR? - How would one reliably obtain an exact string match for e.g. with click(String) operation? To manage notifications about this bug go to: https://bugs.launchpad.net/sikuli/+bug/695616/+subscriptions ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp
[Sikuli-driver] [Bug 695616] Re: X 1.0rc1 - Inconsistency in text recognition and matching, especially with integers-as-text!
** Summary changed: - Sikuli X 1.0rc1 - Inconsistency in text recognition and matching, especially with integers-as-text! + X 1.0rc1 - Inconsistency in text recognition and matching, especially with integers-as-text! -- You received this bug notification because you are a member of Sikuli Drivers, which is subscribed to Sikuli. https://bugs.launchpad.net/bugs/695616 Title: X 1.0rc1 - Inconsistency in text recognition and matching, especially with integers-as-text! Status in Sikuli: New Bug description: I have noticed inconsistency in the Sikuli X text recognition and matching, especially when text on the screen/region are integers. Below are the steps to reproduce the problem scenario: Step1: In your web browser open 'TextOnTheScreen.png' from the following web-link (http://qoydkw.bay.livefilestore.com /y1pSr0oDju9ndvMzJJfo_qWD2qEvMdDDwoKquMNWWkUYAxgrsVcXmETY8yW_LqIieOKXgwlTt8yxe8eQDY- SCySCFljo59mSSIC/TextOnTheScreen.png?psid=1) Step2: To the below sikuli script, provide region coordinates of the 'TextOnTheScreen.png' image which is on your web browser. Note! region coordinates must be significantly bigger because you will zoom-in zoom-out of the visible content on your browser in-order to carry-out multiple text recognition tests. r = Region(“TextOnTheScreen”) t1 = 534438 t2 = 534177 t3 = 4438 t = t2 m = r.exists(t,0) if m: if m.text() == t: popup(Exact match) else: popup(m.nearby().text()) else: popup(Nothing found at all!) Step3(TestCase1): Now, run the above sikuli script with t = t2. You might notice one or more of the following: - Integers such as 7 8 are incorrectly recognised - All the integers-as-text are recognised correctly - Only few integers are recognised correctly Questions: - Does the text recognition depend on how much OCR gets trained? - Does the font-type font-size matter? - Should there be a mechanism for validating an exact-match for text recognition? Step4(TestCase2): Now, back to your browser where the 'TextOnTheScreen.png' image is visible. Try to zoom-out (ctrl-) the image and run the above sikuli script with t = t2. You should now notice variation in the text recognition results. Step5(TestCase3): Repeat 'TestCase2' with different zoom-levels in your browser, you might get varied text recognisation results. Step6(TestCase4): Now, run the above sikuli script with t = t3. You might notice that integers-as-text are not recognised at all or with luck it might recognise some integers-as-text? Hope this provide some test-cases for troubleshooting inconsistency in text/integer recognition. Questions: - Should text and integer recognition be independent operations? Should there be a choice for the application developer if she/he wants to combine and/or isolate such operations where-seen-fit? Would such separation lead to reduced-OCR-complexity/accuracy/better-control over the OCR? - How would one reliably obtain an exact string match for e.g. with click(String) operation? ___ Mailing list: https://launchpad.net/~sikuli-driver Post to : sikuli-driver@lists.launchpad.net Unsubscribe : https://launchpad.net/~sikuli-driver More help : https://help.launchpad.net/ListHelp