(followups to: mozilla.dev.b2g please)
I recently released an experimental user interaction (touch) fuzzer for
Firefox OS, known as orangfuzz[1]. It is based on the Orangutan
framework[2] by wlach.
More details can be found in a Mozilla Security blogpost[3].
Currently it only works with a Unagi B2G test device - I tested on a
Geeksphone Keon but the Orangutan framework wasn't working as expected
there yet.
Some possible ideas/ways to move forward:
* Decide on a common prepopulate state - currently orangfuzz always
starts off on the homescreen, but ideally should be started from a
fixed state of Firefox with a fixed number of apps in a common position
(e.g. from reset) b2gpopulate[4] might help with this.
* Run the generated scripts with the long-running harness script[5] on
pandaboards running B2G and orangutan, possibly via mozpool.
* Find ways to detect crashes - should we monitor
"/data/b2g/mozilla/Crash\ Reports" for new crashes?
* Find a way to detect assertions - monitor logcat?
* Improve the reliability of reproducing testcases by another person -
what are factors involved in one person not reproducing the crash by
running the script on another similar device?
* Come up with a way to reduce testcases generated by the fuzzer
automatically, maybe using Lithium[6].
* Come up with an optimum number of steps (currently 10000) such that we
achieve a fair balance of simulating sufficient user actions, not
taking too long for reduction, etc.
* How about less-obvious bugs which do not manifest as crashes or
assertions, such as hangs / long gc pauses?
Also:
* There could also be an orangutan "action recorder" similar to
Mozmill's Record function, thanks to Arky for this idea.
* Result generation reports could go somewhere, e.g. list of top crashes
found, or assertions, or Gaia errors. Maybe a tbpl-waterfall-style page?
* We could try to run this on an emulator for those without a phone, but
we're not sure on the way forward here.
* Running on a special build of Gaia[7][8] with phone and messaging
capabilities disabled (not yet well tested) is recommended for automated
non-supervised runs, to avoid accidental dialing of emergency numbers.
More ideas/contributions welcome!
-Gary
[1] https://github.com/mozilla/orangfuzz
[2] https://github.com/wlach/orangutan
[3]
https://blog.mozilla.org/security/2013/04/17/orangfuzz-an-experimental-user-interaction-fuzzer-for-firefox-os/
[4] https://github.com/mozilla/b2gpopulate
[5] https://bugzilla.mozilla.org/show_bug.cgi?id=832328
[6]
http://www.squarefree.com/2007/09/15/introducing-lithium-a-testcase-reduction-tool/
[7] https://github.com/gregorwagner/gaia/tree/monkey
[8] https://github.com/nth10sd/no-phone-no-messaging-gaia/tree/monkey
_______________________________________________
dev-security mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-security
