Hands-on session with Spark and Telemetry data at MozLandia
You all know Telemetry, our Firefox data collection system that provides us with real-world data about performance, hardware, usage and customizations. Spark is an open-source in-memory data analytics cluster computing framework originally developed in the AMPLab at UC Berkeley. In contrast to Hadoop's two-stage disk-based MapReduce paradigm, Spark's in-memory primitives provide performance up to 100 times faster which allow to interactively run distributed operations from a shell and analyze millions of Telemetry submissions in a matter of seconds; a great fit for exploratory data analyses. I will briefly go over the data layout of Telemetry and how Spark works under the hood. Finally, I will jump in a hands-on interactive analysis session with real data. No prerequisites are required in terms of Telemetry, Spark or distributed computing. This is a great opportunity to learn how to quickly extract actionable insight from our data. Time: Friday, December 5, 2014, 4:00:00 PM - 5:30:00 PM GMT -05:00 Location: Belmont Room, Marriott Waterfront 2nd, Mozlandia ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: power use on Yosemite
On Saturday, November 1, 2014 7:48:11 PM UTC, Andreas Gal wrote: > I am using Nightly on Yosemite and power use is pretty atrocious. The battery > menu tags Firefox Nightly as a significant battery hog, and I can confirm > this from the user experience perspective as well. My battery time is a > fraction of using Chrome for the same tasks. > > Not every kind of content seems to trigger this behavior, but Google Sheets > in Firefox seems to be a pretty reliable way to drain my battery quickly. Andreas, I used energia (https://github.com/mozilla/energia) to quickly compare the power usage of Nightly vs Chrome on Google Sheets (https://docs.google.com/spreadsheet/ccc?key=0ArS97F99-BEZdF81LXlRNnRJTWphQ3pZcTYxT3NSRXc#gid=0 and other similar sheets) and even though the usage of FF is greater than Chrome, it's not dramatically so, i.e.: Chrome: 2.8W +- 0.03 (confidence interval of the mean) Nightly: 3.1 +- 0.22 The numbers comprehend CPU and GPU usage. Do you have by chance a particular sheet that triggers the issue which you could share? I am going to dig deeper first thing in the morning; there is some suspicious high variance in the measurements for FF. On Monday, November 3, 2014 5:33:11 PM UTC, Martin Thomson wrote: > On 02/11/14 18:13, Mike Hoye wrote: > > "We found that blocking on a slow connection used consistently more > > power than idle. This could be solely due to network traffic, but > > network traffic was very periodic (1 packet in and 1 packet out per > > second), thus much of the work could likely be attributed to the > > animated spinner. The difference was not very large, but it was > > statistically significant." > > That suggests that it could be a result of us not having QUIC and Google > Sheets not being friendly to TCP-based usages. We tracked down many power bugs on Desktop earlier this year using our power profiler, see Bug 948528, Bug 962573, http://robertovitillo.com/2014/01/21/a-matter-of-energy/ and https://github.com/mozilla/energia. Many of those bugs are intimately related to the layout stack and still need to be addressed. David, do you have an ETA for some of them, e.g. Bug 962594? In particular Facebook, which practically appears in any top 10 list, had (has?) a serious power bug that caused FF to render a hidden spinning wheel. Because of this single bug any power benchmark performed by the press, which was likely going to be based on the top N most visited sites on the web, was likely going to be skewed significantly to our loss. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Telemetry alerts for histograms
On Wednesday, August 13, 2014 4:36:39 PM UTC+1, rvit...@mozilla.com wrote: > I requested a public list available over NNTP (see Bug 1053202, > mozilla.qa.telemetry-alerts). That should make everybody happy. The list has been renamed to mozilla.dev.telemetry-alerts. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Telemetry alerts for histograms
On Wednesday, August 13, 2014 8:06:08 PM UTC+1, Steve Fink wrote: > I would sort of like a pulse notification for these, but it probably > > wouldn't get enough usage yet for it to be worthwhile. (I'd feed it into > > an irc bot that notifies me + maybe a channel.) > > > > https://wiki.mozilla.org/Auto-tools/Projects/Pulse Very interesting; I will keep this in mind for future extensions. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Telemetry alerts for histograms
I requested a public list available over NNTP (see Bug 1053202, mozilla.qa.telemetry-alerts). That should make everybody happy. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Telemetry alerts for histograms
>From now on all alerts will be sent by default also to >telemetry-ale...@mozilla.com. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Expiration versions for Telemetry probes.
We are planning to forcefully set the expiration version of many outstanding probes, i.e. those that don't have yet an explicitly set expiration, to version 40. Please have a look at bug 1045108 and comment if you are against it or if you want us to don't set an expiration version for your probes. Thanks. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Telemetry alerts for histograms
On Wednesday, August 6, 2014 9:12:07 PM UTC+1, Jim (Ningjie) Chen wrote: > Looks great! Is the system specific to histograms or can it be adapted to > other telemetry measurements? > > > > Looking at the dashboard, we have a growing list of telemetry measurements > outside of histograms (hangs, experiments, I/O, power usage, etc.). It'd be > great if they can benefit from automatic monitoring as well. The current system is specifically tailored to histograms but we might do something similar for our dashboards once we complete our effort to unify the dataset format across all of them. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Telemetry alerts for histograms
On Thursday, August 7, 2014 9:19:09 AM UTC+1, Gian-Carlo Pascutto wrote: > Does this filter on any cpp_guards? No, it doesn't. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Telemetry alerts for histograms
On Wednesday, August 6, 2014 8:16:09 AM UTC+1, Gian-Carlo Pascutto wrote: > On 5/08/2014 22:09, Andrew McCreight wrote: > > How many alerts is this thing going to generate? Could you have a > > mailing list that just gets all of the telemetry alerts email and > > then people could subscribe to it if they wanted? (Technically, I > > suppose I could make a mailing list myself and land a patch to > > subscribe the list to every one of these...) I'm not sure I really > > want to have my email address distributed with every copy of the > > Firefox source code. :) > > Decoupling who gets the alerts from the Firefox source sounds like a > good idea. Agreed, I am going to create a new catch-all mailing list. Our original intent was actually for authors to use an e-mail alias like in Bug 1047568 or let them use their own mailing lists. But we can certainly send all alerts in one place too. Hopefully receiving all of them is not going to make people ignore the alerts altogether. On the other hand though the number of notifications per week should be mostly under 10, so maybe this isn't an issue after all. > 1) What if lots of people want to monitor a specific value? > 2) What if people stop working on Firefox? They have to "patch out" > their email? What about creating a mailing list just for the histograms you are interested in? > 3) What if the owner of a feature changes? How do the Firefox versions > with different Histogram.json interact with the telemetry alerts? Only the latest version of Histograms.json is taken into account. I am aware that there is some work involved to receive the notifications, unless you are OK with receiving all of them. There isn't any good way to discern automatically who has to receive which notification. I am open to suggestions. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Telemetry alerts for histograms
The dashboard of the alerting system is available at http://vitillo.github.io/cerberus/dashboard/. If you want to be notified when the distribution of your histogram changes significantly, please add your e-mail address to Histograms.json as documented in https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Adding_a_new_Telemetry_probe. Thanks, Roberto ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Deprecating localstore.rdf
On Monday, August 4, 2014 8:55:35 PM UTC+1, Benjamin Smedberg wrote: > This does involve a one-time import of localstore data into the new > format, correct? Correct. > > I'm happy that we are doing this. I *believe* that this may be the last > client of the RDF code in Firefox, which may allow us to remove RDF from > Firefox in a future release. Do you already have an addon validation > warning about addons using localstore? What do you mean exactly by addon validation warning? In an earlier version of the patch there was a warning for any user of the old localstore API but Neil suggested to remove it. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Telemetry alerts for histograms
We, as in the performance team, are currently working on an automatic alerting system for Telemetry's data. At the moment we can detect sensible changes in the distributions of the collected histograms and we would like to notify the authors of the histograms with an e-mail. The e-mail will include the name of the histogram, the date of the regression and a plot that shows two histograms: the reference and the regression distribution. The regression histogram shows the distribution of the histogram taken the day it changed, while the reference histogram shows how the distribution should look like based on the previous days. In the future the e-mail will include a link to Telemetry's dashboard which will likely show some more information. Since using 'hg blame' or a similar approach to pinpoint the author of the histogram doesn't always yield the right match (see Bug 1037494), we are likely going to ask the interested authors to manually add their e-mail address to toolkit/components/telemetry/Histograms.json. The question to the histogram authors now is if receiving alerts through e-mails is OK or they would prefer something else. Note that a great deal of care has been taken to ensure that there the number of false positives is very close to 0, i.e. if you receive an alert then the distribution has clearly changed and the effect size is big enough that it can be considered more than significant. So don't expect your mailbox to be flooded with useless alerts. Roberto ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Deprecating localstore.rdf
Localstore.rdf will soon be replaced with a json store (see Bug 559505). I am currently planning to leave the localstore.rdf implementation as it is and issue a warning when a client tries to access to it. This is needed as some add-ons seem still to rely on it. We could use some Telemetry probes to see effectively how many add-ons are still using the rdf store once the patch lands. Are there any objections or remarks to the deprecation of localstore.rdf? Roberto ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
